INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.19.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50622 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.19.0.1:50626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36410 - "GET /wiki HTTP/1.0" 404 Not Found INFO: 172.19.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58938 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:33:06,297 - lava_callback - INFO - Received callback for node 69cb86a32a64fa9aff936f00 2026-03-31 08:33:06,305 - lava_callback - INFO - Starting async job submit for node 69cb86a32a64fa9aff936f00 INFO: 172.19.0.1:49410 - "POST /node/69cb86a32a64fa9aff936f00 HTTP/1.0" 202 Accepted 2026-03-31 08:33:06,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:06,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a32a64fa9aff936f00 HTTP/1.1" 200 None 2026-03-31 08:33:06,347 - lava_callback - DEBUG - Processing node 69cb86a32a64fa9aff936f00: baseline-x86 INFO: 172.19.0.1:49416 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:33:06,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:06,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:06,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f00/log.txt.gz 2026-03-31 08:33:06,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:06,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:06,977 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f00/lava-logs.yaml 2026-03-31 08:33:06,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:07,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:07,156 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f00/lava_callback.json.gz 2026-03-31 08:33:07,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:07,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a32a64fa9aff936f00 HTTP/1.1" 200 None 2026-03-31 08:33:07,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:07,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83602a64fa9aff936925 HTTP/1.1" 200 None 2026-03-31 08:33:07,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:07,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb86a32a64fa9aff936f00 HTTP/1.1" 200 None 2026-03-31 08:33:07,253 - lava_callback - INFO - Completed processing callback for node 69cb86a32a64fa9aff936f00 2026-03-31 08:33:07,368 - lava_callback - INFO - Received callback for node 69cb86a52a64fa9aff936f09 INFO: 172.19.0.1:49418 - "POST /node/69cb86a52a64fa9aff936f09 HTTP/1.0" 202 Accepted 2026-03-31 08:33:07,375 - lava_callback - INFO - Starting async job submit for node 69cb86a52a64fa9aff936f09 2026-03-31 08:33:07,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:07,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a52a64fa9aff936f09 HTTP/1.1" 200 None 2026-03-31 08:33:07,409 - lava_callback - DEBUG - Processing node 69cb86a52a64fa9aff936f09: baseline-x86 2026-03-31 08:33:07,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:07,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:07,923 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a52a64fa9aff936f09/log.txt.gz 2026-03-31 08:33:08,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:08,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:08,208 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a52a64fa9aff936f09/lava-logs.yaml 2026-03-31 08:33:08,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:08,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:08,366 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a52a64fa9aff936f09/lava_callback.json.gz 2026-03-31 08:33:08,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:08,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a52a64fa9aff936f09 HTTP/1.1" 200 None 2026-03-31 08:33:08,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:08,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83602a64fa9aff936925 HTTP/1.1" 200 None 2026-03-31 08:33:08,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:08,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb86a52a64fa9aff936f09 HTTP/1.1" 200 None 2026-03-31 08:33:08,449 - lava_callback - INFO - Completed processing callback for node 69cb86a52a64fa9aff936f09 INFO: 172.19.0.1:37394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50572 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:33:37,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:37,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:55450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43904 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:34:01,107 - lava_callback - INFO - Received callback for node 69cb86a32a64fa9aff936f03 INFO: 172.19.0.1:60388 - "POST /node/69cb86a32a64fa9aff936f03 HTTP/1.0" 202 Accepted 2026-03-31 08:34:01,113 - lava_callback - INFO - Starting async job submit for node 69cb86a32a64fa9aff936f03 2026-03-31 08:34:01,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:01,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a32a64fa9aff936f03 HTTP/1.1" 200 None 2026-03-31 08:34:01,148 - lava_callback - DEBUG - Processing node 69cb86a32a64fa9aff936f03: baseline-x86 2026-03-31 08:34:01,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:01,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:01,555 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f03/log.txt.gz 2026-03-31 08:34:01,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:01,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:01,930 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f03/lava-logs.yaml 2026-03-31 08:34:01,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:02,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:02,077 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f03/lava_callback.json.gz 2026-03-31 08:34:02,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:02,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a32a64fa9aff936f03 HTTP/1.1" 200 None 2026-03-31 08:34:02,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:02,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83602a64fa9aff936925 HTTP/1.1" 200 None 2026-03-31 08:34:02,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:02,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb86a32a64fa9aff936f03 HTTP/1.1" 200 None 2026-03-31 08:34:02,179 - lava_callback - INFO - Completed processing callback for node 69cb86a32a64fa9aff936f03 2026-03-31 08:34:04,446 - lava_callback - INFO - Received callback for node 69cb86a42a64fa9aff936f06 INFO: 172.19.0.1:60402 - "POST /node/69cb86a42a64fa9aff936f06 HTTP/1.0" 202 Accepted 2026-03-31 08:34:04,452 - lava_callback - INFO - Starting async job submit for node 69cb86a42a64fa9aff936f06 2026-03-31 08:34:04,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:04,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a42a64fa9aff936f06 HTTP/1.1" 200 None 2026-03-31 08:34:04,502 - lava_callback - DEBUG - Processing node 69cb86a42a64fa9aff936f06: baseline-x86 2026-03-31 08:34:04,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:04,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:04,868 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a42a64fa9aff936f06/log.txt.gz 2026-03-31 08:34:05,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:05,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:05,270 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a42a64fa9aff936f06/lava-logs.yaml 2026-03-31 08:34:05,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:05,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:05,450 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a42a64fa9aff936f06/lava_callback.json.gz 2026-03-31 08:34:05,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:05,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a42a64fa9aff936f06 HTTP/1.1" 200 None 2026-03-31 08:34:05,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:05,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83602a64fa9aff936925 HTTP/1.1" 200 None 2026-03-31 08:34:05,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:05,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb86a42a64fa9aff936f06 HTTP/1.1" 200 None 2026-03-31 08:34:05,531 - lava_callback - INFO - Completed processing callback for node 69cb86a42a64fa9aff936f06 INFO: 172.19.0.1:60412 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:34:07,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:07,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:32920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:32924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57536 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:36:46,096 - lava_callback - INFO - Received callback for node 69c8e49c6a4c026e163483b6 INFO: 172.19.0.1:33220 - "POST /node/69c8e49c6a4c026e163483b6 HTTP/1.0" 202 Accepted 2026-03-31 08:36:46,104 - lava_callback - INFO - Starting async job submit for node 69c8e49c6a4c026e163483b6 2026-03-31 08:36:46,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49c6a4c026e163483b6 HTTP/1.1" 200 None 2026-03-31 08:36:46,150 - lava_callback - DEBUG - Processing node 69c8e49c6a4c026e163483b6: baseline-arm-baylibre 2026-03-31 08:36:46,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e49c6a4c026e163483b6 2026-03-31 08:36:46,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49c6a4c026e163483b6 HTTP/1.1" 200 None 2026-03-31 08:36:46,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:36:46,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e49c6a4c026e163483b6 HTTP/1.1" 200 None 2026-03-31 08:36:46,274 - lava_callback - ERROR - Error processing callback for node 69c8e49c6a4c026e163483b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33240 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:36:46,764 - lava_callback - INFO - Received callback for node 69c8e49b6a4c026e163483b3 INFO: 172.19.0.1:33252 - "POST /node/69c8e49b6a4c026e163483b3 HTTP/1.0" 202 Accepted 2026-03-31 08:36:46,772 - lava_callback - INFO - Starting async job submit for node 69c8e49b6a4c026e163483b3 2026-03-31 08:36:46,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49b6a4c026e163483b3 HTTP/1.1" 200 None 2026-03-31 08:36:46,803 - lava_callback - DEBUG - Processing node 69c8e49b6a4c026e163483b3: baseline-arm-baylibre 2026-03-31 08:36:46,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e49b6a4c026e163483b3 2026-03-31 08:36:46,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49b6a4c026e163483b3 HTTP/1.1" 200 None 2026-03-31 08:36:46,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:36:46,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e49b6a4c026e163483b3 HTTP/1.1" 200 None 2026-03-31 08:36:46,889 - lava_callback - ERROR - Error processing callback for node 69c8e49b6a4c026e163483b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55646 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:37:06,055 - lava_callback - INFO - Received callback for node 69c8e49d6a4c026e163483b9 INFO: 172.19.0.1:60096 - "POST /node/69c8e49d6a4c026e163483b9 HTTP/1.0" 202 Accepted 2026-03-31 08:37:06,061 - lava_callback - INFO - Starting async job submit for node 69c8e49d6a4c026e163483b9 2026-03-31 08:37:06,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49d6a4c026e163483b9 HTTP/1.1" 200 None 2026-03-31 08:37:06,077 - lava_callback - DEBUG - Processing node 69c8e49d6a4c026e163483b9: baseline-arm-baylibre 2026-03-31 08:37:06,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e49d6a4c026e163483b9 2026-03-31 08:37:06,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49d6a4c026e163483b9 HTTP/1.1" 200 None 2026-03-31 08:37:06,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:06,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e49d6a4c026e163483b9 HTTP/1.1" 200 None 2026-03-31 08:37:06,153 - lava_callback - ERROR - Error processing callback for node 69c8e49d6a4c026e163483b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:06,246 - lava_callback - INFO - Received callback for node 69c8e4aa6a4c026e163483de INFO: 172.19.0.1:60106 - "POST /node/69c8e4aa6a4c026e163483de HTTP/1.0" 202 Accepted 2026-03-31 08:37:06,256 - lava_callback - INFO - Starting async job submit for node 69c8e4aa6a4c026e163483de 2026-03-31 08:37:06,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4aa6a4c026e163483de HTTP/1.1" 200 None 2026-03-31 08:37:06,314 - lava_callback - DEBUG - Processing node 69c8e4aa6a4c026e163483de: baseline-arm-clabbe 2026-03-31 08:37:06,315 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4aa6a4c026e163483de 2026-03-31 08:37:06,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4aa6a4c026e163483de HTTP/1.1" 200 None 2026-03-31 08:37:06,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:06,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4aa6a4c026e163483de HTTP/1.1" 200 None 2026-03-31 08:37:06,416 - lava_callback - ERROR - Error processing callback for node 69c8e4aa6a4c026e163483de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60120 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:37:06,731 - lava_callback - INFO - Received callback for node 69c8e49f6a4c026e163483bc INFO: 172.19.0.1:60134 - "POST /node/69c8e49f6a4c026e163483bc HTTP/1.0" 202 Accepted 2026-03-31 08:37:06,745 - lava_callback - INFO - Starting async job submit for node 69c8e49f6a4c026e163483bc 2026-03-31 08:37:06,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49f6a4c026e163483bc HTTP/1.1" 200 None 2026-03-31 08:37:06,779 - lava_callback - DEBUG - Processing node 69c8e49f6a4c026e163483bc: baseline-arm-baylibre 2026-03-31 08:37:06,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e49f6a4c026e163483bc 2026-03-31 08:37:06,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49f6a4c026e163483bc HTTP/1.1" 200 None 2026-03-31 08:37:06,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:06,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e49f6a4c026e163483bc HTTP/1.1" 200 None 2026-03-31 08:37:06,869 - lava_callback - ERROR - Error processing callback for node 69c8e49f6a4c026e163483bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:06,929 - lava_callback - INFO - Received callback for node 69c8e4ae6a4c026e16348405 INFO: 172.19.0.1:60138 - "POST /node/69c8e4ae6a4c026e16348405 HTTP/1.0" 202 Accepted 2026-03-31 08:37:06,942 - lava_callback - INFO - Starting async job submit for node 69c8e4ae6a4c026e16348405 2026-03-31 08:37:06,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4ae6a4c026e16348405 HTTP/1.1" 200 None 2026-03-31 08:37:06,993 - lava_callback - DEBUG - Processing node 69c8e4ae6a4c026e16348405: baseline-arm-clabbe 2026-03-31 08:37:06,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4ae6a4c026e16348405 2026-03-31 08:37:06,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4ae6a4c026e16348405 HTTP/1.1" 200 None 2026-03-31 08:37:07,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:07,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4ae6a4c026e16348405 HTTP/1.1" 200 None 2026-03-31 08:37:07,155 - lava_callback - ERROR - Error processing callback for node 69c8e4ae6a4c026e16348405: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:07,410 - lava_callback - INFO - Received callback for node 69c8e4a26a4c026e163483c7 INFO: 172.19.0.1:60150 - "POST /node/69c8e4a26a4c026e163483c7 HTTP/1.0" 202 Accepted 2026-03-31 08:37:07,419 - lava_callback - INFO - Starting async job submit for node 69c8e4a26a4c026e163483c7 2026-03-31 08:37:07,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4a26a4c026e163483c7 HTTP/1.1" 200 None 2026-03-31 08:37:07,446 - lava_callback - DEBUG - Processing node 69c8e4a26a4c026e163483c7: baseline-arm-baylibre 2026-03-31 08:37:07,447 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4a26a4c026e163483c7 2026-03-31 08:37:07,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4a26a4c026e163483c7 HTTP/1.1" 200 None 2026-03-31 08:37:07,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:07,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4a26a4c026e163483c7 HTTP/1.1" 200 None 2026-03-31 08:37:07,533 - lava_callback - ERROR - Error processing callback for node 69c8e4a26a4c026e163483c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:07,624 - lava_callback - INFO - Received callback for node 69c8e4af6a4c026e1634840c INFO: 172.19.0.1:60162 - "POST /node/69c8e4af6a4c026e1634840c HTTP/1.0" 202 Accepted 2026-03-31 08:37:07,635 - lava_callback - INFO - Starting async job submit for node 69c8e4af6a4c026e1634840c 2026-03-31 08:37:07,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4af6a4c026e1634840c HTTP/1.1" 200 None 2026-03-31 08:37:07,684 - lava_callback - DEBUG - Processing node 69c8e4af6a4c026e1634840c: baseline-arm-clabbe 2026-03-31 08:37:07,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4af6a4c026e1634840c 2026-03-31 08:37:07,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4af6a4c026e1634840c HTTP/1.1" 200 None 2026-03-31 08:37:07,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:07,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4af6a4c026e1634840c HTTP/1.1" 200 None 2026-03-31 08:37:07,845 - lava_callback - ERROR - Error processing callback for node 69c8e4af6a4c026e1634840c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:08,318 - lava_callback - INFO - Received callback for node 69c8e4b06a4c026e16348410 INFO: 172.19.0.1:60174 - "POST /node/69c8e4b06a4c026e16348410 HTTP/1.0" 202 Accepted 2026-03-31 08:37:08,333 - lava_callback - INFO - Starting async job submit for node 69c8e4b06a4c026e16348410 2026-03-31 08:37:08,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:08,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4b06a4c026e16348410 HTTP/1.1" 200 None 2026-03-31 08:37:08,365 - lava_callback - DEBUG - Processing node 69c8e4b06a4c026e16348410: baseline-arm-clabbe 2026-03-31 08:37:08,365 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4b06a4c026e16348410 2026-03-31 08:37:08,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:08,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4b06a4c026e16348410 HTTP/1.1" 200 None 2026-03-31 08:37:08,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:08,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:08,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:08,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4b06a4c026e16348410 HTTP/1.1" 200 None 2026-03-31 08:37:08,480 - lava_callback - ERROR - Error processing callback for node 69c8e4b06a4c026e16348410: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:11,669 - lava_callback - INFO - Received callback for node 69cb87a42a64fa9aff936f6d INFO: 172.19.0.1:34136 - "POST /node/69cb87a42a64fa9aff936f6d HTTP/1.0" 202 Accepted 2026-03-31 08:37:11,682 - lava_callback - INFO - Starting async job submit for node 69cb87a42a64fa9aff936f6d 2026-03-31 08:37:11,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:11,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a42a64fa9aff936f6d HTTP/1.1" 200 None 2026-03-31 08:37:11,718 - lava_callback - DEBUG - Processing node 69cb87a42a64fa9aff936f6d: baseline-arm 2026-03-31 08:37:11,726 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:37:11,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:11,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:11,989 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a42a64fa9aff936f6d/lava-logs.yaml 2026-03-31 08:37:11,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:12,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:12,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a42a64fa9aff936f6d/lava_callback.json.gz 2026-03-31 08:37:12,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:12,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a42a64fa9aff936f6d HTTP/1.1" 200 None 2026-03-31 08:37:12,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:12,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:37:12,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:12,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a42a64fa9aff936f6d HTTP/1.1" 200 None 2026-03-31 08:37:12,277 - lava_callback - INFO - Completed processing callback for node 69cb87a42a64fa9aff936f6d INFO: 172.19.0.1:34152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34160 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:37:26,304 - lava_callback - INFO - Received callback for node 69c8e4b36a4c026e1634841f INFO: 172.19.0.1:43268 - "POST /node/69c8e4b36a4c026e1634841f HTTP/1.0" 202 Accepted 2026-03-31 08:37:26,315 - lava_callback - INFO - Starting async job submit for node 69c8e4b36a4c026e1634841f 2026-03-31 08:37:26,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:26,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4b36a4c026e1634841f HTTP/1.1" 200 None 2026-03-31 08:37:26,341 - lava_callback - DEBUG - Processing node 69c8e4b36a4c026e1634841f: baseline-arm-clabbe 2026-03-31 08:37:26,341 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4b36a4c026e1634841f 2026-03-31 08:37:26,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:26,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4b36a4c026e1634841f HTTP/1.1" 200 None 2026-03-31 08:37:26,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:26,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:26,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:26,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4b36a4c026e1634841f HTTP/1.1" 200 None 2026-03-31 08:37:26,412 - lava_callback - ERROR - Error processing callback for node 69c8e4b36a4c026e1634841f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43282 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:37:27,507 - lava_callback - INFO - Received callback for node 69cb87a82a64fa9aff936f79 INFO: 172.19.0.1:43286 - "POST /node/69cb87a82a64fa9aff936f79 HTTP/1.0" 202 Accepted 2026-03-31 08:37:27,558 - lava_callback - INFO - Starting async job submit for node 69cb87a82a64fa9aff936f79 2026-03-31 08:37:27,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:27,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a82a64fa9aff936f79 HTTP/1.1" 200 None 2026-03-31 08:37:27,591 - lava_callback - DEBUG - Processing node 69cb87a82a64fa9aff936f79: baseline-arm 2026-03-31 08:37:27,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:28,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:28,203 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a82a64fa9aff936f79/log.txt.gz 2026-03-31 08:37:28,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:28,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:28,587 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a82a64fa9aff936f79/lava-logs.yaml 2026-03-31 08:37:28,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:28,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:28,909 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a82a64fa9aff936f79/lava_callback.json.gz 2026-03-31 08:37:28,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:28,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a82a64fa9aff936f79 HTTP/1.1" 200 None 2026-03-31 08:37:28,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:29,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:37:29,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:29,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a82a64fa9aff936f79 HTTP/1.1" 200 None 2026-03-31 08:37:29,117 - lava_callback - INFO - Completed processing callback for node 69cb87a82a64fa9aff936f79 INFO: 172.19.0.1:52892 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:37:37,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:37,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:37:42,840 - lava_callback - INFO - Received callback for node 69cb87a52a64fa9aff936f70 INFO: 172.19.0.1:52990 - "POST /node/69cb87a52a64fa9aff936f70 HTTP/1.0" 202 Accepted 2026-03-31 08:37:42,890 - lava_callback - INFO - Starting async job submit for node 69cb87a52a64fa9aff936f70 2026-03-31 08:37:42,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:42,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a52a64fa9aff936f70 HTTP/1.1" 200 None 2026-03-31 08:37:42,923 - lava_callback - DEBUG - Processing node 69cb87a52a64fa9aff936f70: baseline-arm 2026-03-31 08:37:43,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:43,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:43,162 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a52a64fa9aff936f70/log.txt.gz 2026-03-31 08:37:43,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:43,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:43,395 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a52a64fa9aff936f70/lava-logs.yaml 2026-03-31 08:37:43,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:43,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:43,515 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a52a64fa9aff936f70/lava_callback.json.gz 2026-03-31 08:37:43,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:43,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a52a64fa9aff936f70 HTTP/1.1" 200 None 2026-03-31 08:37:43,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:43,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:37:43,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:43,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a52a64fa9aff936f70 HTTP/1.1" 200 None 2026-03-31 08:37:43,625 - lava_callback - INFO - Completed processing callback for node 69cb87a52a64fa9aff936f70 INFO: 172.19.0.1:52996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53262 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:38:07,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:07,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:42786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42788 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:38:20,460 - lava_callback - INFO - Received callback for node 69cb87a42a64fa9aff936f6a INFO: 172.19.0.1:42646 - "POST /node/69cb87a42a64fa9aff936f6a HTTP/1.0" 202 Accepted 2026-03-31 08:38:20,467 - lava_callback - INFO - Starting async job submit for node 69cb87a42a64fa9aff936f6a 2026-03-31 08:38:20,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:20,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a42a64fa9aff936f6a HTTP/1.1" 200 None 2026-03-31 08:38:20,513 - lava_callback - DEBUG - Processing node 69cb87a42a64fa9aff936f6a: baseline-arm 2026-03-31 08:38:20,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:21,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:21,038 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a42a64fa9aff936f6a/log.txt.gz 2026-03-31 08:38:21,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:21,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:21,445 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a42a64fa9aff936f6a/lava-logs.yaml 2026-03-31 08:38:21,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:21,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:21,694 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a42a64fa9aff936f6a/lava_callback.json.gz 2026-03-31 08:38:21,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:21,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a42a64fa9aff936f6a HTTP/1.1" 200 None 2026-03-31 08:38:21,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:21,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:21,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:21,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a42a64fa9aff936f6a HTTP/1.1" 200 None 2026-03-31 08:38:21,831 - lava_callback - INFO - Completed processing callback for node 69cb87a42a64fa9aff936f6a 2026-03-31 08:38:23,951 - lava_callback - INFO - Received callback for node 69cb87a72a64fa9aff936f76 INFO: 172.19.0.1:42654 - "POST /node/69cb87a72a64fa9aff936f76 HTTP/1.0" 202 Accepted 2026-03-31 08:38:23,960 - lava_callback - INFO - Starting async job submit for node 69cb87a72a64fa9aff936f76 2026-03-31 08:38:23,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:24,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a72a64fa9aff936f76 HTTP/1.1" 200 None 2026-03-31 08:38:24,001 - lava_callback - DEBUG - Processing node 69cb87a72a64fa9aff936f76: baseline-arm 2026-03-31 08:38:24,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:24,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:24,425 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a72a64fa9aff936f76/log.txt.gz 2026-03-31 08:38:24,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:24,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:24,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a72a64fa9aff936f76/lava-logs.yaml 2026-03-31 08:38:24,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:24,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:24,959 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a72a64fa9aff936f76/lava_callback.json.gz 2026-03-31 08:38:24,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:25,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a72a64fa9aff936f76 HTTP/1.1" 200 None 2026-03-31 08:38:25,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:25,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:25,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:25,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a72a64fa9aff936f76 HTTP/1.1" 200 None 2026-03-31 08:38:25,059 - lava_callback - INFO - Completed processing callback for node 69cb87a72a64fa9aff936f76 INFO: 172.19.0.1:42656 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:38:36,031 - lava_callback - INFO - Received callback for node 69cb87cd2a64fa9aff937011 INFO: 172.19.0.1:52296 - "POST /node/69cb87cd2a64fa9aff937011 HTTP/1.0" 202 Accepted 2026-03-31 08:38:36,077 - lava_callback - INFO - Starting async job submit for node 69cb87cd2a64fa9aff937011 2026-03-31 08:38:36,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:36,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87cd2a64fa9aff937011 HTTP/1.1" 200 None 2026-03-31 08:38:36,118 - lava_callback - DEBUG - Processing node 69cb87cd2a64fa9aff937011: baseline-arm-pengutronix 2026-03-31 08:38:36,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:36,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:36,368 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cd2a64fa9aff937011/log.txt.gz 2026-03-31 08:38:36,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:52312 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:38:36,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:36,754 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cd2a64fa9aff937011/lava-logs.yaml 2026-03-31 08:38:36,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:36,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:36,885 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cd2a64fa9aff937011/lava_callback.json.gz 2026-03-31 08:38:36,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:36,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87cd2a64fa9aff937011 HTTP/1.1" 200 None 2026-03-31 08:38:36,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:36,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:36,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:36,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87cd2a64fa9aff937011 HTTP/1.1" 200 None 2026-03-31 08:38:36,967 - lava_callback - INFO - Completed processing callback for node 69cb87cd2a64fa9aff937011 2026-03-31 08:38:37,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:37,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:38:38,466 - lava_callback - INFO - Received callback for node 69cb87c92a64fa9aff937005 INFO: 172.19.0.1:52324 - "POST /node/69cb87c92a64fa9aff937005 HTTP/1.0" 202 Accepted 2026-03-31 08:38:38,472 - lava_callback - INFO - Starting async job submit for node 69cb87c92a64fa9aff937005 2026-03-31 08:38:38,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:38,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c92a64fa9aff937005 HTTP/1.1" 200 None 2026-03-31 08:38:38,507 - lava_callback - DEBUG - Processing node 69cb87c92a64fa9aff937005: baseline-arm-pengutronix 2026-03-31 08:38:38,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:38,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:38,797 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87c92a64fa9aff937005/log.txt.gz 2026-03-31 08:38:38,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:39,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:39,073 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87c92a64fa9aff937005/lava-logs.yaml 2026-03-31 08:38:39,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:39,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:39,238 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87c92a64fa9aff937005/lava_callback.json.gz 2026-03-31 08:38:39,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:39,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c92a64fa9aff937005 HTTP/1.1" 200 None 2026-03-31 08:38:39,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:39,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:39,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:39,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87c92a64fa9aff937005 HTTP/1.1" 200 None 2026-03-31 08:38:39,333 - lava_callback - INFO - Completed processing callback for node 69cb87c92a64fa9aff937005 2026-03-31 08:38:39,933 - lava_callback - INFO - Received callback for node 69cb87ca2a64fa9aff937009 INFO: 172.19.0.1:43030 - "POST /node/69cb87ca2a64fa9aff937009 HTTP/1.0" 202 Accepted 2026-03-31 08:38:39,939 - lava_callback - INFO - Starting async job submit for node 69cb87ca2a64fa9aff937009 2026-03-31 08:38:39,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:39,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ca2a64fa9aff937009 HTTP/1.1" 200 None 2026-03-31 08:38:39,972 - lava_callback - DEBUG - Processing node 69cb87ca2a64fa9aff937009: baseline-arm-pengutronix 2026-03-31 08:38:40,131 - lava_callback - INFO - Received callback for node 69cb87be2a64fa9aff936fdd 2026-03-31 08:38:40,137 - lava_callback - INFO - Starting async job submit for node 69cb87be2a64fa9aff936fdd INFO: 172.19.0.1:43038 - "POST /node/69cb87be2a64fa9aff936fdd HTTP/1.0" 202 Accepted 2026-03-31 08:38:40,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:40,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:40,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87be2a64fa9aff936fdd HTTP/1.1" 200 None 2026-03-31 08:38:40,185 - lava_callback - DEBUG - Processing node 69cb87be2a64fa9aff936fdd: baseline-arm-clabbe 2026-03-31 08:38:40,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:40,375 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87ca2a64fa9aff937009/log.txt.gz 2026-03-31 08:38:40,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:40,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:40,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:40,702 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87be2a64fa9aff936fdd/log.txt.gz 2026-03-31 08:38:40,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:40,778 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87ca2a64fa9aff937009/lava-logs.yaml 2026-03-31 08:38:40,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:40,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:41,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:41,020 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87ca2a64fa9aff937009/lava_callback.json.gz 2026-03-31 08:38:41,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:41,059 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87be2a64fa9aff936fdd/lava-logs.yaml 2026-03-31 08:38:41,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ca2a64fa9aff937009 HTTP/1.1" 200 None 2026-03-31 08:38:41,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:41,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:41,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87ca2a64fa9aff937009 HTTP/1.1" 200 None 2026-03-31 08:38:41,122 - lava_callback - INFO - Completed processing callback for node 69cb87ca2a64fa9aff937009 2026-03-31 08:38:41,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:41,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87be2a64fa9aff936fdd/lava_callback.json.gz 2026-03-31 08:38:41,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87be2a64fa9aff936fdd HTTP/1.1" 200 None 2026-03-31 08:38:41,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:41,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87be2a64fa9aff936fdd HTTP/1.1" 200 None 2026-03-31 08:38:41,371 - lava_callback - INFO - Completed processing callback for node 69cb87be2a64fa9aff936fdd INFO: 172.19.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43052 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:38:50,692 - lava_callback - INFO - Received callback for node 69cb87b82a64fa9aff936faa INFO: 172.19.0.1:52502 - "POST /node/69cb87b82a64fa9aff936faa HTTP/1.0" 202 Accepted 2026-03-31 08:38:50,699 - lava_callback - INFO - Starting async job submit for node 69cb87b82a64fa9aff936faa 2026-03-31 08:38:50,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:50,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b82a64fa9aff936faa HTTP/1.1" 200 None 2026-03-31 08:38:50,751 - lava_callback - DEBUG - Processing node 69cb87b82a64fa9aff936faa: baseline-arm-broonie 2026-03-31 08:38:51,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:51,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:51,211 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b82a64fa9aff936faa/log.txt.gz 2026-03-31 08:38:51,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:51,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:51,637 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b82a64fa9aff936faa/lava-logs.yaml 2026-03-31 08:38:51,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:51,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:51,811 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b82a64fa9aff936faa/lava_callback.json.gz 2026-03-31 08:38:51,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:51,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b82a64fa9aff936faa HTTP/1.1" 200 None 2026-03-31 08:38:51,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:51,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:51,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:51,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87b82a64fa9aff936faa HTTP/1.1" 200 None 2026-03-31 08:38:51,933 - lava_callback - INFO - Completed processing callback for node 69cb87b82a64fa9aff936faa INFO: 172.19.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46508 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:39:07,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:07,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:39:09,283 - lava_callback - INFO - Received callback for node 69cb87b92a64fa9aff936faf INFO: 172.19.0.1:46520 - "POST /node/69cb87b92a64fa9aff936faf HTTP/1.0" 202 Accepted 2026-03-31 08:39:09,289 - lava_callback - INFO - Starting async job submit for node 69cb87b92a64fa9aff936faf 2026-03-31 08:39:09,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:09,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b92a64fa9aff936faf HTTP/1.1" 200 None 2026-03-31 08:39:09,328 - lava_callback - DEBUG - Processing node 69cb87b92a64fa9aff936faf: baseline-arm-broonie 2026-03-31 08:39:09,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:09,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:09,673 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b92a64fa9aff936faf/log.txt.gz 2026-03-31 08:39:09,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:09,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:09,994 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b92a64fa9aff936faf/lava-logs.yaml 2026-03-31 08:39:10,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:10,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:10,142 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b92a64fa9aff936faf/lava_callback.json.gz 2026-03-31 08:39:10,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:10,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b92a64fa9aff936faf HTTP/1.1" 200 None 2026-03-31 08:39:10,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:10,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:39:10,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:10,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87b92a64fa9aff936faf HTTP/1.1" 200 None 2026-03-31 08:39:10,241 - lava_callback - INFO - Completed processing callback for node 69cb87b92a64fa9aff936faf 2026-03-31 08:39:10,660 - lava_callback - INFO - Received callback for node 69cb87b62a64fa9aff936fa7 INFO: 172.19.0.1:48296 - "POST /node/69cb87b62a64fa9aff936fa7 HTTP/1.0" 202 Accepted 2026-03-31 08:39:10,666 - lava_callback - INFO - Starting async job submit for node 69cb87b62a64fa9aff936fa7 2026-03-31 08:39:10,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:10,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b62a64fa9aff936fa7 HTTP/1.1" 200 None 2026-03-31 08:39:10,707 - lava_callback - DEBUG - Processing node 69cb87b62a64fa9aff936fa7: baseline-arm-broonie 2026-03-31 08:39:10,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:11,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:11,047 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b62a64fa9aff936fa7/log.txt.gz 2026-03-31 08:39:11,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:11,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:11,346 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b62a64fa9aff936fa7/lava-logs.yaml 2026-03-31 08:39:11,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:11,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:11,493 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b62a64fa9aff936fa7/lava_callback.json.gz 2026-03-31 08:39:11,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:11,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b62a64fa9aff936fa7 HTTP/1.1" 200 None 2026-03-31 08:39:11,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:11,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:39:11,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:11,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87b62a64fa9aff936fa7 HTTP/1.1" 200 None 2026-03-31 08:39:11,576 - lava_callback - INFO - Completed processing callback for node 69cb87b62a64fa9aff936fa7 INFO: 172.19.0.1:48308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60312 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:39:29,526 - lava_callback - INFO - Received callback for node 69cb87b52a64fa9aff936fa4 INFO: 172.19.0.1:60322 - "POST /node/69cb87b52a64fa9aff936fa4 HTTP/1.0" 202 Accepted 2026-03-31 08:39:29,532 - lava_callback - INFO - Starting async job submit for node 69cb87b52a64fa9aff936fa4 2026-03-31 08:39:29,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:29,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b52a64fa9aff936fa4 HTTP/1.1" 200 None 2026-03-31 08:39:29,574 - lava_callback - DEBUG - Processing node 69cb87b52a64fa9aff936fa4: baseline-arm-broonie 2026-03-31 08:39:29,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:29,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:29,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b52a64fa9aff936fa4/log.txt.gz 2026-03-31 08:39:30,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:30,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:30,408 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b52a64fa9aff936fa4/lava-logs.yaml 2026-03-31 08:39:30,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:30,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:30,562 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b52a64fa9aff936fa4/lava_callback.json.gz 2026-03-31 08:39:30,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:30,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b52a64fa9aff936fa4 HTTP/1.1" 200 None 2026-03-31 08:39:30,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:30,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:39:30,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:30,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87b52a64fa9aff936fa4 HTTP/1.1" 200 None 2026-03-31 08:39:30,651 - lava_callback - INFO - Completed processing callback for node 69cb87b52a64fa9aff936fa4 INFO: 172.19.0.1:44268 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:39:37,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:37,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:34680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45128 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:40:00,407 - lava_callback - INFO - Received callback for node 69cb87e52a64fa9aff937046 INFO: 172.19.0.1:51216 - "POST /node/69cb87e52a64fa9aff937046 HTTP/1.0" 202 Accepted 2026-03-31 08:40:00,415 - lava_callback - INFO - Starting async job submit for node 69cb87e52a64fa9aff937046 2026-03-31 08:40:00,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:00,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e52a64fa9aff937046 HTTP/1.1" 200 None 2026-03-31 08:40:00,454 - lava_callback - DEBUG - Processing node 69cb87e52a64fa9aff937046: ltp-dio 2026-03-31 08:40:00,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:00,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:00,978 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69cb87e52a64fa9aff937046/log.txt.gz 2026-03-31 08:40:01,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:01,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:01,454 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69cb87e52a64fa9aff937046/lava-logs.yaml 2026-03-31 08:40:01,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:01,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:01,625 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69cb87e52a64fa9aff937046/lava_callback.json.gz 2026-03-31 08:40:01,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:01,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e52a64fa9aff937046 HTTP/1.1" 200 None 2026-03-31 08:40:01,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:01,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:40:01,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:01,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e52a64fa9aff937046 HTTP/1.1" 200 None 2026-03-31 08:40:01,758 - lava_callback - INFO - Completed processing callback for node 69cb87e52a64fa9aff937046 INFO: 172.19.0.1:51228 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:40:07,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:07,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:40:07,561 - lava_callback - INFO - Received callback for node 69cb87bf2a64fa9aff936fe0 INFO: 172.19.0.1:51238 - "POST /node/69cb87bf2a64fa9aff936fe0 HTTP/1.0" 202 Accepted 2026-03-31 08:40:07,567 - lava_callback - INFO - Starting async job submit for node 69cb87bf2a64fa9aff936fe0 2026-03-31 08:40:07,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:07,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87bf2a64fa9aff936fe0 HTTP/1.1" 200 None 2026-03-31 08:40:07,604 - lava_callback - DEBUG - Processing node 69cb87bf2a64fa9aff936fe0: baseline-arm-clabbe 2026-03-31 08:40:07,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:07,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:07,929 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87bf2a64fa9aff936fe0/log.txt.gz 2026-03-31 08:40:08,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:08,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:08,353 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87bf2a64fa9aff936fe0/lava-logs.yaml 2026-03-31 08:40:08,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:08,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:08,542 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87bf2a64fa9aff936fe0/lava_callback.json.gz 2026-03-31 08:40:08,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:08,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87bf2a64fa9aff936fe0 HTTP/1.1" 200 None 2026-03-31 08:40:08,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:08,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:40:08,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:08,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87bf2a64fa9aff936fe0 HTTP/1.1" 200 None 2026-03-31 08:40:08,627 - lava_callback - INFO - Completed processing callback for node 69cb87bf2a64fa9aff936fe0 INFO: 172.19.0.1:58894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40660 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:40:30,130 - lava_callback - INFO - Received callback for node 69cb87fa2a64fa9aff937085 INFO: 172.19.0.1:39768 - "POST /node/69cb87fa2a64fa9aff937085 HTTP/1.0" 202 Accepted 2026-03-31 08:40:30,136 - lava_callback - INFO - Starting async job submit for node 69cb87fa2a64fa9aff937085 2026-03-31 08:40:30,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:30,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87fa2a64fa9aff937085 HTTP/1.1" 200 None 2026-03-31 08:40:30,174 - lava_callback - DEBUG - Processing node 69cb87fa2a64fa9aff937085: baseline-arm 2026-03-31 08:40:30,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:30,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:30,498 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87fa2a64fa9aff937085/log.txt.gz 2026-03-31 08:40:30,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:30,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:30,792 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87fa2a64fa9aff937085/lava-logs.yaml 2026-03-31 08:40:30,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:30,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:30,977 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87fa2a64fa9aff937085/lava_callback.json.gz 2026-03-31 08:40:31,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:31,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87fa2a64fa9aff937085 HTTP/1.1" 200 None 2026-03-31 08:40:31,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:31,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83462a64fa9aff936905 HTTP/1.1" 200 None 2026-03-31 08:40:31,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:31,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87fa2a64fa9aff937085 HTTP/1.1" 200 None 2026-03-31 08:40:31,109 - lava_callback - INFO - Completed processing callback for node 69cb87fa2a64fa9aff937085 INFO: 172.19.0.1:39778 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:40:37,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:37,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:47726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44078 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:41:42,211 - lava_callback - INFO - Received callback for node 69cb87c32a64fa9aff936fec INFO: 172.19.0.1:35790 - "POST /node/69cb87c32a64fa9aff936fec HTTP/1.0" 202 Accepted 2026-03-31 08:41:42,220 - lava_callback - INFO - Starting async job submit for node 69cb87c32a64fa9aff936fec 2026-03-31 08:41:42,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:41:42,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c32a64fa9aff936fec HTTP/1.1" 200 None 2026-03-31 08:41:42,292 - lava_callback - DEBUG - Processing node 69cb87c32a64fa9aff936fec: baseline-arm-clabbe 2026-03-31 08:41:42,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:41:42,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:41:42,751 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c32a64fa9aff936fec/log.txt.gz 2026-03-31 08:41:42,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:41:43,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:41:43,255 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c32a64fa9aff936fec/lava-logs.yaml 2026-03-31 08:41:43,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:41:43,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:41:43,681 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c32a64fa9aff936fec/lava_callback.json.gz 2026-03-31 08:41:43,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:41:43,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c32a64fa9aff936fec HTTP/1.1" 200 None 2026-03-31 08:41:43,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:41:43,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:41:43,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:41:43,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87c32a64fa9aff936fec HTTP/1.1" 200 None 2026-03-31 08:41:43,807 - lava_callback - INFO - Completed processing callback for node 69cb87c32a64fa9aff936fec INFO: 172.19.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34512 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:42:07,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:07,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:42:09,634 - lava_callback - INFO - Received callback for node 69cb87de2a64fa9aff93703a INFO: 172.19.0.1:34514 - "POST /node/69cb87de2a64fa9aff93703a HTTP/1.0" 202 Accepted 2026-03-31 08:42:09,648 - lava_callback - INFO - Starting async job submit for node 69cb87de2a64fa9aff93703a 2026-03-31 08:42:09,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:09,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87de2a64fa9aff93703a HTTP/1.1" 200 None 2026-03-31 08:42:09,726 - lava_callback - DEBUG - Processing node 69cb87de2a64fa9aff93703a: ltp-cap_bounds 2026-03-31 08:42:10,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:10,681 - lava_callback - INFO - Received callback for node 69cb87e02a64fa9aff93703d INFO: 172.19.0.1:54582 - "POST /node/69cb87e02a64fa9aff93703d HTTP/1.0" 202 Accepted 2026-03-31 08:42:10,691 - lava_callback - INFO - Starting async job submit for node 69cb87e02a64fa9aff93703d 2026-03-31 08:42:10,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:10,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e02a64fa9aff93703d HTTP/1.1" 200 None 2026-03-31 08:42:10,771 - lava_callback - DEBUG - Processing node 69cb87e02a64fa9aff93703d: ltp-containers 2026-03-31 08:42:10,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:10,894 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cb87de2a64fa9aff93703a/log.txt.gz 2026-03-31 08:42:11,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:11,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:12,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:12,520 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69cb87e02a64fa9aff93703d/log.txt.gz 2026-03-31 08:42:12,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:12,888 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cb87de2a64fa9aff93703a/lava-logs.yaml 2026-03-31 08:42:12,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:12,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:13,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:13,766 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cb87de2a64fa9aff93703a/lava_callback.json.gz 2026-03-31 08:42:13,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:13,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87de2a64fa9aff93703a HTTP/1.1" 200 None 2026-03-31 08:42:13,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:13,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:42:13,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:13,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87de2a64fa9aff93703a HTTP/1.1" 200 None 2026-03-31 08:42:13,929 - lava_callback - INFO - Completed processing callback for node 69cb87de2a64fa9aff93703a 2026-03-31 08:42:14,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:14,156 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69cb87e02a64fa9aff93703d/lava-logs.yaml 2026-03-31 08:42:14,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:15,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:15,092 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69cb87e02a64fa9aff93703d/lava_callback.json.gz 2026-03-31 08:42:15,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:54590 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 08:42:15,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e02a64fa9aff93703d HTTP/1.1" 200 None 2026-03-31 08:42:15,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:15,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:42:15,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:15,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e02a64fa9aff93703d HTTP/1.1" 200 None 2026-03-31 08:42:15,255 - lava_callback - INFO - Completed processing callback for node 69cb87e02a64fa9aff93703d INFO: 172.19.0.1:54604 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49668 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:42:29,649 - lava_callback - INFO - Received callback for node 69cb87e12a64fa9aff937040 INFO: 172.19.0.1:49676 - "POST /node/69cb87e12a64fa9aff937040 HTTP/1.0" 202 Accepted 2026-03-31 08:42:29,660 - lava_callback - INFO - Starting async job submit for node 69cb87e12a64fa9aff937040 2026-03-31 08:42:29,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:29,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e12a64fa9aff937040 HTTP/1.1" 200 None 2026-03-31 08:42:29,753 - lava_callback - DEBUG - Processing node 69cb87e12a64fa9aff937040: ltp-controllers 2026-03-31 08:42:30,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:31,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:31,024 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69cb87e12a64fa9aff937040/log.txt.gz 2026-03-31 08:42:31,324 - lava_callback - INFO - Received callback for node 69cb87f22a64fa9aff937065 INFO: 172.19.0.1:45458 - "POST /node/69cb87f22a64fa9aff937065 HTTP/1.0" 202 Accepted 2026-03-31 08:42:31,336 - lava_callback - INFO - Starting async job submit for node 69cb87f22a64fa9aff937065 2026-03-31 08:42:31,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:31,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f22a64fa9aff937065 HTTP/1.1" 200 None 2026-03-31 08:42:31,500 - lava_callback - DEBUG - Processing node 69cb87f22a64fa9aff937065: ltp-syscalls 2026-03-31 08:42:31,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:32,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:32,599 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69cb87e12a64fa9aff937040/lava-logs.yaml 2026-03-31 08:42:32,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:32,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:33,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:33,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cb87f22a64fa9aff937065/log.txt.gz 2026-03-31 08:42:33,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:33,583 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69cb87e12a64fa9aff937040/lava_callback.json.gz 2026-03-31 08:42:33,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:33,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e12a64fa9aff937040 HTTP/1.1" 200 None 2026-03-31 08:42:33,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:33,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:42:33,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:33,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e12a64fa9aff937040 HTTP/1.1" 200 None 2026-03-31 08:42:33,934 - lava_callback - INFO - Completed processing callback for node 69cb87e12a64fa9aff937040 2026-03-31 08:42:34,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:35,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:35,551 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cb87f22a64fa9aff937065/lava-logs.yaml 2026-03-31 08:42:35,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:36,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:36,395 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cb87f22a64fa9aff937065/lava_callback.json.gz 2026-03-31 08:42:36,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:36,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f22a64fa9aff937065 HTTP/1.1" 200 None 2026-03-31 08:42:36,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:36,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:42:36,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:45464 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:42:36,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87f22a64fa9aff937065 HTTP/1.1" 200 None 2026-03-31 08:42:36,538 - lava_callback - INFO - Completed processing callback for node 69cb87f22a64fa9aff937065 2026-03-31 08:42:37,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:37,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:42:46,436 - lava_callback - INFO - Received callback for node 69c8e6026a4c026e163485a0 INFO: 172.19.0.1:58186 - "POST /node/69c8e6026a4c026e163485a0 HTTP/1.0" 202 Accepted 2026-03-31 08:42:46,445 - lava_callback - INFO - Starting async job submit for node 69c8e6026a4c026e163485a0 2026-03-31 08:42:46,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:46,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6026a4c026e163485a0 HTTP/1.1" 200 None 2026-03-31 08:42:46,497 - lava_callback - DEBUG - Processing node 69c8e6026a4c026e163485a0: baseline-arm-baylibre 2026-03-31 08:42:46,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e6026a4c026e163485a0 2026-03-31 08:42:46,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:58202 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:42:46,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6026a4c026e163485a0 HTTP/1.1" 200 None 2026-03-31 08:42:46,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:46,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:42:46,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:46,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e6026a4c026e163485a0 HTTP/1.1" 200 None 2026-03-31 08:42:46,600 - lava_callback - ERROR - Error processing callback for node 69c8e6026a4c026e163485a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41092 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:43:09,308 - lava_callback - INFO - Received callback for node 69cb87c42a64fa9aff936fef INFO: 172.19.0.1:41094 - "POST /node/69cb87c42a64fa9aff936fef HTTP/1.0" 202 Accepted 2026-03-31 08:43:09,319 - lava_callback - INFO - Starting async job submit for node 69cb87c42a64fa9aff936fef 2026-03-31 08:43:09,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:09,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c42a64fa9aff936fef HTTP/1.1" 200 None 2026-03-31 08:43:09,396 - lava_callback - DEBUG - Processing node 69cb87c42a64fa9aff936fef: baseline-arm-clabbe 2026-03-31 08:43:09,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:10,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:10,112 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c42a64fa9aff936fef/log.txt.gz 2026-03-31 08:43:10,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:10,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:10,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c42a64fa9aff936fef/lava-logs.yaml 2026-03-31 08:43:10,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:11,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:11,512 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c42a64fa9aff936fef/lava_callback.json.gz 2026-03-31 08:43:11,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:11,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c42a64fa9aff936fef HTTP/1.1" 200 None 2026-03-31 08:43:11,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:11,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:43:11,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:11,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87c42a64fa9aff936fef HTTP/1.1" 200 None 2026-03-31 08:43:11,698 - lava_callback - INFO - Completed processing callback for node 69cb87c42a64fa9aff936fef INFO: 172.19.0.1:43020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38956 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:43:37,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:37,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:51894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51898 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:43:49,753 - lava_callback - INFO - Received callback for node 69cb87e32a64fa9aff937043 INFO: 172.19.0.1:40428 - "POST /node/69cb87e32a64fa9aff937043 HTTP/1.0" 202 Accepted 2026-03-31 08:43:49,760 - lava_callback - INFO - Starting async job submit for node 69cb87e32a64fa9aff937043 2026-03-31 08:43:49,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:49,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e32a64fa9aff937043 HTTP/1.1" 200 None 2026-03-31 08:43:49,802 - lava_callback - DEBUG - Processing node 69cb87e32a64fa9aff937043: ltp-cve 2026-03-31 08:43:50,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:50,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:50,424 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69cb87e32a64fa9aff937043/log.txt.gz 2026-03-31 08:43:50,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:50,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:50,941 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69cb87e32a64fa9aff937043/lava-logs.yaml 2026-03-31 08:43:50,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:51,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:51,142 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69cb87e32a64fa9aff937043/lava_callback.json.gz 2026-03-31 08:43:51,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:51,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e32a64fa9aff937043 HTTP/1.1" 200 None 2026-03-31 08:43:51,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:51,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:43:51,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:51,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e32a64fa9aff937043 HTTP/1.1" 200 None 2026-03-31 08:43:51,261 - lava_callback - INFO - Completed processing callback for node 69cb87e32a64fa9aff937043 INFO: 172.19.0.1:40442 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:44:06,419 - lava_callback - INFO - Received callback for node 69c8e64d6a4c026e163485c8 INFO: 172.19.0.1:32852 - "POST /node/69c8e64d6a4c026e163485c8 HTTP/1.0" 202 Accepted 2026-03-31 08:44:06,427 - lava_callback - INFO - Starting async job submit for node 69c8e64d6a4c026e163485c8 2026-03-31 08:44:06,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:06,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e64d6a4c026e163485c8 HTTP/1.1" 200 None 2026-03-31 08:44:06,450 - lava_callback - DEBUG - Processing node 69c8e64d6a4c026e163485c8: baseline-arm-baylibre 2026-03-31 08:44:06,450 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e64d6a4c026e163485c8 2026-03-31 08:44:06,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:06,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e64d6a4c026e163485c8 HTTP/1.1" 200 None 2026-03-31 08:44:06,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:06,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:44:06,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:32854 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:44:06,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e64d6a4c026e163485c8 HTTP/1.1" 200 None 2026-03-31 08:44:06,526 - lava_callback - ERROR - Error processing callback for node 69c8e64d6a4c026e163485c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:44:07,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:07,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:44:10,041 - lava_callback - INFO - Received callback for node 69cb89512a64fa9aff93716a INFO: 172.19.0.1:42186 - "POST /node/69cb89512a64fa9aff93716a HTTP/1.0" 202 Accepted 2026-03-31 08:44:10,047 - lava_callback - INFO - Starting async job submit for node 69cb89512a64fa9aff93716a 2026-03-31 08:44:10,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:10,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb89512a64fa9aff93716a HTTP/1.1" 200 None 2026-03-31 08:44:10,067 - lava_callback - DEBUG - Processing node 69cb89512a64fa9aff93716a: baseline-arm 2026-03-31 08:44:10,075 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:44:10,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:44:10,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:44:10,247 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb89512a64fa9aff93716a/lava-logs.yaml 2026-03-31 08:44:10,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:44:10,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:44:10,395 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb89512a64fa9aff93716a/lava_callback.json.gz 2026-03-31 08:44:10,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:10,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb89512a64fa9aff93716a HTTP/1.1" 200 None 2026-03-31 08:44:10,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:10,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:44:10,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:10,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb89512a64fa9aff93716a HTTP/1.1" 200 None 2026-03-31 08:44:10,464 - lava_callback - INFO - Completed processing callback for node 69cb89512a64fa9aff93716a INFO: 172.19.0.1:42198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48050 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:44:37,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:37,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:44:46,437 - lava_callback - INFO - Received callback for node 69c8e6736a4c026e163485d6 INFO: 172.19.0.1:36958 - "POST /node/69c8e6736a4c026e163485d6 HTTP/1.0" 202 Accepted 2026-03-31 08:44:46,442 - lava_callback - INFO - Starting async job submit for node 69c8e6736a4c026e163485d6 2026-03-31 08:44:46,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:46,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6736a4c026e163485d6 HTTP/1.1" 200 None 2026-03-31 08:44:46,458 - lava_callback - DEBUG - Processing node 69c8e6736a4c026e163485d6: baseline-arm-baylibre 2026-03-31 08:44:46,459 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e6736a4c026e163485d6 2026-03-31 08:44:46,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:46,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6736a4c026e163485d6 HTTP/1.1" 200 None 2026-03-31 08:44:46,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:46,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:44:46,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:36972 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:44:46,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e6736a4c026e163485d6 HTTP/1.1" 200 None 2026-03-31 08:44:46,533 - lava_callback - ERROR - Error processing callback for node 69c8e6736a4c026e163485d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53770 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:45:26,638 - lava_callback - INFO - Received callback for node 69c8e6996a4c026e163485e7 INFO: 172.19.0.1:53780 - "POST /node/69c8e6996a4c026e163485e7 HTTP/1.0" 202 Accepted 2026-03-31 08:45:26,644 - lava_callback - INFO - Starting async job submit for node 69c8e6996a4c026e163485e7 2026-03-31 08:45:26,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:45:26,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6996a4c026e163485e7 HTTP/1.1" 200 None 2026-03-31 08:45:26,661 - lava_callback - DEBUG - Processing node 69c8e6996a4c026e163485e7: baseline-arm-baylibre 2026-03-31 08:45:26,661 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e6996a4c026e163485e7 2026-03-31 08:45:26,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:45:26,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6996a4c026e163485e7 HTTP/1.1" 200 None 2026-03-31 08:45:26,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:45:26,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:45:26,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:45:26,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e6996a4c026e163485e7 HTTP/1.1" 200 None 2026-03-31 08:45:26,719 - lava_callback - ERROR - Error processing callback for node 69c8e6996a4c026e163485e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58512 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:46:06,491 - lava_callback - INFO - Received callback for node 69c8e6bf6a4c026e163485fe INFO: 172.19.0.1:54770 - "POST /node/69c8e6bf6a4c026e163485fe HTTP/1.0" 202 Accepted 2026-03-31 08:46:06,497 - lava_callback - INFO - Starting async job submit for node 69c8e6bf6a4c026e163485fe 2026-03-31 08:46:06,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:06,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6bf6a4c026e163485fe HTTP/1.1" 200 None 2026-03-31 08:46:06,514 - lava_callback - DEBUG - Processing node 69c8e6bf6a4c026e163485fe: baseline-arm-baylibre 2026-03-31 08:46:06,514 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e6bf6a4c026e163485fe 2026-03-31 08:46:06,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:54782 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:46:06,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6bf6a4c026e163485fe HTTP/1.1" 200 None 2026-03-31 08:46:06,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:06,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:46:06,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:06,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e6bf6a4c026e163485fe HTTP/1.1" 200 None 2026-03-31 08:46:06,571 - lava_callback - ERROR - Error processing callback for node 69c8e6bf6a4c026e163485fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33428 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:46:30,655 - lava_callback - INFO - Received callback for node 69cb87e62a64fa9aff937049 INFO: 172.19.0.1:49728 - "POST /node/69cb87e62a64fa9aff937049 HTTP/1.0" 202 Accepted 2026-03-31 08:46:30,671 - lava_callback - INFO - Starting async job submit for node 69cb87e62a64fa9aff937049 2026-03-31 08:46:30,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:30,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e62a64fa9aff937049 HTTP/1.1" 200 None 2026-03-31 08:46:30,752 - lava_callback - DEBUG - Processing node 69cb87e62a64fa9aff937049: ltp-fs_bind 2026-03-31 08:46:31,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:46:31,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:46:31,943 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cb87e62a64fa9aff937049/log.txt.gz 2026-03-31 08:46:32,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:46:33,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:46:33,125 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cb87e62a64fa9aff937049/lava-logs.yaml 2026-03-31 08:46:33,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:46:33,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:46:33,699 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cb87e62a64fa9aff937049/lava_callback.json.gz 2026-03-31 08:46:33,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:33,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e62a64fa9aff937049 HTTP/1.1" 200 None 2026-03-31 08:46:33,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:33,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:46:33,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:33,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e62a64fa9aff937049 HTTP/1.1" 200 None 2026-03-31 08:46:33,860 - lava_callback - INFO - Completed processing callback for node 69cb87e62a64fa9aff937049 INFO: 172.19.0.1:49742 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:46:37,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:37,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:48496 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:46:46,731 - lava_callback - INFO - Received callback for node 69c8e6e36a4c026e1634860e INFO: 172.19.0.1:48510 - "POST /node/69c8e6e36a4c026e1634860e HTTP/1.0" 202 Accepted 2026-03-31 08:46:46,741 - lava_callback - INFO - Starting async job submit for node 69c8e6e36a4c026e1634860e 2026-03-31 08:46:46,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:46,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6e36a4c026e1634860e HTTP/1.1" 200 None 2026-03-31 08:46:46,776 - lava_callback - DEBUG - Processing node 69c8e6e36a4c026e1634860e: baseline-arm-baylibre 2026-03-31 08:46:46,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e6e36a4c026e1634860e 2026-03-31 08:46:46,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:46,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6e36a4c026e1634860e HTTP/1.1" 200 None 2026-03-31 08:46:46,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:46,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:46:46,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:46,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e6e36a4c026e1634860e HTTP/1.1" 200 None 2026-03-31 08:46:46,879 - lava_callback - ERROR - Error processing callback for node 69c8e6e36a4c026e1634860e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57876 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:47:09,798 - lava_callback - INFO - Received callback for node 69cb87e82a64fa9aff93704c INFO: 172.19.0.1:58954 - "POST /node/69cb87e82a64fa9aff93704c HTTP/1.0" 202 Accepted 2026-03-31 08:47:09,808 - lava_callback - INFO - Starting async job submit for node 69cb87e82a64fa9aff93704c 2026-03-31 08:47:09,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:09,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e82a64fa9aff93704c HTTP/1.1" 200 None 2026-03-31 08:47:09,902 - lava_callback - DEBUG - Processing node 69cb87e82a64fa9aff93704c: ltp-fsx 2026-03-31 08:47:10,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:47:10,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:47:10,945 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69cb87e82a64fa9aff93704c/log.txt.gz 2026-03-31 08:47:11,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:47:12,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:47:12,362 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69cb87e82a64fa9aff93704c/lava-logs.yaml 2026-03-31 08:47:12,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:47:13,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:47:13,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69cb87e82a64fa9aff93704c/lava_callback.json.gz 2026-03-31 08:47:13,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:13,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e82a64fa9aff93704c HTTP/1.1" 200 None 2026-03-31 08:47:13,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:13,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:47:13,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:13,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e82a64fa9aff93704c HTTP/1.1" 200 None 2026-03-31 08:47:13,486 - lava_callback - INFO - Completed processing callback for node 69cb87e82a64fa9aff93704c INFO: 172.19.0.1:58970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60628 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:47:37,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:37,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:59994 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:47:46,754 - lava_callback - INFO - Received callback for node 69c8e72d6a4c026e16348639 INFO: 172.19.0.1:59998 - "POST /node/69c8e72d6a4c026e16348639 HTTP/1.0" 202 Accepted 2026-03-31 08:47:46,762 - lava_callback - INFO - Starting async job submit for node 69c8e72d6a4c026e16348639 2026-03-31 08:47:46,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:46,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e72d6a4c026e16348639 HTTP/1.1" 200 None 2026-03-31 08:47:46,802 - lava_callback - DEBUG - Processing node 69c8e72d6a4c026e16348639: baseline-arm-clabbe 2026-03-31 08:47:46,803 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e72d6a4c026e16348639 2026-03-31 08:47:46,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:46,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e72d6a4c026e16348639 HTTP/1.1" 200 None 2026-03-31 08:47:46,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:46,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:47:46,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:46,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e72d6a4c026e16348639 HTTP/1.1" 200 None 2026-03-31 08:47:46,899 - lava_callback - ERROR - Error processing callback for node 69c8e72d6a4c026e16348639: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40164 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:48:09,908 - lava_callback - INFO - Received callback for node 69cb87ea2a64fa9aff93704f INFO: 172.19.0.1:40006 - "POST /node/69cb87ea2a64fa9aff93704f HTTP/1.0" 202 Accepted 2026-03-31 08:48:09,918 - lava_callback - INFO - Starting async job submit for node 69cb87ea2a64fa9aff93704f 2026-03-31 08:48:09,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:09,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ea2a64fa9aff93704f HTTP/1.1" 200 None 2026-03-31 08:48:10,000 - lava_callback - DEBUG - Processing node 69cb87ea2a64fa9aff93704f: ltp-input 2026-03-31 08:48:10,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:10,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:10,798 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69cb87ea2a64fa9aff93704f/log.txt.gz 2026-03-31 08:48:11,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:11,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:11,561 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69cb87ea2a64fa9aff93704f/lava-logs.yaml 2026-03-31 08:48:11,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:11,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:11,871 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69cb87ea2a64fa9aff93704f/lava_callback.json.gz 2026-03-31 08:48:11,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:11,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ea2a64fa9aff93704f HTTP/1.1" 200 None 2026-03-31 08:48:11,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:11,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:48:11,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:12,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87ea2a64fa9aff93704f HTTP/1.1" 200 None 2026-03-31 08:48:12,023 - lava_callback - INFO - Completed processing callback for node 69cb87ea2a64fa9aff93704f INFO: 172.19.0.1:40010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40830 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:48:26,753 - lava_callback - INFO - Received callback for node 69c8e7536a4c026e16348646 INFO: 172.19.0.1:40838 - "POST /node/69c8e7536a4c026e16348646 HTTP/1.0" 202 Accepted 2026-03-31 08:48:26,759 - lava_callback - INFO - Starting async job submit for node 69c8e7536a4c026e16348646 2026-03-31 08:48:26,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:26,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7536a4c026e16348646 HTTP/1.1" 200 None 2026-03-31 08:48:26,782 - lava_callback - DEBUG - Processing node 69c8e7536a4c026e16348646: baseline-arm-clabbe 2026-03-31 08:48:26,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e7536a4c026e16348646 2026-03-31 08:48:26,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:26,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7536a4c026e16348646 HTTP/1.1" 200 None 2026-03-31 08:48:26,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:26,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:48:26,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:26,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e7536a4c026e16348646 HTTP/1.1" 200 None 2026-03-31 08:48:26,872 - lava_callback - ERROR - Error processing callback for node 69c8e7536a4c026e16348646: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44162 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:48:37,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:37,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54264 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:48:49,902 - lava_callback - INFO - Received callback for node 69cb87eb2a64fa9aff937052 INFO: 172.19.0.1:53312 - "POST /node/69cb87eb2a64fa9aff937052 HTTP/1.0" 202 Accepted 2026-03-31 08:48:49,908 - lava_callback - INFO - Starting async job submit for node 69cb87eb2a64fa9aff937052 2026-03-31 08:48:49,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:49,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87eb2a64fa9aff937052 HTTP/1.1" 200 None 2026-03-31 08:48:49,962 - lava_callback - DEBUG - Processing node 69cb87eb2a64fa9aff937052: ltp-ipc 2026-03-31 08:48:50,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:51,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:51,037 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69cb87eb2a64fa9aff937052/log.txt.gz 2026-03-31 08:48:51,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:51,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:51,888 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69cb87eb2a64fa9aff937052/lava-logs.yaml 2026-03-31 08:48:51,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:52,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:52,318 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69cb87eb2a64fa9aff937052/lava_callback.json.gz 2026-03-31 08:48:52,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:52,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87eb2a64fa9aff937052 HTTP/1.1" 200 None 2026-03-31 08:48:52,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:52,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:48:52,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:52,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87eb2a64fa9aff937052 HTTP/1.1" 200 None 2026-03-31 08:48:52,492 - lava_callback - INFO - Completed processing callback for node 69cb87eb2a64fa9aff937052 INFO: 172.19.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42704 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:49:06,823 - lava_callback - INFO - Received callback for node 69c8e7796a4c026e16348653 INFO: 172.19.0.1:42712 - "POST /node/69c8e7796a4c026e16348653 HTTP/1.0" 202 Accepted 2026-03-31 08:49:06,831 - lava_callback - INFO - Starting async job submit for node 69c8e7796a4c026e16348653 2026-03-31 08:49:06,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:06,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7796a4c026e16348653 HTTP/1.1" 200 None 2026-03-31 08:49:06,854 - lava_callback - DEBUG - Processing node 69c8e7796a4c026e16348653: baseline-arm-clabbe 2026-03-31 08:49:06,854 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e7796a4c026e16348653 2026-03-31 08:49:06,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:06,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7796a4c026e16348653 HTTP/1.1" 200 None 2026-03-31 08:49:06,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:06,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:49:06,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:06,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e7796a4c026e16348653 HTTP/1.1" 200 None 2026-03-31 08:49:06,926 - lava_callback - ERROR - Error processing callback for node 69c8e7796a4c026e16348653: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:49:07,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:07,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:58334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36434 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:49:46,778 - lava_callback - INFO - Received callback for node 69c8e7a06a4c026e16348666 INFO: 172.19.0.1:36438 - "POST /node/69c8e7a06a4c026e16348666 HTTP/1.0" 202 Accepted 2026-03-31 08:49:46,784 - lava_callback - INFO - Starting async job submit for node 69c8e7a06a4c026e16348666 2026-03-31 08:49:46,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:46,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7a06a4c026e16348666 HTTP/1.1" 200 None 2026-03-31 08:49:46,801 - lava_callback - DEBUG - Processing node 69c8e7a06a4c026e16348666: baseline-arm-clabbe 2026-03-31 08:49:46,802 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e7a06a4c026e16348666 2026-03-31 08:49:46,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:46,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7a06a4c026e16348666 HTTP/1.1" 200 None 2026-03-31 08:49:46,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:46,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:49:46,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:46,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e7a06a4c026e16348666 HTTP/1.1" 200 None 2026-03-31 08:49:46,904 - lava_callback - ERROR - Error processing callback for node 69c8e7a06a4c026e16348666: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54302 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:50:26,872 - lava_callback - INFO - Received callback for node 69c8e7c66a4c026e16348688 INFO: 172.19.0.1:54308 - "POST /node/69c8e7c66a4c026e16348688 HTTP/1.0" 202 Accepted 2026-03-31 08:50:26,879 - lava_callback - INFO - Starting async job submit for node 69c8e7c66a4c026e16348688 2026-03-31 08:50:26,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:26,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7c66a4c026e16348688 HTTP/1.1" 200 None 2026-03-31 08:50:26,905 - lava_callback - DEBUG - Processing node 69c8e7c66a4c026e16348688: baseline-arm-clabbe 2026-03-31 08:50:26,905 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e7c66a4c026e16348688 2026-03-31 08:50:26,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:26,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7c66a4c026e16348688 HTTP/1.1" 200 None 2026-03-31 08:50:26,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:26,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:50:26,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:26,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e7c66a4c026e16348688 HTTP/1.1" 200 None 2026-03-31 08:50:26,981 - lava_callback - ERROR - Error processing callback for node 69c8e7c66a4c026e16348688: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40842 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:50:50,315 - lava_callback - INFO - Received callback for node 69cb8ad82a64fa9aff93720c INFO: 172.19.0.1:52038 - "POST /node/69cb8ad82a64fa9aff93720c HTTP/1.0" 202 Accepted 2026-03-31 08:50:50,324 - lava_callback - INFO - Starting async job submit for node 69cb8ad82a64fa9aff93720c 2026-03-31 08:50:50,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:50,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8ad82a64fa9aff93720c HTTP/1.1" 200 None 2026-03-31 08:50:50,359 - lava_callback - DEBUG - Processing node 69cb8ad82a64fa9aff93720c: baseline-arm 2026-03-31 08:50:50,367 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:50:50,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:50:50,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:50:50,691 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb8ad82a64fa9aff93720c/lava-logs.yaml 2026-03-31 08:50:50,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:50:50,694 - lava_callback - INFO - Received callback for node 69cb87ed2a64fa9aff937055 INFO: 172.19.0.1:52050 - "POST /node/69cb87ed2a64fa9aff937055 HTTP/1.0" 202 Accepted 2026-03-31 08:50:50,702 - lava_callback - INFO - Starting async job submit for node 69cb87ed2a64fa9aff937055 2026-03-31 08:50:50,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:50,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ed2a64fa9aff937055 HTTP/1.1" 200 None 2026-03-31 08:50:50,746 - lava_callback - DEBUG - Processing node 69cb87ed2a64fa9aff937055: ltp-pty 2026-03-31 08:50:50,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:50:50,934 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb8ad82a64fa9aff93720c/lava_callback.json.gz 2026-03-31 08:50:50,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:51,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8ad82a64fa9aff93720c HTTP/1.1" 200 None 2026-03-31 08:50:51,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:51,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:50:51,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:51,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8ad82a64fa9aff93720c HTTP/1.1" 200 None 2026-03-31 08:50:51,246 - lava_callback - INFO - Completed processing callback for node 69cb8ad82a64fa9aff93720c 2026-03-31 08:50:51,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:50:51,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:50:51,640 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69cb87ed2a64fa9aff937055/log.txt.gz 2026-03-31 08:50:51,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:50:52,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:50:52,301 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69cb87ed2a64fa9aff937055/lava-logs.yaml 2026-03-31 08:50:52,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:50:52,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:50:52,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69cb87ed2a64fa9aff937055/lava_callback.json.gz 2026-03-31 08:50:52,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:52,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ed2a64fa9aff937055 HTTP/1.1" 200 None 2026-03-31 08:50:52,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:52,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:50:52,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:52,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87ed2a64fa9aff937055 HTTP/1.1" 200 None 2026-03-31 08:50:52,573 - lava_callback - INFO - Completed processing callback for node 69cb87ed2a64fa9aff937055 INFO: 172.19.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42632 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:51:07,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:07,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:49626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34902 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:51:30,047 - lava_callback - INFO - Received callback for node 69cb87ef2a64fa9aff93705b INFO: 172.19.0.1:56186 - "POST /node/69cb87ef2a64fa9aff93705b HTTP/1.0" 202 Accepted 2026-03-31 08:51:30,054 - lava_callback - INFO - Starting async job submit for node 69cb87ef2a64fa9aff93705b 2026-03-31 08:51:30,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:30,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ef2a64fa9aff93705b HTTP/1.1" 200 None 2026-03-31 08:51:30,095 - lava_callback - DEBUG - Processing node 69cb87ef2a64fa9aff93705b: ltp-sched 2026-03-31 08:51:30,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:51:30,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:51:30,622 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69cb87ef2a64fa9aff93705b/log.txt.gz 2026-03-31 08:51:30,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:51:31,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:51:31,037 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69cb87ef2a64fa9aff93705b/lava-logs.yaml 2026-03-31 08:51:31,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:51:31,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:51:31,213 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69cb87ef2a64fa9aff93705b/lava_callback.json.gz 2026-03-31 08:51:31,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:31,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ef2a64fa9aff93705b HTTP/1.1" 200 None 2026-03-31 08:51:31,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:31,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:51:31,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:31,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87ef2a64fa9aff93705b HTTP/1.1" 200 None 2026-03-31 08:51:31,305 - lava_callback - INFO - Completed processing callback for node 69cb87ef2a64fa9aff93705b INFO: 172.19.0.1:56194 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:51:37,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:37,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:43892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46450 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:52:06,849 - lava_callback - INFO - Received callback for node 69c8e82d6a4c026e1634872a INFO: 172.19.0.1:46462 - "POST /node/69c8e82d6a4c026e1634872a HTTP/1.0" 202 Accepted 2026-03-31 08:52:06,855 - lava_callback - INFO - Starting async job submit for node 69c8e82d6a4c026e1634872a 2026-03-31 08:52:06,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:06,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e82d6a4c026e1634872a HTTP/1.1" 200 None 2026-03-31 08:52:06,872 - lava_callback - DEBUG - Processing node 69c8e82d6a4c026e1634872a: baseline-arm64-baylibre 2026-03-31 08:52:06,872 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e82d6a4c026e1634872a 2026-03-31 08:52:06,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:06,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e82d6a4c026e1634872a HTTP/1.1" 200 None 2026-03-31 08:52:06,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:06,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:06,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:06,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e82d6a4c026e1634872a HTTP/1.1" 200 None 2026-03-31 08:52:06,934 - lava_callback - ERROR - Error processing callback for node 69c8e82d6a4c026e1634872a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:07,492 - lava_callback - INFO - Received callback for node 69c8e82f6a4c026e1634872d INFO: 172.19.0.1:46472 - "POST /node/69c8e82f6a4c026e1634872d HTTP/1.0" 202 Accepted 2026-03-31 08:52:07,497 - lava_callback - INFO - Starting async job submit for node 69c8e82f6a4c026e1634872d 2026-03-31 08:52:07,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:07,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e82f6a4c026e1634872d HTTP/1.1" 200 None 2026-03-31 08:52:07,513 - lava_callback - DEBUG - Processing node 69c8e82f6a4c026e1634872d: baseline-arm64-baylibre 2026-03-31 08:52:07,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e82f6a4c026e1634872d 2026-03-31 08:52:07,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:07,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e82f6a4c026e1634872d HTTP/1.1" 200 None 2026-03-31 08:52:07,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:07,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:07,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:07,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e82f6a4c026e1634872d HTTP/1.1" 200 None 2026-03-31 08:52:07,569 - lava_callback - ERROR - Error processing callback for node 69c8e82f6a4c026e1634872d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:08,135 - lava_callback - INFO - Received callback for node 69c8e8306a4c026e16348730 INFO: 172.19.0.1:46482 - "POST /node/69c8e8306a4c026e16348730 HTTP/1.0" 202 Accepted 2026-03-31 08:52:08,143 - lava_callback - INFO - Starting async job submit for node 69c8e8306a4c026e16348730 2026-03-31 08:52:08,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8306a4c026e16348730 HTTP/1.1" 200 None 2026-03-31 08:52:08,163 - lava_callback - DEBUG - Processing node 69c8e8306a4c026e16348730: baseline-arm64-baylibre 2026-03-31 08:52:08,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8306a4c026e16348730 2026-03-31 08:52:08,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8306a4c026e16348730 HTTP/1.1" 200 None 2026-03-31 08:52:08,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:08,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8306a4c026e16348730 HTTP/1.1" 200 None 2026-03-31 08:52:08,225 - lava_callback - ERROR - Error processing callback for node 69c8e8306a4c026e16348730: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:08,822 - lava_callback - INFO - Received callback for node 69c8e8326a4c026e16348754 INFO: 172.19.0.1:46488 - "POST /node/69c8e8326a4c026e16348754 HTTP/1.0" 202 Accepted 2026-03-31 08:52:08,827 - lava_callback - INFO - Starting async job submit for node 69c8e8326a4c026e16348754 2026-03-31 08:52:08,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8326a4c026e16348754 HTTP/1.1" 200 None 2026-03-31 08:52:08,844 - lava_callback - DEBUG - Processing node 69c8e8326a4c026e16348754: baseline-arm64-baylibre 2026-03-31 08:52:08,844 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8326a4c026e16348754 2026-03-31 08:52:08,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8326a4c026e16348754 HTTP/1.1" 200 None 2026-03-31 08:52:08,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:08,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8326a4c026e16348754 HTTP/1.1" 200 None 2026-03-31 08:52:08,904 - lava_callback - ERROR - Error processing callback for node 69c8e8326a4c026e16348754: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:09,490 - lava_callback - INFO - Received callback for node 69c8e8346a4c026e16348757 INFO: 172.19.0.1:46496 - "POST /node/69c8e8346a4c026e16348757 HTTP/1.0" 202 Accepted 2026-03-31 08:52:09,495 - lava_callback - INFO - Starting async job submit for node 69c8e8346a4c026e16348757 2026-03-31 08:52:09,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:09,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8346a4c026e16348757 HTTP/1.1" 200 None 2026-03-31 08:52:09,512 - lava_callback - DEBUG - Processing node 69c8e8346a4c026e16348757: baseline-arm64-baylibre 2026-03-31 08:52:09,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8346a4c026e16348757 2026-03-31 08:52:09,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:09,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8346a4c026e16348757 HTTP/1.1" 200 None 2026-03-31 08:52:09,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:09,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:09,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:09,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8346a4c026e16348757 HTTP/1.1" 200 None 2026-03-31 08:52:09,569 - lava_callback - ERROR - Error processing callback for node 69c8e8346a4c026e16348757: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:09,933 - lava_callback - INFO - Received callback for node 69cb87f02a64fa9aff937062 INFO: 172.19.0.1:52678 - "POST /node/69cb87f02a64fa9aff937062 HTTP/1.0" 202 Accepted 2026-03-31 08:52:09,940 - lava_callback - INFO - Starting async job submit for node 69cb87f02a64fa9aff937062 2026-03-31 08:52:09,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:09,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f02a64fa9aff937062 HTTP/1.1" 200 None 2026-03-31 08:52:09,990 - lava_callback - DEBUG - Processing node 69cb87f02a64fa9aff937062: ltp-smoke 2026-03-31 08:52:10,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:10,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:10,521 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87f02a64fa9aff937062/log.txt.gz 2026-03-31 08:52:10,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:10,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:10,951 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87f02a64fa9aff937062/lava-logs.yaml 2026-03-31 08:52:10,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:11,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:11,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87f02a64fa9aff937062/lava_callback.json.gz 2026-03-31 08:52:11,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:11,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f02a64fa9aff937062 HTTP/1.1" 200 None 2026-03-31 08:52:11,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:11,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:52:11,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:11,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87f02a64fa9aff937062 HTTP/1.1" 200 None 2026-03-31 08:52:11,279 - lava_callback - INFO - Completed processing callback for node 69cb87f02a64fa9aff937062 INFO: 172.19.0.1:52682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38840 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:52:26,886 - lava_callback - INFO - Received callback for node 69c8e8356a4c026e1634875a INFO: 172.19.0.1:38854 - "POST /node/69c8e8356a4c026e1634875a HTTP/1.0" 202 Accepted 2026-03-31 08:52:26,892 - lava_callback - INFO - Starting async job submit for node 69c8e8356a4c026e1634875a 2026-03-31 08:52:26,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8356a4c026e1634875a HTTP/1.1" 200 None 2026-03-31 08:52:26,911 - lava_callback - DEBUG - Processing node 69c8e8356a4c026e1634875a: baseline-arm64-baylibre 2026-03-31 08:52:26,911 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8356a4c026e1634875a 2026-03-31 08:52:26,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8356a4c026e1634875a HTTP/1.1" 200 None 2026-03-31 08:52:26,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,940 - lava_callback - INFO - Received callback for node 69c8e8446a4c026e16348780 INFO: 172.19.0.1:38866 - "POST /node/69c8e8446a4c026e16348780 HTTP/1.0" 202 Accepted 2026-03-31 08:52:26,946 - lava_callback - INFO - Starting async job submit for node 69c8e8446a4c026e16348780 2026-03-31 08:52:26,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:26,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8446a4c026e16348780 HTTP/1.1" 200 None 2026-03-31 08:52:26,964 - lava_callback - DEBUG - Processing node 69c8e8446a4c026e16348780: baseline-arm64-clabbe 2026-03-31 08:52:26,964 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8446a4c026e16348780 2026-03-31 08:52:26,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8356a4c026e1634875a HTTP/1.1" 200 None 2026-03-31 08:52:26,979 - lava_callback - ERROR - Error processing callback for node 69c8e8356a4c026e1634875a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:27,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8446a4c026e16348780 HTTP/1.1" 200 None 2026-03-31 08:52:27,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:27,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8446a4c026e16348780 HTTP/1.1" 200 None 2026-03-31 08:52:27,122 - lava_callback - ERROR - Error processing callback for node 69c8e8446a4c026e16348780: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:27,623 - lava_callback - INFO - Received callback for node 69c8e8466a4c026e16348783 INFO: 172.19.0.1:38882 - "POST /node/69c8e8466a4c026e16348783 HTTP/1.0" 202 Accepted 2026-03-31 08:52:27,629 - lava_callback - INFO - Starting async job submit for node 69c8e8466a4c026e16348783 2026-03-31 08:52:27,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8466a4c026e16348783 HTTP/1.1" 200 None 2026-03-31 08:52:27,648 - lava_callback - DEBUG - Processing node 69c8e8466a4c026e16348783: baseline-arm64-clabbe 2026-03-31 08:52:27,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8466a4c026e16348783 2026-03-31 08:52:27,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8466a4c026e16348783 HTTP/1.1" 200 None 2026-03-31 08:52:27,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:27,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8466a4c026e16348783 HTTP/1.1" 200 None 2026-03-31 08:52:27,722 - lava_callback - ERROR - Error processing callback for node 69c8e8466a4c026e16348783: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:28,304 - lava_callback - INFO - Received callback for node 69c8e8476a4c026e16348786 INFO: 172.19.0.1:38884 - "POST /node/69c8e8476a4c026e16348786 HTTP/1.0" 202 Accepted 2026-03-31 08:52:28,312 - lava_callback - INFO - Starting async job submit for node 69c8e8476a4c026e16348786 2026-03-31 08:52:28,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:28,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8476a4c026e16348786 HTTP/1.1" 200 None 2026-03-31 08:52:28,330 - lava_callback - DEBUG - Processing node 69c8e8476a4c026e16348786: baseline-arm64-clabbe 2026-03-31 08:52:28,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8476a4c026e16348786 2026-03-31 08:52:28,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:28,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8476a4c026e16348786 HTTP/1.1" 200 None 2026-03-31 08:52:28,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:28,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:28,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:28,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8476a4c026e16348786 HTTP/1.1" 200 None 2026-03-31 08:52:28,396 - lava_callback - ERROR - Error processing callback for node 69c8e8476a4c026e16348786: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:28,986 - lava_callback - INFO - Received callback for node 69c8e8486a4c026e16348789 INFO: 172.19.0.1:38890 - "POST /node/69c8e8486a4c026e16348789 HTTP/1.0" 202 Accepted 2026-03-31 08:52:28,992 - lava_callback - INFO - Starting async job submit for node 69c8e8486a4c026e16348789 2026-03-31 08:52:28,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:29,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8486a4c026e16348789 HTTP/1.1" 200 None 2026-03-31 08:52:29,009 - lava_callback - DEBUG - Processing node 69c8e8486a4c026e16348789: baseline-arm64-clabbe 2026-03-31 08:52:29,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8486a4c026e16348789 2026-03-31 08:52:29,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:29,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8486a4c026e16348789 HTTP/1.1" 200 None 2026-03-31 08:52:29,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:29,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:29,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:29,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8486a4c026e16348789 HTTP/1.1" 200 None 2026-03-31 08:52:29,079 - lava_callback - ERROR - Error processing callback for node 69c8e8486a4c026e16348789: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60690 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:52:37,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:37,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:40078 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:52:46,901 - lava_callback - INFO - Received callback for node 69c8e84a6a4c026e163487a4 INFO: 172.19.0.1:40094 - "POST /node/69c8e84a6a4c026e163487a4 HTTP/1.0" 202 Accepted 2026-03-31 08:52:46,907 - lava_callback - INFO - Starting async job submit for node 69c8e84a6a4c026e163487a4 2026-03-31 08:52:46,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:46,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e84a6a4c026e163487a4 HTTP/1.1" 200 None 2026-03-31 08:52:46,924 - lava_callback - DEBUG - Processing node 69c8e84a6a4c026e163487a4: baseline-arm64-clabbe 2026-03-31 08:52:46,924 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e84a6a4c026e163487a4 2026-03-31 08:52:46,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:46,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e84a6a4c026e163487a4 HTTP/1.1" 200 None 2026-03-31 08:52:46,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:46,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:46,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:46,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e84a6a4c026e163487a4 HTTP/1.1" 200 None 2026-03-31 08:52:46,977 - lava_callback - ERROR - Error processing callback for node 69c8e84a6a4c026e163487a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40096 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:52:47,580 - lava_callback - INFO - Received callback for node 69c8e8496a4c026e1634879e INFO: 172.19.0.1:40104 - "POST /node/69c8e8496a4c026e1634879e HTTP/1.0" 202 Accepted 2026-03-31 08:52:47,586 - lava_callback - INFO - Starting async job submit for node 69c8e8496a4c026e1634879e 2026-03-31 08:52:47,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:47,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8496a4c026e1634879e HTTP/1.1" 200 None 2026-03-31 08:52:47,607 - lava_callback - DEBUG - Processing node 69c8e8496a4c026e1634879e: baseline-arm64-clabbe 2026-03-31 08:52:47,607 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8496a4c026e1634879e 2026-03-31 08:52:47,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:47,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8496a4c026e1634879e HTTP/1.1" 200 None 2026-03-31 08:52:47,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:47,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:47,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:47,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8496a4c026e1634879e HTTP/1.1" 200 None 2026-03-31 08:52:47,665 - lava_callback - ERROR - Error processing callback for node 69c8e8496a4c026e1634879e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:50,029 - lava_callback - INFO - Received callback for node 69cb87f32a64fa9aff937068 INFO: 172.19.0.1:36986 - "POST /node/69cb87f32a64fa9aff937068 HTTP/1.0" 202 Accepted 2026-03-31 08:52:50,036 - lava_callback - INFO - Starting async job submit for node 69cb87f32a64fa9aff937068 2026-03-31 08:52:50,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:50,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f32a64fa9aff937068 HTTP/1.1" 200 None 2026-03-31 08:52:50,075 - lava_callback - DEBUG - Processing node 69cb87f32a64fa9aff937068: ltp-syscalls-ipc 2026-03-31 08:52:50,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:50,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:50,550 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cb87f32a64fa9aff937068/log.txt.gz 2026-03-31 08:52:50,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:51,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:51,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cb87f32a64fa9aff937068/lava-logs.yaml 2026-03-31 08:52:51,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:51,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:51,184 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cb87f32a64fa9aff937068/lava_callback.json.gz 2026-03-31 08:52:51,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:51,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f32a64fa9aff937068 HTTP/1.1" 200 None 2026-03-31 08:52:51,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:51,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:52:51,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:51,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87f32a64fa9aff937068 HTTP/1.1" 200 None 2026-03-31 08:52:51,282 - lava_callback - INFO - Completed processing callback for node 69cb87f32a64fa9aff937068 INFO: 172.19.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42414 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:53:07,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:07,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:44606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42000 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:53:27,371 - lava_callback - INFO - Received callback for node 69cb87a62a64fa9aff936f73 INFO: 172.19.0.1:42002 - "POST /node/69cb87a62a64fa9aff936f73 HTTP/1.0" 202 Accepted 2026-03-31 08:53:27,383 - lava_callback - INFO - Starting async job submit for node 69cb87a62a64fa9aff936f73 2026-03-31 08:53:27,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:27,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a62a64fa9aff936f73 HTTP/1.1" 200 None 2026-03-31 08:53:27,502 - lava_callback - DEBUG - Processing node 69cb87a62a64fa9aff936f73: baseline-arm 2026-03-31 08:53:28,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:53:32,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:53:32,656 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a62a64fa9aff936f73/log.txt.gz 2026-03-31 08:53:33,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:53:33,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:53:33,750 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a62a64fa9aff936f73/lava-logs.yaml 2026-03-31 08:53:33,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:53:33,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:53:33,964 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a62a64fa9aff936f73/lava_callback.json.gz DEBUG: Setup failed for 69cb87a62a64fa9aff936f73. Transitting job node result: incomplete -> fail 2026-03-31 08:53:34,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:34,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a62a64fa9aff936f73 HTTP/1.1" 200 None 2026-03-31 08:53:34,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:34,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:53:34,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:34,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a62a64fa9aff936f73 HTTP/1.1" 200 None 2026-03-31 08:53:34,360 - lava_callback - INFO - Completed processing callback for node 69cb87a62a64fa9aff936f73 INFO: 172.19.0.1:36370 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:53:37,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:37,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:45386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51288 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:55:10,854 - lava_callback - INFO - Received callback for node 69cb87f52a64fa9aff93706b INFO: 172.19.0.1:37982 - "POST /node/69cb87f52a64fa9aff93706b HTTP/1.0" 202 Accepted 2026-03-31 08:55:10,864 - lava_callback - INFO - Starting async job submit for node 69cb87f52a64fa9aff93706b 2026-03-31 08:55:10,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:10,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f52a64fa9aff93706b HTTP/1.1" 200 None 2026-03-31 08:55:10,920 - lava_callback - DEBUG - Processing node 69cb87f52a64fa9aff93706b: ltp-timers 2026-03-31 08:55:11,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:11,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:11,912 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69cb87f52a64fa9aff93706b/log.txt.gz 2026-03-31 08:55:12,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:12,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:12,597 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69cb87f52a64fa9aff93706b/lava-logs.yaml 2026-03-31 08:55:12,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:13,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:13,194 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69cb87f52a64fa9aff93706b/lava_callback.json.gz 2026-03-31 08:55:13,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:13,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f52a64fa9aff93706b HTTP/1.1" 200 None 2026-03-31 08:55:13,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:13,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:55:13,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:13,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87f52a64fa9aff93706b HTTP/1.1" 200 None 2026-03-31 08:55:13,336 - lava_callback - INFO - Completed processing callback for node 69cb87f52a64fa9aff93706b INFO: 172.19.0.1:37984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54916 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:55:27,243 - lava_callback - INFO - Received callback for node 69cb8bdc2a64fa9aff9372ab INFO: 172.19.0.1:54930 - "POST /node/69cb8bdc2a64fa9aff9372ab HTTP/1.0" 202 Accepted 2026-03-31 08:55:27,250 - lava_callback - INFO - Starting async job submit for node 69cb8bdc2a64fa9aff9372ab 2026-03-31 08:55:27,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:27,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdc2a64fa9aff9372ab HTTP/1.1" 200 None 2026-03-31 08:55:27,307 - lava_callback - DEBUG - Processing node 69cb8bdc2a64fa9aff9372ab: baseline-arm64 2026-03-31 08:55:27,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:27,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:27,767 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdc2a64fa9aff9372ab/log.txt.gz 2026-03-31 08:55:27,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:28,189 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:28,191 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdc2a64fa9aff9372ab/lava-logs.yaml 2026-03-31 08:55:28,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:28,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:28,409 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdc2a64fa9aff9372ab/lava_callback.json.gz 2026-03-31 08:55:28,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:28,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdc2a64fa9aff9372ab HTTP/1.1" 200 None 2026-03-31 08:55:28,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:28,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:55:28,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:28,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bdc2a64fa9aff9372ab HTTP/1.1" 200 None 2026-03-31 08:55:28,605 - lava_callback - INFO - Completed processing callback for node 69cb8bdc2a64fa9aff9372ab 2026-03-31 08:55:29,062 - lava_callback - INFO - Received callback for node 69cb8bfc2a64fa9aff93731b INFO: 172.19.0.1:54938 - "POST /node/69cb8bfc2a64fa9aff93731b HTTP/1.0" 202 Accepted 2026-03-31 08:55:29,071 - lava_callback - INFO - Starting async job submit for node 69cb8bfc2a64fa9aff93731b 2026-03-31 08:55:29,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:29,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfc2a64fa9aff93731b HTTP/1.1" 200 None 2026-03-31 08:55:29,091 - lava_callback - DEBUG - Processing node 69cb8bfc2a64fa9aff93731b: baseline-nfs-arm64 2026-03-31 08:55:29,093 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:55:29,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:29,218 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:29,219 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfc2a64fa9aff93731b/lava-logs.yaml 2026-03-31 08:55:29,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:29,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:29,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfc2a64fa9aff93731b/lava_callback.json.gz 2026-03-31 08:55:29,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:29,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfc2a64fa9aff93731b HTTP/1.1" 200 None 2026-03-31 08:55:29,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:29,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:55:29,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:29,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bfc2a64fa9aff93731b HTTP/1.1" 200 None 2026-03-31 08:55:29,491 - lava_callback - INFO - Completed processing callback for node 69cb8bfc2a64fa9aff93731b 2026-03-31 08:55:30,269 - lava_callback - INFO - Received callback for node 69cb8bea2a64fa9aff9372cc INFO: 172.19.0.1:44438 - "POST /node/69cb8bea2a64fa9aff9372cc HTTP/1.0" 202 Accepted 2026-03-31 08:55:30,279 - lava_callback - INFO - Starting async job submit for node 69cb8bea2a64fa9aff9372cc 2026-03-31 08:55:30,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:30,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bea2a64fa9aff9372cc HTTP/1.1" 200 None 2026-03-31 08:55:30,374 - lava_callback - DEBUG - Processing node 69cb8bea2a64fa9aff9372cc: baseline-arm64 2026-03-31 08:55:30,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:30,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:30,830 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bea2a64fa9aff9372cc/log.txt.gz 2026-03-31 08:55:31,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:31,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:31,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bea2a64fa9aff9372cc/lava-logs.yaml 2026-03-31 08:55:31,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:31,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:31,598 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bea2a64fa9aff9372cc/lava_callback.json.gz 2026-03-31 08:55:31,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:31,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bea2a64fa9aff9372cc HTTP/1.1" 200 None 2026-03-31 08:55:31,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:31,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:55:31,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:31,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bea2a64fa9aff9372cc HTTP/1.1" 200 None 2026-03-31 08:55:31,750 - lava_callback - INFO - Completed processing callback for node 69cb8bea2a64fa9aff9372cc INFO: 172.19.0.1:44446 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:55:37,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:37,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.19.0.1:37500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37502 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:55:50,191 - lava_callback - INFO - Received callback for node 69cb87f62a64fa9aff93706e INFO: 172.19.0.1:32848 - "POST /node/69cb87f62a64fa9aff93706e HTTP/1.0" 202 Accepted 2026-03-31 08:55:50,203 - lava_callback - INFO - Starting async job submit for node 69cb87f62a64fa9aff93706e 2026-03-31 08:55:50,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:50,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f62a64fa9aff93706e HTTP/1.1" 200 None 2026-03-31 08:55:50,278 - lava_callback - DEBUG - Processing node 69cb87f62a64fa9aff93706e: ltp-watchqueue 2026-03-31 08:55:50,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:51,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:51,076 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cb87f62a64fa9aff93706e/log.txt.gz 2026-03-31 08:55:51,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:51,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:51,683 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cb87f62a64fa9aff93706e/lava-logs.yaml 2026-03-31 08:55:51,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:51,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:51,841 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cb87f62a64fa9aff93706e/lava_callback.json.gz 2026-03-31 08:55:51,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:51,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f62a64fa9aff93706e HTTP/1.1" 200 None 2026-03-31 08:55:51,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:51,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:55:51,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:51,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87f62a64fa9aff93706e HTTP/1.1" 200 None 2026-03-31 08:55:51,935 - lava_callback - INFO - Completed processing callback for node 69cb87f62a64fa9aff93706e INFO: 172.19.0.1:32860 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:06,443 - lava_callback - INFO - Received callback for node 69cb8bde2a64fa9aff9372b1 INFO: 172.19.0.1:41456 - "POST /node/69cb8bde2a64fa9aff9372b1 HTTP/1.0" 202 Accepted 2026-03-31 08:56:06,452 - lava_callback - INFO - Starting async job submit for node 69cb8bde2a64fa9aff9372b1 2026-03-31 08:56:06,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:41470 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:06,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bde2a64fa9aff9372b1 HTTP/1.1" 200 None 2026-03-31 08:56:06,543 - lava_callback - DEBUG - Processing node 69cb8bde2a64fa9aff9372b1: baseline-arm64 2026-03-31 08:56:06,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:07,082 - lava_callback - INFO - Received callback for node 69c8e9226a4c026e16348909 INFO: 172.19.0.1:41474 - "POST /node/69c8e9226a4c026e16348909 HTTP/1.0" 202 Accepted 2026-03-31 08:56:07,100 - lava_callback - INFO - Starting async job submit for node 69c8e9226a4c026e16348909 2026-03-31 08:56:07,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:07,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:07,129 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bde2a64fa9aff9372b1/log.txt.gz 2026-03-31 08:56:07,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9226a4c026e16348909 HTTP/1.1" 200 None 2026-03-31 08:56:07,145 - lava_callback - DEBUG - Processing node 69c8e9226a4c026e16348909: baseline-arm-baylibre 2026-03-31 08:56:07,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e9226a4c026e16348909 2026-03-31 08:56:07,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:07,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9226a4c026e16348909 HTTP/1.1" 200 None 2026-03-31 08:56:07,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:07,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 08:56:07,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:07,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e9226a4c026e16348909 HTTP/1.1" 200 None 2026-03-31 08:56:07,376 - lava_callback - ERROR - Error processing callback for node 69c8e9226a4c026e16348909: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:56:07,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:07,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:07,821 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bde2a64fa9aff9372b1/lava-logs.yaml 2026-03-31 08:56:07,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:07,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:07,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:56:08,006 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:08,009 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bde2a64fa9aff9372b1/lava_callback.json.gz 2026-03-31 08:56:08,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:08,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bde2a64fa9aff9372b1 HTTP/1.1" 200 None 2026-03-31 08:56:08,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:08,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:08,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:08,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bde2a64fa9aff9372b1 HTTP/1.1" 200 None 2026-03-31 08:56:08,120 - lava_callback - INFO - Completed processing callback for node 69cb8bde2a64fa9aff9372b1 2026-03-31 08:56:13,786 - lava_callback - INFO - Received callback for node 69cb8be62a64fa9aff9372c3 INFO: 172.19.0.1:34914 - "POST /node/69cb8be62a64fa9aff9372c3 HTTP/1.0" 202 Accepted 2026-03-31 08:56:13,796 - lava_callback - INFO - Starting async job submit for node 69cb8be62a64fa9aff9372c3 2026-03-31 08:56:13,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:13,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be62a64fa9aff9372c3 HTTP/1.1" 200 None 2026-03-31 08:56:13,864 - lava_callback - DEBUG - Processing node 69cb8be62a64fa9aff9372c3: baseline-arm64 2026-03-31 08:56:14,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:14,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:14,692 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be62a64fa9aff9372c3/log.txt.gz 2026-03-31 08:56:14,798 - lava_callback - INFO - Received callback for node 69cb8be22a64fa9aff9372ba INFO: 172.19.0.1:34916 - "POST /node/69cb8be22a64fa9aff9372ba HTTP/1.0" 202 Accepted 2026-03-31 08:56:14,829 - lava_callback - INFO - Starting async job submit for node 69cb8be22a64fa9aff9372ba 2026-03-31 08:56:14,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:15,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be22a64fa9aff9372ba HTTP/1.1" 200 None 2026-03-31 08:56:15,054 - lava_callback - DEBUG - Processing node 69cb8be22a64fa9aff9372ba: baseline-arm64 INFO: 172.19.0.1:34924 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 08:56:15,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:15,423 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:56:15,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:15,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:15,727 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be62a64fa9aff9372c3/lava-logs.yaml 2026-03-31 08:56:15,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:16,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:16,031 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be22a64fa9aff9372ba/lava-logs.yaml 2026-03-31 08:56:16,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:16,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:16,139 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be62a64fa9aff9372c3/lava_callback.json.gz 2026-03-31 08:56:16,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be62a64fa9aff9372c3 HTTP/1.1" 200 None 2026-03-31 08:56:16,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:16,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:16,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be22a64fa9aff9372ba/lava_callback.json.gz 2026-03-31 08:56:16,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be62a64fa9aff9372c3 HTTP/1.1" 200 None 2026-03-31 08:56:16,350 - lava_callback - INFO - Completed processing callback for node 69cb8be62a64fa9aff9372c3 2026-03-31 08:56:16,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be22a64fa9aff9372ba HTTP/1.1" 200 None 2026-03-31 08:56:16,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:16,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be22a64fa9aff9372ba HTTP/1.1" 200 None INFO: 172.19.0.1:34938 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:16,525 - lava_callback - INFO - Completed processing callback for node 69cb8be22a64fa9aff9372ba 2026-03-31 08:56:16,589 - lava_callback - INFO - Received callback for node 69cb8be02a64fa9aff9372b7 INFO: 172.19.0.1:34940 - "POST /node/69cb8be02a64fa9aff9372b7 HTTP/1.0" 202 Accepted 2026-03-31 08:56:16,602 - lava_callback - INFO - Starting async job submit for node 69cb8be02a64fa9aff9372b7 2026-03-31 08:56:16,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be02a64fa9aff9372b7 HTTP/1.1" 200 None 2026-03-31 08:56:16,695 - lava_callback - DEBUG - Processing node 69cb8be02a64fa9aff9372b7: baseline-arm64 2026-03-31 08:56:16,974 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:56:17,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:17,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:17,450 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be02a64fa9aff9372b7/lava-logs.yaml 2026-03-31 08:56:17,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:17,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:17,582 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be02a64fa9aff9372b7/lava_callback.json.gz 2026-03-31 08:56:17,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:17,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be02a64fa9aff9372b7 HTTP/1.1" 200 None 2026-03-31 08:56:17,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:17,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:17,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:17,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be02a64fa9aff9372b7 HTTP/1.1" 200 None 2026-03-31 08:56:17,747 - lava_callback - INFO - Completed processing callback for node 69cb8be02a64fa9aff9372b7 2026-03-31 08:56:26,504 - lava_callback - INFO - Received callback for node 69cb8bdf2a64fa9aff9372b4 INFO: 172.19.0.1:37070 - "POST /node/69cb8bdf2a64fa9aff9372b4 HTTP/1.0" 202 Accepted 2026-03-31 08:56:26,513 - lava_callback - INFO - Starting async job submit for node 69cb8bdf2a64fa9aff9372b4 INFO: 172.19.0.1:37086 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:26,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:26,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdf2a64fa9aff9372b4 HTTP/1.1" 200 None 2026-03-31 08:56:26,560 - lava_callback - DEBUG - Processing node 69cb8bdf2a64fa9aff9372b4: baseline-arm64 2026-03-31 08:56:26,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:27,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:27,230 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdf2a64fa9aff9372b4/log.txt.gz 2026-03-31 08:56:27,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:27,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:27,547 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdf2a64fa9aff9372b4/lava-logs.yaml 2026-03-31 08:56:27,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:27,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:27,767 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdf2a64fa9aff9372b4/lava_callback.json.gz 2026-03-31 08:56:27,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:27,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdf2a64fa9aff9372b4 HTTP/1.1" 200 None 2026-03-31 08:56:27,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:27,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:27,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:27,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bdf2a64fa9aff9372b4 HTTP/1.1" 200 None 2026-03-31 08:56:27,864 - lava_callback - INFO - Completed processing callback for node 69cb8bdf2a64fa9aff9372b4 2026-03-31 08:56:28,615 - lava_callback - INFO - Received callback for node 69cb8be72a64fa9aff9372c6 INFO: 172.19.0.1:37090 - "POST /node/69cb8be72a64fa9aff9372c6 HTTP/1.0" 202 Accepted 2026-03-31 08:56:28,622 - lava_callback - INFO - Starting async job submit for node 69cb8be72a64fa9aff9372c6 2026-03-31 08:56:28,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:28,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be72a64fa9aff9372c6 HTTP/1.1" 200 None 2026-03-31 08:56:28,700 - lava_callback - DEBUG - Processing node 69cb8be72a64fa9aff9372c6: baseline-arm64 2026-03-31 08:56:28,903 - lava_callback - INFO - Received callback for node 69cb8bee2a64fa9aff9372d8 INFO: 172.19.0.1:37094 - "POST /node/69cb8bee2a64fa9aff9372d8 HTTP/1.0" 202 Accepted 2026-03-31 08:56:28,923 - lava_callback - INFO - Starting async job submit for node 69cb8bee2a64fa9aff9372d8 2026-03-31 08:56:29,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:29,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bee2a64fa9aff9372d8 HTTP/1.1" 200 None 2026-03-31 08:56:29,139 - lava_callback - DEBUG - Processing node 69cb8bee2a64fa9aff9372d8: baseline-arm64 2026-03-31 08:56:29,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:29,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:29,816 - lava_callback - INFO - Received callback for node 69cb8beb2a64fa9aff9372cf 2026-03-31 08:56:29,828 - lava_callback - INFO - Starting async job submit for node 69cb8beb2a64fa9aff9372cf INFO: 172.19.0.1:47666 - "POST /node/69cb8beb2a64fa9aff9372cf HTTP/1.0" 202 Accepted 2026-03-31 08:56:29,847 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be72a64fa9aff9372c6/log.txt.gz 2026-03-31 08:56:29,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:29,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:29,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8beb2a64fa9aff9372cf HTTP/1.1" 200 None 2026-03-31 08:56:29,956 - lava_callback - DEBUG - Processing node 69cb8beb2a64fa9aff9372cf: baseline-arm64 2026-03-31 08:56:30,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:30,138 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bee2a64fa9aff9372d8/log.txt.gz 2026-03-31 08:56:30,211 - lava_callback - INFO - Received callback for node 69cb8bf02a64fa9aff9372df 2026-03-31 08:56:30,313 - lava_callback - INFO - Starting async job submit for node 69cb8bf02a64fa9aff9372df INFO: 172.19.0.1:47674 - "POST /node/69cb8bf02a64fa9aff9372df HTTP/1.0" 202 Accepted 2026-03-31 08:56:30,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:30,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:30,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf02a64fa9aff9372df HTTP/1.1" 200 None 2026-03-31 08:56:30,620 - lava_callback - DEBUG - Processing node 69cb8bf02a64fa9aff9372df: baseline-arm64 2026-03-31 08:56:31,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:31,377 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be72a64fa9aff9372c6/lava-logs.yaml 2026-03-31 08:56:31,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:31,461 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8beb2a64fa9aff9372cf/log.txt.gz 2026-03-31 08:56:31,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:31,609 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bee2a64fa9aff9372d8/lava-logs.yaml 2026-03-31 08:56:31,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:31,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,703 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be72a64fa9aff9372c6/lava_callback.json.gz 2026-03-31 08:56:31,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:31,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be72a64fa9aff9372c6 HTTP/1.1" 200 None 2026-03-31 08:56:31,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:31,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:31,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:31,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be72a64fa9aff9372c6 HTTP/1.1" 200 None 2026-03-31 08:56:31,952 - lava_callback - INFO - Completed processing callback for node 69cb8be72a64fa9aff9372c6 2026-03-31 08:56:31,958 - lava_callback - INFO - Received callback for node 69cb8bdd2a64fa9aff9372ae INFO: 172.19.0.1:47678 - "POST /node/69cb8bdd2a64fa9aff9372ae HTTP/1.0" 202 Accepted 2026-03-31 08:56:31,984 - lava_callback - INFO - Starting async job submit for node 69cb8bdd2a64fa9aff9372ae 2026-03-31 08:56:32,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:32,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf02a64fa9aff9372df/log.txt.gz 2026-03-31 08:56:32,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdd2a64fa9aff9372ae HTTP/1.1" 200 None 2026-03-31 08:56:32,130 - lava_callback - DEBUG - Processing node 69cb8bdd2a64fa9aff9372ae: baseline-arm64 2026-03-31 08:56:32,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:32,189 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bee2a64fa9aff9372d8/lava_callback.json.gz 2026-03-31 08:56:32,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:32,456 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8beb2a64fa9aff9372cf/lava-logs.yaml 2026-03-31 08:56:32,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bee2a64fa9aff9372d8 HTTP/1.1" 200 None 2026-03-31 08:56:32,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:32,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:32,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:32,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:32,850 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8beb2a64fa9aff9372cf/lava_callback.json.gz 2026-03-31 08:56:32,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bee2a64fa9aff9372d8 HTTP/1.1" 200 None 2026-03-31 08:56:32,869 - lava_callback - INFO - Completed processing callback for node 69cb8bee2a64fa9aff9372d8 2026-03-31 08:56:32,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:32,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8beb2a64fa9aff9372cf HTTP/1.1" 200 None 2026-03-31 08:56:32,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:33,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:33,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8beb2a64fa9aff9372cf HTTP/1.1" 200 None 2026-03-31 08:56:33,051 - lava_callback - INFO - Completed processing callback for node 69cb8beb2a64fa9aff9372cf 2026-03-31 08:56:33,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:33,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdd2a64fa9aff9372ae/log.txt.gz 2026-03-31 08:56:33,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:33,257 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf02a64fa9aff9372df/lava-logs.yaml 2026-03-31 08:56:33,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:33,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:33,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:33,914 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf02a64fa9aff9372df/lava_callback.json.gz 2026-03-31 08:56:33,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:33,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf02a64fa9aff9372df HTTP/1.1" 200 None 2026-03-31 08:56:33,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:33,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:33,997 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdd2a64fa9aff9372ae/lava-logs.yaml 2026-03-31 08:56:34,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:34,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:34,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:34,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf02a64fa9aff9372df HTTP/1.1" 200 None 2026-03-31 08:56:34,046 - lava_callback - INFO - Completed processing callback for node 69cb8bf02a64fa9aff9372df 2026-03-31 08:56:34,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:34,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdd2a64fa9aff9372ae/lava_callback.json.gz 2026-03-31 08:56:34,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:34,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdd2a64fa9aff9372ae HTTP/1.1" 200 None 2026-03-31 08:56:34,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:34,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:34,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:34,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bdd2a64fa9aff9372ae HTTP/1.1" 200 None 2026-03-31 08:56:34,345 - lava_callback - INFO - Completed processing callback for node 69cb8bdd2a64fa9aff9372ae INFO: 172.19.0.1:47682 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:37,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:37,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.19.0.1:43064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43074 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:47,937 - lava_callback - INFO - Received callback for node 69cb8bf82a64fa9aff937315 INFO: 172.19.0.1:43082 - "POST /node/69cb8bf82a64fa9aff937315 HTTP/1.0" 202 Accepted 2026-03-31 08:56:47,945 - lava_callback - INFO - Starting async job submit for node 69cb8bf82a64fa9aff937315 2026-03-31 08:56:47,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:47,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf82a64fa9aff937315 HTTP/1.1" 200 None 2026-03-31 08:56:47,997 - lava_callback - DEBUG - Processing node 69cb8bf82a64fa9aff937315: baseline-nfs-arm64 2026-03-31 08:56:48,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:48,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:48,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf82a64fa9aff937315/log.txt.gz 2026-03-31 08:56:48,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:49,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:49,155 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf82a64fa9aff937315/lava-logs.yaml 2026-03-31 08:56:49,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:49,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:49,321 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf82a64fa9aff937315/lava_callback.json.gz 2026-03-31 08:56:49,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:49,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf82a64fa9aff937315 HTTP/1.1" 200 None 2026-03-31 08:56:49,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:49,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:49,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:49,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf82a64fa9aff937315 HTTP/1.1" 200 None 2026-03-31 08:56:49,429 - lava_callback - INFO - Completed processing callback for node 69cb8bf82a64fa9aff937315 2026-03-31 08:56:49,984 - lava_callback - INFO - Received callback for node 69cb8bf62a64fa9aff93730d INFO: 172.19.0.1:52554 - "POST /node/69cb8bf62a64fa9aff93730d HTTP/1.0" 202 Accepted 2026-03-31 08:56:49,990 - lava_callback - INFO - Starting async job submit for node 69cb8bf62a64fa9aff93730d 2026-03-31 08:56:50,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:50,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf62a64fa9aff93730d HTTP/1.1" 200 None 2026-03-31 08:56:50,042 - lava_callback - DEBUG - Processing node 69cb8bf62a64fa9aff93730d: baseline-nfs-arm64 2026-03-31 08:56:50,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:50,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:50,614 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf62a64fa9aff93730d/log.txt.gz 2026-03-31 08:56:50,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:51,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:51,028 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf62a64fa9aff93730d/lava-logs.yaml 2026-03-31 08:56:51,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:51,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:51,189 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf62a64fa9aff93730d/lava_callback.json.gz 2026-03-31 08:56:51,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:51,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf62a64fa9aff93730d HTTP/1.1" 200 None 2026-03-31 08:56:51,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:51,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:51,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:51,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf62a64fa9aff93730d HTTP/1.1" 200 None 2026-03-31 08:56:51,283 - lava_callback - INFO - Completed processing callback for node 69cb8bf62a64fa9aff93730d 2026-03-31 08:56:55,306 - lava_callback - INFO - Received callback for node 69cb8bfa2a64fa9aff937318 INFO: 172.19.0.1:52560 - "POST /node/69cb8bfa2a64fa9aff937318 HTTP/1.0" 202 Accepted 2026-03-31 08:56:55,313 - lava_callback - INFO - Starting async job submit for node 69cb8bfa2a64fa9aff937318 2026-03-31 08:56:55,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:55,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfa2a64fa9aff937318 HTTP/1.1" 200 None 2026-03-31 08:56:55,371 - lava_callback - DEBUG - Processing node 69cb8bfa2a64fa9aff937318: baseline-nfs-arm64 2026-03-31 08:56:55,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:56,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:56,127 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfa2a64fa9aff937318/log.txt.gz 2026-03-31 08:56:56,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:52574 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:56,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:56,687 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfa2a64fa9aff937318/lava-logs.yaml 2026-03-31 08:56:56,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:56,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:56,873 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfa2a64fa9aff937318/lava_callback.json.gz 2026-03-31 08:56:56,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:56,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfa2a64fa9aff937318 HTTP/1.1" 200 None 2026-03-31 08:56:56,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:56,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:56,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:56,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bfa2a64fa9aff937318 HTTP/1.1" 200 None 2026-03-31 08:56:56,971 - lava_callback - INFO - Completed processing callback for node 69cb8bfa2a64fa9aff937318 2026-03-31 08:57:02,409 - lava_callback - INFO - Received callback for node 69cb87cc2a64fa9aff93700c INFO: 172.19.0.1:48246 - "POST /node/69cb87cc2a64fa9aff93700c HTTP/1.0" 202 Accepted 2026-03-31 08:57:02,458 - lava_callback - INFO - Starting async job submit for node 69cb87cc2a64fa9aff93700c 2026-03-31 08:57:02,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:02,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87cc2a64fa9aff93700c HTTP/1.1" 200 None 2026-03-31 08:57:02,499 - lava_callback - DEBUG - Processing node 69cb87cc2a64fa9aff93700c: baseline-arm-pengutronix 2026-03-31 08:57:02,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:02,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:02,733 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cc2a64fa9aff93700c/log.txt.gz 2026-03-31 08:57:02,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:02,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:02,945 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cc2a64fa9aff93700c/lava-logs.yaml 2026-03-31 08:57:02,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:03,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:03,075 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cc2a64fa9aff93700c/lava_callback.json.gz DEBUG: Setup failed for 69cb87cc2a64fa9aff93700c. Transitting job node result: incomplete -> fail 2026-03-31 08:57:03,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:03,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87cc2a64fa9aff93700c HTTP/1.1" 200 None 2026-03-31 08:57:03,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:03,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:57:03,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:03,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87cc2a64fa9aff93700c HTTP/1.1" 200 None 2026-03-31 08:57:03,243 - lava_callback - INFO - Completed processing callback for node 69cb87cc2a64fa9aff93700c INFO: 172.19.0.1:48248 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:57:07,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:07,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:57:11,051 - lava_callback - INFO - Received callback for node 69cb8c102a64fa9aff93735a INFO: 172.19.0.1:57014 - "POST /node/69cb8c102a64fa9aff93735a HTTP/1.0" 202 Accepted 2026-03-31 08:57:11,059 - lava_callback - INFO - Starting async job submit for node 69cb8c102a64fa9aff93735a 2026-03-31 08:57:11,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:11,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c102a64fa9aff93735a HTTP/1.1" 200 None 2026-03-31 08:57:11,103 - lava_callback - DEBUG - Processing node 69cb8c102a64fa9aff93735a: baseline-arm64-broonie 2026-03-31 08:57:11,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:11,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:11,552 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c102a64fa9aff93735a/log.txt.gz 2026-03-31 08:57:11,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:11,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:11,950 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c102a64fa9aff93735a/lava-logs.yaml 2026-03-31 08:57:11,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:12,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:12,098 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c102a64fa9aff93735a/lava_callback.json.gz 2026-03-31 08:57:12,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:12,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c102a64fa9aff93735a HTTP/1.1" 200 None 2026-03-31 08:57:12,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:12,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:12,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:12,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c102a64fa9aff93735a HTTP/1.1" 200 None 2026-03-31 08:57:12,223 - lava_callback - INFO - Completed processing callback for node 69cb8c102a64fa9aff93735a 2026-03-31 08:57:13,608 - lava_callback - INFO - Received callback for node 69cb8c022a64fa9aff937331 INFO: 172.19.0.1:57020 - "POST /node/69cb8c022a64fa9aff937331 HTTP/1.0" 202 Accepted 2026-03-31 08:57:13,615 - lava_callback - INFO - Starting async job submit for node 69cb8c022a64fa9aff937331 2026-03-31 08:57:13,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:13,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c022a64fa9aff937331 HTTP/1.1" 200 None 2026-03-31 08:57:13,653 - lava_callback - DEBUG - Processing node 69cb8c022a64fa9aff937331: baseline-nfs-arm64 2026-03-31 08:57:14,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:14,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:14,166 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c022a64fa9aff937331/log.txt.gz 2026-03-31 08:57:14,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:14,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:14,596 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c022a64fa9aff937331/lava-logs.yaml 2026-03-31 08:57:14,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:14,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:14,775 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c022a64fa9aff937331/lava_callback.json.gz 2026-03-31 08:57:14,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:14,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c022a64fa9aff937331 HTTP/1.1" 200 None 2026-03-31 08:57:14,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:14,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:14,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:14,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c022a64fa9aff937331 HTTP/1.1" 200 None 2026-03-31 08:57:14,900 - lava_callback - INFO - Completed processing callback for node 69cb8c022a64fa9aff937331 2026-03-31 08:57:15,058 - lava_callback - INFO - Received callback for node 69cb8c2d2a64fa9aff9373bb INFO: 172.19.0.1:57034 - "POST /node/69cb8c2d2a64fa9aff9373bb HTTP/1.0" 202 Accepted 2026-03-31 08:57:15,068 - lava_callback - INFO - Starting async job submit for node 69cb8c2d2a64fa9aff9373bb 2026-03-31 08:57:15,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:15,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2d2a64fa9aff9373bb HTTP/1.1" 200 None 2026-03-31 08:57:15,122 - lava_callback - DEBUG - Processing node 69cb8c2d2a64fa9aff9373bb: baseline-arm64-foundriesio INFO: 172.19.0.1:57038 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 08:57:15,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:15,431 - lava_callback - INFO - Received callback for node 69cb8c2e2a64fa9aff9373be INFO: 172.19.0.1:57050 - "POST /node/69cb8c2e2a64fa9aff9373be HTTP/1.0" 202 Accepted 2026-03-31 08:57:15,443 - lava_callback - INFO - Starting async job submit for node 69cb8c2e2a64fa9aff9373be 2026-03-31 08:57:15,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:15,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2e2a64fa9aff9373be HTTP/1.1" 200 None 2026-03-31 08:57:15,499 - lava_callback - DEBUG - Processing node 69cb8c2e2a64fa9aff9373be: baseline-arm64-foundriesio 2026-03-31 08:57:15,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:15,578 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2d2a64fa9aff9373bb/log.txt.gz 2026-03-31 08:57:16,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:16,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:16,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:16,277 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2d2a64fa9aff9373bb/lava-logs.yaml 2026-03-31 08:57:16,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:16,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:16,297 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2e2a64fa9aff9373be/log.txt.gz 2026-03-31 08:57:16,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:16,507 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2d2a64fa9aff9373bb/lava_callback.json.gz INFO: 172.19.0.1:57062 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:57:16,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:16,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:16,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2d2a64fa9aff9373bb HTTP/1.1" 200 None 2026-03-31 08:57:16,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:16,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:16,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:16,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c2d2a64fa9aff9373bb HTTP/1.1" 200 None 2026-03-31 08:57:16,640 - lava_callback - INFO - Completed processing callback for node 69cb8c2d2a64fa9aff9373bb 2026-03-31 08:57:16,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:16,812 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2e2a64fa9aff9373be/lava-logs.yaml 2026-03-31 08:57:16,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:16,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:16,966 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2e2a64fa9aff9373be/lava_callback.json.gz 2026-03-31 08:57:16,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:17,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2e2a64fa9aff9373be HTTP/1.1" 200 None 2026-03-31 08:57:17,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:17,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:17,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:17,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c2e2a64fa9aff9373be HTTP/1.1" 200 None 2026-03-31 08:57:17,053 - lava_callback - INFO - Completed processing callback for node 69cb8c2e2a64fa9aff9373be 2026-03-31 08:57:19,575 - lava_callback - INFO - Received callback for node 69cb8c282a64fa9aff9373b2 INFO: 172.19.0.1:57078 - "POST /node/69cb8c282a64fa9aff9373b2 HTTP/1.0" 202 Accepted 2026-03-31 08:57:19,581 - lava_callback - INFO - Starting async job submit for node 69cb8c282a64fa9aff9373b2 2026-03-31 08:57:19,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:19,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c282a64fa9aff9373b2 HTTP/1.1" 200 None 2026-03-31 08:57:19,616 - lava_callback - DEBUG - Processing node 69cb8c282a64fa9aff9373b2: baseline-arm64-pengutronix 2026-03-31 08:57:19,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:19,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:19,906 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c282a64fa9aff9373b2/log.txt.gz 2026-03-31 08:57:20,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:20,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:20,257 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c282a64fa9aff9373b2/lava-logs.yaml 2026-03-31 08:57:20,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:20,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:20,435 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c282a64fa9aff9373b2/lava_callback.json.gz 2026-03-31 08:57:20,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:20,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c282a64fa9aff9373b2 HTTP/1.1" 200 None 2026-03-31 08:57:20,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:20,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:20,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:20,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c282a64fa9aff9373b2 HTTP/1.1" 200 None 2026-03-31 08:57:20,520 - lava_callback - INFO - Completed processing callback for node 69cb8c282a64fa9aff9373b2 2026-03-31 08:57:20,750 - lava_callback - INFO - Received callback for node 69cb8bf32a64fa9aff9372e9 INFO: 172.19.0.1:38982 - "POST /node/69cb8bf32a64fa9aff9372e9 HTTP/1.0" 202 Accepted 2026-03-31 08:57:20,757 - lava_callback - INFO - Starting async job submit for node 69cb8bf32a64fa9aff9372e9 2026-03-31 08:57:20,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:20,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf32a64fa9aff9372e9 HTTP/1.1" 200 None 2026-03-31 08:57:20,814 - lava_callback - DEBUG - Processing node 69cb8bf32a64fa9aff9372e9: baseline-nfs-arm64 2026-03-31 08:57:21,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:21,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:21,339 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf32a64fa9aff9372e9/log.txt.gz 2026-03-31 08:57:21,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:21,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:21,737 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf32a64fa9aff9372e9/lava-logs.yaml 2026-03-31 08:57:21,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:21,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:21,873 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf32a64fa9aff9372e9/lava_callback.json.gz 2026-03-31 08:57:21,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:21,902 - lava_callback - INFO - Received callback for node 69cb8c292a64fa9aff9373b5 INFO: 172.19.0.1:38994 - "POST /node/69cb8c292a64fa9aff9373b5 HTTP/1.0" 202 Accepted 2026-03-31 08:57:21,911 - lava_callback - INFO - Starting async job submit for node 69cb8c292a64fa9aff9373b5 2026-03-31 08:57:21,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:21,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf32a64fa9aff9372e9 HTTP/1.1" 200 None 2026-03-31 08:57:21,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:21,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c292a64fa9aff9373b5 HTTP/1.1" 200 None 2026-03-31 08:57:21,950 - lava_callback - DEBUG - Processing node 69cb8c292a64fa9aff9373b5: baseline-arm64-pengutronix 2026-03-31 08:57:21,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:22,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:22,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf32a64fa9aff9372e9 HTTP/1.1" 200 None 2026-03-31 08:57:22,102 - lava_callback - INFO - Completed processing callback for node 69cb8bf32a64fa9aff9372e9 2026-03-31 08:57:22,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:22,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:22,342 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c292a64fa9aff9373b5/log.txt.gz 2026-03-31 08:57:22,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:22,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:22,641 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c292a64fa9aff9373b5/lava-logs.yaml 2026-03-31 08:57:22,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:22,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:22,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c292a64fa9aff9373b5/lava_callback.json.gz 2026-03-31 08:57:22,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:22,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c292a64fa9aff9373b5 HTTP/1.1" 200 None 2026-03-31 08:57:22,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:22,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:22,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:22,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c292a64fa9aff9373b5 HTTP/1.1" 200 None 2026-03-31 08:57:22,890 - lava_callback - INFO - Completed processing callback for node 69cb8c292a64fa9aff9373b5 INFO: 172.19.0.1:38998 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:57:27,182 - lava_callback - INFO - Received callback for node 69c8e96a6a4c026e1634892f INFO: 172.19.0.1:39004 - "POST /node/69c8e96a6a4c026e1634892f HTTP/1.0" 202 Accepted 2026-03-31 08:57:27,188 - lava_callback - INFO - Starting async job submit for node 69c8e96a6a4c026e1634892f 2026-03-31 08:57:27,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:27,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e96a6a4c026e1634892f HTTP/1.1" 200 None 2026-03-31 08:57:27,205 - lava_callback - DEBUG - Processing node 69c8e96a6a4c026e1634892f: baseline-arm-baylibre 2026-03-31 08:57:27,205 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e96a6a4c026e1634892f 2026-03-31 08:57:27,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:27,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e96a6a4c026e1634892f HTTP/1.1" 200 None 2026-03-31 08:57:27,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:27,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 08:57:27,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:27,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e96a6a4c026e1634892f HTTP/1.1" 200 None 2026-03-31 08:57:27,279 - lava_callback - ERROR - Error processing callback for node 69c8e96a6a4c026e1634892f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:57:30,811 - lava_callback - INFO - Received callback for node 69cb8c182a64fa9aff937384 INFO: 172.19.0.1:52956 - "POST /node/69cb8c182a64fa9aff937384 HTTP/1.0" 202 Accepted 2026-03-31 08:57:30,817 - lava_callback - INFO - Starting async job submit for node 69cb8c182a64fa9aff937384 2026-03-31 08:57:30,834 - lava_callback - INFO - Received callback for node 69cb8c2b2a64fa9aff9373b8 INFO: 172.19.0.1:52968 - "POST /node/69cb8c2b2a64fa9aff9373b8 HTTP/1.0" 202 Accepted 2026-03-31 08:57:30,847 - lava_callback - INFO - Starting async job submit for node 69cb8c2b2a64fa9aff9373b8 2026-03-31 08:57:30,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:30,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:30,870 - lava_callback - INFO - Received callback for node 69cb8be42a64fa9aff9372c0 INFO: 172.19.0.1:52972 - "POST /node/69cb8be42a64fa9aff9372c0 HTTP/1.0" 202 Accepted 2026-03-31 08:57:30,876 - lava_callback - INFO - Starting async job submit for node 69cb8be42a64fa9aff9372c0 2026-03-31 08:57:30,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:30,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c182a64fa9aff937384 HTTP/1.1" 200 None 2026-03-31 08:57:30,909 - lava_callback - DEBUG - Processing node 69cb8c182a64fa9aff937384: baseline-arm64-broonie 2026-03-31 08:57:30,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2b2a64fa9aff9373b8 HTTP/1.1" 200 None 2026-03-31 08:57:30,932 - lava_callback - DEBUG - Processing node 69cb8c2b2a64fa9aff9373b8: baseline-arm64-pengutronix 2026-03-31 08:57:31,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be42a64fa9aff9372c0 HTTP/1.1" 200 None 2026-03-31 08:57:31,396 - lava_callback - DEBUG - Processing node 69cb8be42a64fa9aff9372c0: baseline-arm64 2026-03-31 08:57:31,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:31,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:31,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:31,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:31,713 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c182a64fa9aff937384/log.txt.gz 2026-03-31 08:57:31,724 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c2b2a64fa9aff9373b8/log.txt.gz 2026-03-31 08:57:31,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,219 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be42a64fa9aff9372c0/log.txt.gz 2026-03-31 08:57:32,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,385 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c2b2a64fa9aff9373b8/lava-logs.yaml 2026-03-31 08:57:32,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c182a64fa9aff937384/lava-logs.yaml 2026-03-31 08:57:32,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,608 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c2b2a64fa9aff9373b8/lava_callback.json.gz 2026-03-31 08:57:32,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2b2a64fa9aff9373b8 HTTP/1.1" 200 None 2026-03-31 08:57:32,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:32,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,716 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c182a64fa9aff937384/lava_callback.json.gz 2026-03-31 08:57:32,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c2b2a64fa9aff9373b8 HTTP/1.1" 200 None 2026-03-31 08:57:32,744 - lava_callback - INFO - Completed processing callback for node 69cb8c2b2a64fa9aff9373b8 2026-03-31 08:57:32,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,827 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be42a64fa9aff9372c0/lava-logs.yaml 2026-03-31 08:57:32,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c182a64fa9aff937384 HTTP/1.1" 200 None 2026-03-31 08:57:32,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:32,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c182a64fa9aff937384 HTTP/1.1" 200 None 2026-03-31 08:57:32,909 - lava_callback - INFO - Completed processing callback for node 69cb8c182a64fa9aff937384 2026-03-31 08:57:33,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:33,069 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be42a64fa9aff9372c0/lava_callback.json.gz 2026-03-31 08:57:33,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:33,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be42a64fa9aff9372c0 HTTP/1.1" 200 None 2026-03-31 08:57:33,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:33,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:33,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:33,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be42a64fa9aff9372c0 HTTP/1.1" 200 None 2026-03-31 08:57:33,244 - lava_callback - INFO - Received callback for node 69cb8c232a64fa9aff9373a0 INFO: 172.19.0.1:52976 - "POST /node/69cb8c232a64fa9aff9373a0 HTTP/1.0" 202 Accepted 2026-03-31 08:57:33,281 - lava_callback - INFO - Starting async job submit for node 69cb8c232a64fa9aff9373a0 2026-03-31 08:57:33,357 - lava_callback - INFO - Completed processing callback for node 69cb8be42a64fa9aff9372c0 2026-03-31 08:57:33,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:33,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c232a64fa9aff9373a0 HTTP/1.1" 200 None 2026-03-31 08:57:33,395 - lava_callback - DEBUG - Processing node 69cb8c232a64fa9aff9373a0: baseline-arm64-clabbe 2026-03-31 08:57:33,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:33,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:33,822 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cb8c232a64fa9aff9373a0/log.txt.gz 2026-03-31 08:57:33,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:34,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:34,125 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cb8c232a64fa9aff9373a0/lava-logs.yaml 2026-03-31 08:57:34,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:34,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:34,256 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cb8c232a64fa9aff9373a0/lava_callback.json.gz 2026-03-31 08:57:34,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:34,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c232a64fa9aff9373a0 HTTP/1.1" 200 None 2026-03-31 08:57:34,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:34,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:34,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:34,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c232a64fa9aff9373a0 HTTP/1.1" 200 None 2026-03-31 08:57:34,335 - lava_callback - INFO - Completed processing callback for node 69cb8c232a64fa9aff9373a0 INFO: 172.19.0.1:52990 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:57:37,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:37,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:57:39,455 - lava_callback - INFO - Received callback for node 69cb8bfd2a64fa9aff93731e INFO: 172.19.0.1:52996 - "POST /node/69cb8bfd2a64fa9aff93731e HTTP/1.0" 202 Accepted 2026-03-31 08:57:39,462 - lava_callback - INFO - Starting async job submit for node 69cb8bfd2a64fa9aff93731e 2026-03-31 08:57:39,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:39,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfd2a64fa9aff93731e HTTP/1.1" 200 None 2026-03-31 08:57:39,501 - lava_callback - DEBUG - Processing node 69cb8bfd2a64fa9aff93731e: baseline-nfs-arm64 2026-03-31 08:57:39,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:39,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:39,926 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfd2a64fa9aff93731e/log.txt.gz 2026-03-31 08:57:40,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:40,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:40,297 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfd2a64fa9aff93731e/lava-logs.yaml 2026-03-31 08:57:40,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:40,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:40,430 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfd2a64fa9aff93731e/lava_callback.json.gz 2026-03-31 08:57:40,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:40,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfd2a64fa9aff93731e HTTP/1.1" 200 None 2026-03-31 08:57:40,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:40,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:40,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:40,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bfd2a64fa9aff93731e HTTP/1.1" 200 None 2026-03-31 08:57:40,529 - lava_callback - INFO - Completed processing callback for node 69cb8bfd2a64fa9aff93731e INFO: 172.19.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56382 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:57:50,075 - lava_callback - INFO - Received callback for node 69cb8c132a64fa9aff937377 INFO: 172.19.0.1:37136 - "POST /node/69cb8c132a64fa9aff937377 HTTP/1.0" 202 Accepted 2026-03-31 08:57:50,082 - lava_callback - INFO - Starting async job submit for node 69cb8c132a64fa9aff937377 2026-03-31 08:57:50,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:50,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c132a64fa9aff937377 HTTP/1.1" 200 None 2026-03-31 08:57:50,121 - lava_callback - DEBUG - Processing node 69cb8c132a64fa9aff937377: baseline-arm64-broonie 2026-03-31 08:57:50,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:50,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:50,506 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c132a64fa9aff937377/log.txt.gz 2026-03-31 08:57:50,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:50,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:50,858 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c132a64fa9aff937377/lava-logs.yaml 2026-03-31 08:57:50,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:50,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:50,993 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c132a64fa9aff937377/lava_callback.json.gz 2026-03-31 08:57:51,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:51,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c132a64fa9aff937377 HTTP/1.1" 200 None 2026-03-31 08:57:51,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:51,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:51,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:51,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c132a64fa9aff937377 HTTP/1.1" 200 None 2026-03-31 08:57:51,089 - lava_callback - INFO - Completed processing callback for node 69cb8c132a64fa9aff937377 2026-03-31 08:57:51,536 - lava_callback - INFO - Received callback for node 69cb8c152a64fa9aff93737a INFO: 172.19.0.1:37152 - "POST /node/69cb8c152a64fa9aff93737a HTTP/1.0" 202 Accepted 2026-03-31 08:57:51,543 - lava_callback - INFO - Starting async job submit for node 69cb8c152a64fa9aff93737a 2026-03-31 08:57:51,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:51,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c152a64fa9aff93737a HTTP/1.1" 200 None 2026-03-31 08:57:51,589 - lava_callback - DEBUG - Processing node 69cb8c152a64fa9aff93737a: baseline-arm64-broonie 2026-03-31 08:57:51,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:52,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:52,038 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c152a64fa9aff93737a/log.txt.gz 2026-03-31 08:57:52,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:52,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:52,437 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c152a64fa9aff93737a/lava-logs.yaml 2026-03-31 08:57:52,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:52,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:52,599 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c152a64fa9aff93737a/lava_callback.json.gz 2026-03-31 08:57:52,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:52,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c152a64fa9aff93737a HTTP/1.1" 200 None 2026-03-31 08:57:52,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:52,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:52,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:52,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c152a64fa9aff93737a HTTP/1.1" 200 None 2026-03-31 08:57:52,707 - lava_callback - INFO - Completed processing callback for node 69cb8c152a64fa9aff93737a 2026-03-31 08:57:52,929 - lava_callback - INFO - Received callback for node 69cb8c1c2a64fa9aff93738e INFO: 172.19.0.1:37168 - "POST /node/69cb8c1c2a64fa9aff93738e HTTP/1.0" 202 Accepted 2026-03-31 08:57:52,935 - lava_callback - INFO - Starting async job submit for node 69cb8c1c2a64fa9aff93738e 2026-03-31 08:57:52,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:52,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1c2a64fa9aff93738e HTTP/1.1" 200 None 2026-03-31 08:57:52,975 - lava_callback - DEBUG - Processing node 69cb8c1c2a64fa9aff93738e: baseline-arm64-broonie 2026-03-31 08:57:53,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:53,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:53,355 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1c2a64fa9aff93738e/log.txt.gz 2026-03-31 08:57:53,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:53,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:53,670 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1c2a64fa9aff93738e/lava-logs.yaml 2026-03-31 08:57:53,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:53,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:53,811 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1c2a64fa9aff93738e/lava_callback.json.gz 2026-03-31 08:57:53,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:53,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1c2a64fa9aff93738e HTTP/1.1" 200 None 2026-03-31 08:57:53,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:53,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:53,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:53,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c1c2a64fa9aff93738e HTTP/1.1" 200 None 2026-03-31 08:57:53,902 - lava_callback - INFO - Completed processing callback for node 69cb8c1c2a64fa9aff93738e INFO: 172.19.0.1:37172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54584 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:58:07,149 - lava_callback - INFO - Received callback for node 69c8e98e6a4c026e1634893b INFO: 172.19.0.1:54592 - "POST /node/69c8e98e6a4c026e1634893b HTTP/1.0" 202 Accepted 2026-03-31 08:58:07,156 - lava_callback - INFO - Starting async job submit for node 69c8e98e6a4c026e1634893b 2026-03-31 08:58:07,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:07,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e98e6a4c026e1634893b HTTP/1.1" 200 None 2026-03-31 08:58:07,172 - lava_callback - DEBUG - Processing node 69c8e98e6a4c026e1634893b: baseline-arm-baylibre 2026-03-31 08:58:07,172 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e98e6a4c026e1634893b 2026-03-31 08:58:07,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:07,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e98e6a4c026e1634893b HTTP/1.1" 200 None 2026-03-31 08:58:07,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:07,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 08:58:07,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:07,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e98e6a4c026e1634893b HTTP/1.1" 200 None 2026-03-31 08:58:07,250 - lava_callback - ERROR - Error processing callback for node 69c8e98e6a4c026e1634893b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:58:07,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:07,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:58:10,019 - lava_callback - INFO - Received callback for node 69cb8c1a2a64fa9aff937389 INFO: 172.19.0.1:51080 - "POST /node/69cb8c1a2a64fa9aff937389 HTTP/1.0" 202 Accepted 2026-03-31 08:58:10,029 - lava_callback - INFO - Starting async job submit for node 69cb8c1a2a64fa9aff937389 2026-03-31 08:58:10,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:10,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1a2a64fa9aff937389 HTTP/1.1" 200 None 2026-03-31 08:58:10,073 - lava_callback - DEBUG - Processing node 69cb8c1a2a64fa9aff937389: baseline-arm64-broonie 2026-03-31 08:58:10,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:10,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:10,552 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1a2a64fa9aff937389/log.txt.gz 2026-03-31 08:58:10,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:10,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:10,954 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1a2a64fa9aff937389/lava-logs.yaml 2026-03-31 08:58:10,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:10,994 - lava_callback - INFO - Received callback for node 69cb87fb2a64fa9aff937088 INFO: 172.19.0.1:51086 - "POST /node/69cb87fb2a64fa9aff937088 HTTP/1.0" 202 Accepted 2026-03-31 08:58:11,005 - lava_callback - INFO - Starting async job submit for node 69cb87fb2a64fa9aff937088 2026-03-31 08:58:11,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:11,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87fb2a64fa9aff937088 HTTP/1.1" 200 None 2026-03-31 08:58:11,088 - lava_callback - DEBUG - Processing node 69cb87fb2a64fa9aff937088: ltp-smoke 2026-03-31 08:58:11,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:11,161 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1a2a64fa9aff937389/lava_callback.json.gz 2026-03-31 08:58:11,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:11,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1a2a64fa9aff937389 HTTP/1.1" 200 None 2026-03-31 08:58:11,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:11,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:58:11,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:11,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c1a2a64fa9aff937389 HTTP/1.1" 200 None 2026-03-31 08:58:11,602 - lava_callback - INFO - Completed processing callback for node 69cb8c1a2a64fa9aff937389 2026-03-31 08:58:11,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:11,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:11,809 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87fb2a64fa9aff937088/log.txt.gz 2026-03-31 08:58:12,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:12,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:12,277 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87fb2a64fa9aff937088/lava-logs.yaml 2026-03-31 08:58:12,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:12,406 - lava_callback - INFO - Received callback for node 69cb8c1e2a64fa9aff937391 INFO: 172.19.0.1:51088 - "POST /node/69cb8c1e2a64fa9aff937391 HTTP/1.0" 202 Accepted 2026-03-31 08:58:12,414 - lava_callback - INFO - Starting async job submit for node 69cb8c1e2a64fa9aff937391 2026-03-31 08:58:12,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:12,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:12,461 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87fb2a64fa9aff937088/lava_callback.json.gz 2026-03-31 08:58:12,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1e2a64fa9aff937391 HTTP/1.1" 200 None 2026-03-31 08:58:12,478 - lava_callback - DEBUG - Processing node 69cb8c1e2a64fa9aff937391: baseline-arm64-broonie 2026-03-31 08:58:12,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:12,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87fb2a64fa9aff937088 HTTP/1.1" 200 None 2026-03-31 08:58:12,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:12,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:12,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83462a64fa9aff936905 HTTP/1.1" 200 None 2026-03-31 08:58:12,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:12,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87fb2a64fa9aff937088 HTTP/1.1" 200 None 2026-03-31 08:58:13,152 - lava_callback - INFO - Completed processing callback for node 69cb87fb2a64fa9aff937088 2026-03-31 08:58:13,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:13,180 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1e2a64fa9aff937391/log.txt.gz 2026-03-31 08:58:13,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:13,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:13,571 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1e2a64fa9aff937391/lava-logs.yaml 2026-03-31 08:58:13,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:13,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:13,743 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1e2a64fa9aff937391/lava_callback.json.gz 2026-03-31 08:58:13,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:13,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1e2a64fa9aff937391 HTTP/1.1" 200 None 2026-03-31 08:58:13,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:13,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:58:13,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:13,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c1e2a64fa9aff937391 HTTP/1.1" 200 None 2026-03-31 08:58:13,832 - lava_callback - INFO - Completed processing callback for node 69cb8c1e2a64fa9aff937391 INFO: 172.19.0.1:51094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41306 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:58:30,871 - lava_callback - INFO - Received callback for node 69cb8c112a64fa9aff93735d INFO: 172.19.0.1:39276 - "POST /node/69cb8c112a64fa9aff93735d HTTP/1.0" 202 Accepted 2026-03-31 08:58:30,876 - lava_callback - INFO - Starting async job submit for node 69cb8c112a64fa9aff93735d 2026-03-31 08:58:30,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:30,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c112a64fa9aff93735d HTTP/1.1" 200 None 2026-03-31 08:58:30,915 - lava_callback - DEBUG - Processing node 69cb8c112a64fa9aff93735d: baseline-arm64-broonie 2026-03-31 08:58:31,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:31,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:31,275 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c112a64fa9aff93735d/log.txt.gz 2026-03-31 08:58:31,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:31,599 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:31,601 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c112a64fa9aff93735d/lava-logs.yaml 2026-03-31 08:58:31,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:31,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:31,755 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c112a64fa9aff93735d/lava_callback.json.gz 2026-03-31 08:58:31,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:31,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c112a64fa9aff93735d HTTP/1.1" 200 None 2026-03-31 08:58:31,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:31,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:58:31,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:31,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c112a64fa9aff93735d HTTP/1.1" 200 None 2026-03-31 08:58:31,861 - lava_callback - INFO - Completed processing callback for node 69cb8c112a64fa9aff93735d INFO: 172.19.0.1:39284 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:58:37,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:37,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:58:39,394 - lava_callback - INFO - Received callback for node 69cb8be32a64fa9aff9372bd INFO: 172.19.0.1:39294 - "POST /node/69cb8be32a64fa9aff9372bd HTTP/1.0" 202 Accepted 2026-03-31 08:58:39,400 - lava_callback - INFO - Starting async job submit for node 69cb8be32a64fa9aff9372bd 2026-03-31 08:58:39,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:39,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be32a64fa9aff9372bd HTTP/1.1" 200 None 2026-03-31 08:58:39,434 - lava_callback - DEBUG - Processing node 69cb8be32a64fa9aff9372bd: baseline-arm64 2026-03-31 08:58:39,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:40,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:40,073 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be32a64fa9aff9372bd/log.txt.gz 2026-03-31 08:58:40,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:40,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:40,563 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be32a64fa9aff9372bd/lava-logs.yaml 2026-03-31 08:58:40,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:40,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:40,731 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be32a64fa9aff9372bd/lava_callback.json.gz 2026-03-31 08:58:40,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:40,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be32a64fa9aff9372bd HTTP/1.1" 200 None 2026-03-31 08:58:40,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:40,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:58:40,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:40,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be32a64fa9aff9372bd HTTP/1.1" 200 None 2026-03-31 08:58:40,835 - lava_callback - INFO - Completed processing callback for node 69cb8be32a64fa9aff9372bd INFO: 172.19.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52878 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:58:47,204 - lava_callback - INFO - Received callback for node 69cb8c002a64fa9aff937324 INFO: 172.19.0.1:52884 - "POST /node/69cb8c002a64fa9aff937324 HTTP/1.0" 202 Accepted 2026-03-31 08:58:47,211 - lava_callback - INFO - Starting async job submit for node 69cb8c002a64fa9aff937324 2026-03-31 08:58:47,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:47,244 - lava_callback - INFO - Received callback for node 69c8e9b66a4c026e16348947 INFO: 172.19.0.1:52898 - "POST /node/69c8e9b66a4c026e16348947 HTTP/1.0" 202 Accepted 2026-03-31 08:58:47,250 - lava_callback - INFO - Starting async job submit for node 69c8e9b66a4c026e16348947 2026-03-31 08:58:47,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:47,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c002a64fa9aff937324 HTTP/1.1" 200 None 2026-03-31 08:58:47,254 - lava_callback - DEBUG - Processing node 69cb8c002a64fa9aff937324: baseline-nfs-arm64 2026-03-31 08:58:47,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9b66a4c026e16348947 HTTP/1.1" 200 None 2026-03-31 08:58:47,318 - lava_callback - DEBUG - Processing node 69c8e9b66a4c026e16348947: baseline-arm-clabbe 2026-03-31 08:58:47,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e9b66a4c026e16348947 2026-03-31 08:58:47,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:47,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9b66a4c026e16348947 HTTP/1.1" 200 None 2026-03-31 08:58:47,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:47,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 08:58:47,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:47,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:47,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e9b66a4c026e16348947 HTTP/1.1" 200 None 2026-03-31 08:58:47,656 - lava_callback - ERROR - Error processing callback for node 69c8e9b66a4c026e16348947: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:58:47,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:47,892 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c002a64fa9aff937324/log.txt.gz 2026-03-31 08:58:48,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:48,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:48,333 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c002a64fa9aff937324/lava-logs.yaml 2026-03-31 08:58:48,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:48,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:48,513 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c002a64fa9aff937324/lava_callback.json.gz 2026-03-31 08:58:48,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:48,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c002a64fa9aff937324 HTTP/1.1" 200 None 2026-03-31 08:58:48,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:48,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:58:48,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:48,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c002a64fa9aff937324 HTTP/1.1" 200 None 2026-03-31 08:58:48,608 - lava_callback - INFO - Completed processing callback for node 69cb8c002a64fa9aff937324 INFO: 172.19.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35032 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:59:07,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:07,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:43302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44714 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:59:47,160 - lava_callback - INFO - Received callback for node 69cb8bf42a64fa9aff93730a INFO: 172.19.0.1:44722 - "POST /node/69cb8bf42a64fa9aff93730a HTTP/1.0" 202 Accepted 2026-03-31 08:59:47,166 - lava_callback - INFO - Starting async job submit for node 69cb8bf42a64fa9aff93730a 2026-03-31 08:59:47,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:47,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf42a64fa9aff93730a HTTP/1.1" 200 None 2026-03-31 08:59:47,206 - lava_callback - DEBUG - Processing node 69cb8bf42a64fa9aff93730a: baseline-nfs-arm64 2026-03-31 08:59:47,251 - lava_callback - INFO - Received callback for node 69c8e9fe6a4c026e16348963 INFO: 172.19.0.1:44732 - "POST /node/69c8e9fe6a4c026e16348963 HTTP/1.0" 202 Accepted 2026-03-31 08:59:47,272 - lava_callback - INFO - Starting async job submit for node 69c8e9fe6a4c026e16348963 2026-03-31 08:59:47,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:47,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9fe6a4c026e16348963 HTTP/1.1" 200 None 2026-03-31 08:59:47,378 - lava_callback - DEBUG - Processing node 69c8e9fe6a4c026e16348963: baseline-arm-clabbe 2026-03-31 08:59:47,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e9fe6a4c026e16348963 2026-03-31 08:59:47,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:47,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9fe6a4c026e16348963 HTTP/1.1" 200 None 2026-03-31 08:59:47,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:47,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 08:59:47,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:47,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:47,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e9fe6a4c026e16348963 HTTP/1.1" 200 None 2026-03-31 08:59:47,677 - lava_callback - ERROR - Error processing callback for node 69c8e9fe6a4c026e16348963: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:59:47,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:47,877 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf42a64fa9aff93730a/log.txt.gz 2026-03-31 08:59:48,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:48,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:48,433 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf42a64fa9aff93730a/lava-logs.yaml 2026-03-31 08:59:48,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:48,625 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:48,626 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf42a64fa9aff93730a/lava_callback.json.gz 2026-03-31 08:59:48,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:48,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf42a64fa9aff93730a HTTP/1.1" 200 None 2026-03-31 08:59:48,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:48,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:59:48,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:48,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf42a64fa9aff93730a HTTP/1.1" 200 None 2026-03-31 08:59:48,719 - lava_callback - INFO - Completed processing callback for node 69cb8bf42a64fa9aff93730a 2026-03-31 08:59:51,253 - lava_callback - INFO - Received callback for node 69cb8c172a64fa9aff93737d INFO: 172.19.0.1:36542 - "POST /node/69cb8c172a64fa9aff93737d HTTP/1.0" 202 Accepted 2026-03-31 08:59:51,259 - lava_callback - INFO - Starting async job submit for node 69cb8c172a64fa9aff93737d 2026-03-31 08:59:51,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:51,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c172a64fa9aff93737d HTTP/1.1" 200 None 2026-03-31 08:59:51,297 - lava_callback - DEBUG - Processing node 69cb8c172a64fa9aff93737d: baseline-arm64-broonie 2026-03-31 08:59:51,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:51,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:51,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c172a64fa9aff93737d/log.txt.gz 2026-03-31 08:59:51,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:52,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:52,022 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c172a64fa9aff93737d/lava-logs.yaml 2026-03-31 08:59:52,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:52,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:52,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c172a64fa9aff93737d/lava_callback.json.gz 2026-03-31 08:59:52,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:52,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c172a64fa9aff93737d HTTP/1.1" 200 None 2026-03-31 08:59:52,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:52,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:59:52,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:52,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c172a64fa9aff93737d HTTP/1.1" 200 None 2026-03-31 08:59:52,310 - lava_callback - INFO - Completed processing callback for node 69cb8c172a64fa9aff93737d INFO: 172.19.0.1:36546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41314 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:00:07,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:00:08,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:34634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54506 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:00:27,232 - lava_callback - INFO - Received callback for node 69c8ea236a4c026e16348970 INFO: 172.19.0.1:54508 - "POST /node/69c8ea236a4c026e16348970 HTTP/1.0" 202 Accepted 2026-03-31 09:00:27,237 - lava_callback - INFO - Starting async job submit for node 69c8ea236a4c026e16348970 2026-03-31 09:00:27,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:00:27,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ea236a4c026e16348970 HTTP/1.1" 200 None 2026-03-31 09:00:27,254 - lava_callback - DEBUG - Processing node 69c8ea236a4c026e16348970: baseline-arm-clabbe 2026-03-31 09:00:27,255 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ea236a4c026e16348970 2026-03-31 09:00:27,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:00:27,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ea236a4c026e16348970 HTTP/1.1" 200 None 2026-03-31 09:00:27,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:00:27,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 09:00:27,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:00:27,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ea236a4c026e16348970 HTTP/1.1" 200 None 2026-03-31 09:00:27,322 - lava_callback - ERROR - Error processing callback for node 69c8ea236a4c026e16348970: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49304 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:01:07,296 - lava_callback - INFO - Received callback for node 69c8ea496a4c026e1634897d INFO: 172.19.0.1:49312 - "POST /node/69c8ea496a4c026e1634897d HTTP/1.0" 202 Accepted 2026-03-31 09:01:07,302 - lava_callback - INFO - Starting async job submit for node 69c8ea496a4c026e1634897d 2026-03-31 09:01:07,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:07,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ea496a4c026e1634897d HTTP/1.1" 200 None 2026-03-31 09:01:07,320 - lava_callback - DEBUG - Processing node 69c8ea496a4c026e1634897d: baseline-arm-clabbe 2026-03-31 09:01:07,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ea496a4c026e1634897d 2026-03-31 09:01:07,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:07,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ea496a4c026e1634897d HTTP/1.1" 200 None 2026-03-31 09:01:07,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:07,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 09:01:07,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:07,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ea496a4c026e1634897d HTTP/1.1" 200 None 2026-03-31 09:01:07,382 - lava_callback - ERROR - Error processing callback for node 69c8ea496a4c026e1634897d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 09:01:10,228 - lava_callback - INFO - Received callback for node 69cb8d402a64fa9aff93752a INFO: 172.19.0.1:59542 - "POST /node/69cb8d402a64fa9aff93752a HTTP/1.0" 202 Accepted 2026-03-31 09:01:10,234 - lava_callback - INFO - Starting async job submit for node 69cb8d402a64fa9aff93752a 2026-03-31 09:01:10,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:10,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8d402a64fa9aff93752a HTTP/1.1" 200 None 2026-03-31 09:01:10,251 - lava_callback - DEBUG - Processing node 69cb8d402a64fa9aff93752a: baseline-arm 2026-03-31 09:01:10,257 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:01:10,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:01:10,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:01:10,382 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb8d402a64fa9aff93752a/lava-logs.yaml 2026-03-31 09:01:10,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:01:10,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:01:10,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb8d402a64fa9aff93752a/lava_callback.json.gz 2026-03-31 09:01:10,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:10,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8d402a64fa9aff93752a HTTP/1.1" 200 None 2026-03-31 09:01:10,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:10,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 09:01:10,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:10,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8d402a64fa9aff93752a HTTP/1.1" 200 None 2026-03-31 09:01:10,566 - lava_callback - INFO - Completed processing callback for node 69cb8d402a64fa9aff93752a INFO: 172.19.0.1:59556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50748 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:01:38,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:38,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:41834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41276 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:02:42,723 - lava_callback - INFO - Received callback for node 69cb8bf22a64fa9aff9372e6 INFO: 172.19.0.1:50108 - "POST /node/69cb8bf22a64fa9aff9372e6 HTTP/1.0" 202 Accepted 2026-03-31 09:02:42,730 - lava_callback - INFO - Starting async job submit for node 69cb8bf22a64fa9aff9372e6 2026-03-31 09:02:42,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:02:42,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf22a64fa9aff9372e6 HTTP/1.1" 200 None 2026-03-31 09:02:42,785 - lava_callback - DEBUG - Processing node 69cb8bf22a64fa9aff9372e6: baseline-arm64 2026-03-31 09:02:43,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:02:43,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:02:43,309 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf22a64fa9aff9372e6/log.txt.gz 2026-03-31 09:02:43,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:02:43,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:02:43,765 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf22a64fa9aff9372e6/lava-logs.yaml 2026-03-31 09:02:43,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:02:43,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:02:43,901 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf22a64fa9aff9372e6/lava_callback.json.gz DEBUG: Setup failed for 69cb8bf22a64fa9aff9372e6. Transitting job node result: incomplete -> fail 2026-03-31 09:02:43,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:02:43,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf22a64fa9aff9372e6 HTTP/1.1" 200 None 2026-03-31 09:02:43,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:02:44,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:02:44,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:02:44,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf22a64fa9aff9372e6 HTTP/1.1" 200 None 2026-03-31 09:02:44,124 - lava_callback - INFO - Completed processing callback for node 69cb8bf22a64fa9aff9372e6 INFO: 172.19.0.1:50120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50618 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:03:07,694 - lava_callback - INFO - Received callback for node 69cb8be82a64fa9aff9372c9 INFO: 172.19.0.1:50626 - "POST /node/69cb8be82a64fa9aff9372c9 HTTP/1.0" 202 Accepted 2026-03-31 09:03:07,701 - lava_callback - INFO - Starting async job submit for node 69cb8be82a64fa9aff9372c9 2026-03-31 09:03:07,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:07,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be82a64fa9aff9372c9 HTTP/1.1" 200 None 2026-03-31 09:03:07,750 - lava_callback - DEBUG - Processing node 69cb8be82a64fa9aff9372c9: baseline-arm64 2026-03-31 09:03:08,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:08,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 09:03:08,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:03:08,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:03:08,722 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be82a64fa9aff9372c9/log.txt.gz 2026-03-31 09:03:09,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:03:09,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:03:09,383 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be82a64fa9aff9372c9/lava-logs.yaml 2026-03-31 09:03:09,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:03:09,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:03:09,561 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be82a64fa9aff9372c9/lava_callback.json.gz 2026-03-31 09:03:09,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:09,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be82a64fa9aff9372c9 HTTP/1.1" 200 None 2026-03-31 09:03:09,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:09,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:03:09,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:09,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be82a64fa9aff9372c9 HTTP/1.1" 200 None 2026-03-31 09:03:09,658 - lava_callback - INFO - Completed processing callback for node 69cb8be82a64fa9aff9372c9 INFO: 172.19.0.1:35938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32860 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:03:38,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:38,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:32862 - "GET /node/69b90f330ec88e58bad0d13c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47010 - "GET /node/69b912060ec88e58bad0d7b1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51444 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:07:27,636 - lava_callback - INFO - Received callback for node 69c8ebbe6a4c026e163489ca INFO: 172.19.0.1:51450 - "POST /node/69c8ebbe6a4c026e163489ca HTTP/1.0" 202 Accepted 2026-03-31 09:07:27,645 - lava_callback - INFO - Starting async job submit for node 69c8ebbe6a4c026e163489ca 2026-03-31 09:07:27,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:07:27,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ebbe6a4c026e163489ca HTTP/1.1" 200 None 2026-03-31 09:07:27,687 - lava_callback - DEBUG - Processing node 69c8ebbe6a4c026e163489ca: baseline-arm-clabbe 2026-03-31 09:07:27,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ebbe6a4c026e163489ca 2026-03-31 09:07:27,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:07:27,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ebbe6a4c026e163489ca HTTP/1.1" 200 None 2026-03-31 09:07:27,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:07:27,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 09:07:27,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:07:27,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ebbe6a4c026e163489ca HTTP/1.1" 200 None 2026-03-31 09:07:27,775 - lava_callback - ERROR - Error processing callback for node 69c8ebbe6a4c026e163489ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38338 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:15:20,180 - lava_callback - INFO - Received callback for node 69cb8bf72a64fa9aff937312 INFO: 172.19.0.1:35978 - "POST /node/69cb8bf72a64fa9aff937312 HTTP/1.0" 202 Accepted 2026-03-31 09:15:20,213 - lava_callback - INFO - Starting async job submit for node 69cb8bf72a64fa9aff937312 2026-03-31 09:15:20,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:15:20,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf72a64fa9aff937312 HTTP/1.1" 200 None 2026-03-31 09:15:20,404 - lava_callback - DEBUG - Processing node 69cb8bf72a64fa9aff937312: baseline-nfs-arm64 2026-03-31 09:15:21,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:15:21,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:15:21,698 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf72a64fa9aff937312/log.txt.gz 2026-03-31 09:15:22,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:15:22,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:15:22,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf72a64fa9aff937312/lava-logs.yaml 2026-03-31 09:15:22,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:15:22,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:15:22,809 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf72a64fa9aff937312/lava_callback.json.gz DEBUG: Setup failed for 69cb8bf72a64fa9aff937312. Transitting job node result: incomplete -> fail 2026-03-31 09:15:22,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:15:22,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf72a64fa9aff937312 HTTP/1.1" 200 None 2026-03-31 09:15:22,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:15:22,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:15:22,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:15:22,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf72a64fa9aff937312 HTTP/1.1" 200 None 2026-03-31 09:15:23,053 - lava_callback - INFO - Completed processing callback for node 69cb8bf72a64fa9aff937312 INFO: 172.19.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40032 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:15:38,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:15:38,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38200 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:20:00,803 - lava_callback - INFO - Received callback for node 69cb8bed2a64fa9aff9372d5 INFO: 172.19.0.1:37416 - "POST /node/69cb8bed2a64fa9aff9372d5 HTTP/1.0" 202 Accepted 2026-03-31 09:20:00,810 - lava_callback - INFO - Starting async job submit for node 69cb8bed2a64fa9aff9372d5 2026-03-31 09:20:00,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:20:00,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bed2a64fa9aff9372d5 HTTP/1.1" 200 None 2026-03-31 09:20:00,851 - lava_callback - DEBUG - Processing node 69cb8bed2a64fa9aff9372d5: baseline-arm64 2026-03-31 09:20:01,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:20:01,537 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:20:01,539 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bed2a64fa9aff9372d5/log.txt.gz 2026-03-31 09:20:01,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:20:02,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:20:02,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bed2a64fa9aff9372d5/lava-logs.yaml 2026-03-31 09:20:02,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:20:02,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:20:02,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bed2a64fa9aff9372d5/lava_callback.json.gz 2026-03-31 09:20:02,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:20:02,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bed2a64fa9aff9372d5 HTTP/1.1" 200 None 2026-03-31 09:20:02,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:20:02,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:20:02,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:20:02,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bed2a64fa9aff9372d5 HTTP/1.1" 200 None 2026-03-31 09:20:02,417 - lava_callback - INFO - Completed processing callback for node 69cb8bed2a64fa9aff9372d5 INFO: 172.19.0.1:37422 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:20:08,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:20:08,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:57298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44396 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:21:10,498 - lava_callback - INFO - Received callback for node 69cb91ef2a64fa9aff93760b INFO: 172.19.0.1:46708 - "POST /node/69cb91ef2a64fa9aff93760b HTTP/1.0" 202 Accepted 2026-03-31 09:21:10,508 - lava_callback - INFO - Starting async job submit for node 69cb91ef2a64fa9aff93760b 2026-03-31 09:21:10,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:21:10,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb91ef2a64fa9aff93760b HTTP/1.1" 200 None 2026-03-31 09:21:10,537 - lava_callback - DEBUG - Processing node 69cb91ef2a64fa9aff93760b: baseline-nfs-arm64 2026-03-31 09:21:10,539 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:21:10,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:21:10,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:21:10,708 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb91ef2a64fa9aff93760b/lava-logs.yaml 2026-03-31 09:21:10,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:21:10,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:21:10,845 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb91ef2a64fa9aff93760b/lava_callback.json.gz 2026-03-31 09:21:10,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:21:10,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb91ef2a64fa9aff93760b HTTP/1.1" 200 None 2026-03-31 09:21:10,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:21:10,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:21:10,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:21:10,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb91ef2a64fa9aff93760b HTTP/1.1" 200 None 2026-03-31 09:21:10,928 - lava_callback - INFO - Completed processing callback for node 69cb91ef2a64fa9aff93760b INFO: 172.19.0.1:46714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60774 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:21:38,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:21:38,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36604 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:24:08,612 - lava_callback - INFO - Received callback for node 69c8efab6a4c026e16348ae6 INFO: 172.19.0.1:36606 - "POST /node/69c8efab6a4c026e16348ae6 HTTP/1.0" 202 Accepted 2026-03-31 09:24:08,618 - lava_callback - INFO - Starting async job submit for node 69c8efab6a4c026e16348ae6 2026-03-31 09:24:08,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:08,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8efab6a4c026e16348ae6 HTTP/1.1" 200 None 2026-03-31 09:24:08,640 - lava_callback - DEBUG - Processing node 69c8efab6a4c026e16348ae6: baseline-arm64-baylibre 2026-03-31 09:24:08,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8efab6a4c026e16348ae6 2026-03-31 09:24:08,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:08,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8efab6a4c026e16348ae6 HTTP/1.1" 200 None 2026-03-31 09:24:08,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:08,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:24:08,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:08,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8efab6a4c026e16348ae6 HTTP/1.1" 200 None 2026-03-31 09:24:08,753 - lava_callback - ERROR - Error processing callback for node 69c8efab6a4c026e16348ae6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 09:24:11,264 - lava_callback - INFO - Received callback for node 69cb8bfe2a64fa9aff937321 INFO: 172.19.0.1:42568 - "POST /node/69cb8bfe2a64fa9aff937321 HTTP/1.0" 202 Accepted 2026-03-31 09:24:11,272 - lava_callback - INFO - Starting async job submit for node 69cb8bfe2a64fa9aff937321 2026-03-31 09:24:11,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:11,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfe2a64fa9aff937321 HTTP/1.1" 200 None 2026-03-31 09:24:11,320 - lava_callback - DEBUG - Processing node 69cb8bfe2a64fa9aff937321: baseline-nfs-arm64 2026-03-31 09:24:11,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:24:12,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:24:12,090 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfe2a64fa9aff937321/log.txt.gz 2026-03-31 09:24:12,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:24:12,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:24:12,562 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfe2a64fa9aff937321/lava-logs.yaml 2026-03-31 09:24:12,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:24:12,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:24:12,746 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfe2a64fa9aff937321/lava_callback.json.gz 2026-03-31 09:24:12,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:12,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfe2a64fa9aff937321 HTTP/1.1" 200 None 2026-03-31 09:24:12,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:12,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:24:12,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:12,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bfe2a64fa9aff937321 HTTP/1.1" 200 None 2026-03-31 09:24:12,901 - lava_callback - INFO - Completed processing callback for node 69cb8bfe2a64fa9aff937321 INFO: 172.19.0.1:42584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49322 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:24:38,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:38,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43538 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:25:28,661 - lava_callback - INFO - Received callback for node 69c8effb6a4c026e16348af2 INFO: 172.19.0.1:43548 - "POST /node/69c8effb6a4c026e16348af2 HTTP/1.0" 202 Accepted 2026-03-31 09:25:28,667 - lava_callback - INFO - Starting async job submit for node 69c8effb6a4c026e16348af2 2026-03-31 09:25:28,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:25:28,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8effb6a4c026e16348af2 HTTP/1.1" 200 None 2026-03-31 09:25:28,683 - lava_callback - DEBUG - Processing node 69c8effb6a4c026e16348af2: baseline-arm64-baylibre 2026-03-31 09:25:28,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8effb6a4c026e16348af2 2026-03-31 09:25:28,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:25:28,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8effb6a4c026e16348af2 HTTP/1.1" 200 None 2026-03-31 09:25:28,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:25:28,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:25:28,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:25:28,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8effb6a4c026e16348af2 HTTP/1.1" 200 None 2026-03-31 09:25:28,760 - lava_callback - ERROR - Error processing callback for node 69c8effb6a4c026e16348af2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58600 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:27:24,217 - lava_callback - INFO - Received callback for node 69cb93142a64fa9aff937644 INFO: 172.19.0.1:60880 - "POST /node/69cb93142a64fa9aff937644 HTTP/1.0" 202 Accepted 2026-03-31 09:27:24,224 - lava_callback - INFO - Starting async job submit for node 69cb93142a64fa9aff937644 2026-03-31 09:27:24,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:24,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb93142a64fa9aff937644 HTTP/1.1" 200 None 2026-03-31 09:27:24,269 - lava_callback - DEBUG - Processing node 69cb93142a64fa9aff937644: baseline-arm64 2026-03-31 09:27:24,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:27:24,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:27:24,639 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93142a64fa9aff937644/log.txt.gz 2026-03-31 09:27:24,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:27:24,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:27:24,954 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93142a64fa9aff937644/lava-logs.yaml 2026-03-31 09:27:24,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:27:25,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:27:25,121 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93142a64fa9aff937644/lava_callback.json.gz 2026-03-31 09:27:25,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:25,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb93142a64fa9aff937644 HTTP/1.1" 200 None 2026-03-31 09:27:25,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:25,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:27:25,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:25,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb93142a64fa9aff937644 HTTP/1.1" 200 None 2026-03-31 09:27:25,263 - lava_callback - INFO - Completed processing callback for node 69cb93142a64fa9aff937644 INFO: 172.19.0.1:60884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38588 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:27:38,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:38,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51376 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:27:56,171 - lava_callback - INFO - Received callback for node 69cb935f2a64fa9aff93764c INFO: 172.19.0.1:33230 - "POST /node/69cb935f2a64fa9aff93764c HTTP/1.0" 202 Accepted 2026-03-31 09:27:56,177 - lava_callback - INFO - Starting async job submit for node 69cb935f2a64fa9aff93764c 2026-03-31 09:27:56,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:56,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb935f2a64fa9aff93764c HTTP/1.1" 200 None 2026-03-31 09:27:56,222 - lava_callback - DEBUG - Processing node 69cb935f2a64fa9aff93764c: baseline-arm64 2026-03-31 09:27:56,352 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:27:56,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:33232 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:27:56,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:27:56,603 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb935f2a64fa9aff93764c/lava-logs.yaml 2026-03-31 09:27:56,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:27:56,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:27:56,723 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb935f2a64fa9aff93764c/lava_callback.json.gz 2026-03-31 09:27:56,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:56,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb935f2a64fa9aff93764c HTTP/1.1" 200 None 2026-03-31 09:27:56,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:56,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:27:56,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:56,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb935f2a64fa9aff93764c HTTP/1.1" 200 None 2026-03-31 09:27:56,897 - lava_callback - INFO - Completed processing callback for node 69cb935f2a64fa9aff93764c INFO: 172.19.0.1:35108 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:28:08,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:28:08,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 09:28:08,845 - lava_callback - INFO - Received callback for node 69c8f0976a4c026e16348b0c INFO: 172.19.0.1:35110 - "POST /node/69c8f0976a4c026e16348b0c HTTP/1.0" 202 Accepted 2026-03-31 09:28:08,851 - lava_callback - INFO - Starting async job submit for node 69c8f0976a4c026e16348b0c 2026-03-31 09:28:08,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:28:08,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f0976a4c026e16348b0c HTTP/1.1" 200 None 2026-03-31 09:28:08,869 - lava_callback - DEBUG - Processing node 69c8f0976a4c026e16348b0c: baseline-arm64-baylibre 2026-03-31 09:28:08,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f0976a4c026e16348b0c 2026-03-31 09:28:08,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:28:08,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f0976a4c026e16348b0c HTTP/1.1" 200 None 2026-03-31 09:28:08,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:28:08,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:28:08,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:28:08,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f0976a4c026e16348b0c HTTP/1.1" 200 None 2026-03-31 09:28:08,926 - lava_callback - ERROR - Error processing callback for node 69c8f0976a4c026e16348b0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58146 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 09:29:16,414 - lava_callback - INFO - Received callback for node 69cb93b22a64fa9aff93765f INFO: 172.19.0.1:58150 - "POST /node/69cb93b22a64fa9aff93765f HTTP/1.0" 202 Accepted 2026-03-31 09:29:16,420 - lava_callback - INFO - Starting async job submit for node 69cb93b22a64fa9aff93765f 2026-03-31 09:29:16,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:16,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb93b22a64fa9aff93765f HTTP/1.1" 200 None 2026-03-31 09:29:16,457 - lava_callback - DEBUG - Processing node 69cb93b22a64fa9aff93765f: baseline-arm64 INFO: 172.19.0.1:58158 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:29:16,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:29:17,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:29:17,084 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93b22a64fa9aff93765f/log.txt.gz 2026-03-31 09:29:17,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:29:17,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:29:17,530 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93b22a64fa9aff93765f/lava-logs.yaml 2026-03-31 09:29:17,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:29:17,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:29:17,698 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93b22a64fa9aff93765f/lava_callback.json.gz 2026-03-31 09:29:17,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:17,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb93b22a64fa9aff93765f HTTP/1.1" 200 None 2026-03-31 09:29:17,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:17,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:29:17,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:17,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb93b22a64fa9aff93765f HTTP/1.1" 200 None 2026-03-31 09:29:17,798 - lava_callback - INFO - Completed processing callback for node 69cb93b22a64fa9aff93765f INFO: 172.19.0.1:58950 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:29:28,887 - lava_callback - INFO - Received callback for node 69c8f0e46a4c026e16348b1b INFO: 172.19.0.1:58962 - "POST /node/69c8f0e46a4c026e16348b1b HTTP/1.0" 202 Accepted 2026-03-31 09:29:28,893 - lava_callback - INFO - Starting async job submit for node 69c8f0e46a4c026e16348b1b 2026-03-31 09:29:28,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:28,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f0e46a4c026e16348b1b HTTP/1.1" 200 None 2026-03-31 09:29:28,910 - lava_callback - DEBUG - Processing node 69c8f0e46a4c026e16348b1b: baseline-arm64-baylibre 2026-03-31 09:29:28,910 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f0e46a4c026e16348b1b 2026-03-31 09:29:28,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:28,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f0e46a4c026e16348b1b HTTP/1.1" 200 None 2026-03-31 09:29:28,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:28,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:29:28,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:28,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f0e46a4c026e16348b1b HTTP/1.1" 200 None 2026-03-31 09:29:28,962 - lava_callback - ERROR - Error processing callback for node 69c8f0e46a4c026e16348b1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39592 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:29:38,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:38,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:41246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56498 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:30:28,918 - lava_callback - INFO - Received callback for node 69c8f1326a4c026e16348b21 INFO: 172.19.0.1:56514 - "POST /node/69c8f1326a4c026e16348b21 HTTP/1.0" 202 Accepted 2026-03-31 09:30:28,924 - lava_callback - INFO - Starting async job submit for node 69c8f1326a4c026e16348b21 2026-03-31 09:30:28,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:30:28,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f1326a4c026e16348b21 HTTP/1.1" 200 None 2026-03-31 09:30:28,940 - lava_callback - DEBUG - Processing node 69c8f1326a4c026e16348b21: baseline-arm64-baylibre 2026-03-31 09:30:28,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f1326a4c026e16348b21 2026-03-31 09:30:28,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:30:28,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f1326a4c026e16348b21 HTTP/1.1" 200 None 2026-03-31 09:30:28,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:30:28,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:30:28,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:30:29,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f1326a4c026e16348b21 HTTP/1.1" 200 None 2026-03-31 09:30:29,022 - lava_callback - ERROR - Error processing callback for node 69c8f1326a4c026e16348b21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41000 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:31:49,129 - lava_callback - INFO - Received callback for node 69cb94472a64fa9aff937684 INFO: 172.19.0.1:41006 - "POST /node/69cb94472a64fa9aff937684 HTTP/1.0" 202 Accepted 2026-03-31 09:31:49,135 - lava_callback - INFO - Starting async job submit for node 69cb94472a64fa9aff937684 2026-03-31 09:31:49,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:49,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94472a64fa9aff937684 HTTP/1.1" 200 None 2026-03-31 09:31:49,182 - lava_callback - DEBUG - Processing node 69cb94472a64fa9aff937684: baseline-arm64 2026-03-31 09:31:49,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:31:49,474 - lava_callback - INFO - Received callback for node 69c8f17e6a4c026e16348b2d INFO: 172.19.0.1:41016 - "POST /node/69c8f17e6a4c026e16348b2d HTTP/1.0" 202 Accepted 2026-03-31 09:31:49,480 - lava_callback - INFO - Starting async job submit for node 69c8f17e6a4c026e16348b2d 2026-03-31 09:31:49,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:49,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f17e6a4c026e16348b2d HTTP/1.1" 200 None 2026-03-31 09:31:49,497 - lava_callback - DEBUG - Processing node 69c8f17e6a4c026e16348b2d: baseline-arm64-baylibre 2026-03-31 09:31:49,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f17e6a4c026e16348b2d 2026-03-31 09:31:49,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:49,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f17e6a4c026e16348b2d HTTP/1.1" 200 None 2026-03-31 09:31:49,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:49,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:31:49,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:49,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:31:49,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb94472a64fa9aff937684/log.txt.gz 2026-03-31 09:31:49,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f17e6a4c026e16348b2d HTTP/1.1" 200 None 2026-03-31 09:31:49,582 - lava_callback - ERROR - Error processing callback for node 69c8f17e6a4c026e16348b2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 09:31:49,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:31:49,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:31:49,990 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb94472a64fa9aff937684/lava-logs.yaml 2026-03-31 09:31:49,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:31:50,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:31:50,162 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb94472a64fa9aff937684/lava_callback.json.gz 2026-03-31 09:31:50,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:50,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94472a64fa9aff937684 HTTP/1.1" 200 None 2026-03-31 09:31:50,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:50,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:31:50,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:50,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb94472a64fa9aff937684 HTTP/1.1" 200 None 2026-03-31 09:31:50,281 - lava_callback - INFO - Completed processing callback for node 69cb94472a64fa9aff937684 INFO: 172.19.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43774 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:32:08,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:32:08,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:36354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57512 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:33:09,087 - lava_callback - INFO - Received callback for node 69c8f1cc6a4c026e16348b40 INFO: 172.19.0.1:57528 - "POST /node/69c8f1cc6a4c026e16348b40 HTTP/1.0" 202 Accepted 2026-03-31 09:33:09,093 - lava_callback - INFO - Starting async job submit for node 69c8f1cc6a4c026e16348b40 2026-03-31 09:33:09,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:33:09,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f1cc6a4c026e16348b40 HTTP/1.1" 200 None 2026-03-31 09:33:09,110 - lava_callback - DEBUG - Processing node 69c8f1cc6a4c026e16348b40: baseline-arm64-baylibre 2026-03-31 09:33:09,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f1cc6a4c026e16348b40 2026-03-31 09:33:09,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:33:09,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f1cc6a4c026e16348b40 HTTP/1.1" 200 None 2026-03-31 09:33:09,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:33:09,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:33:09,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:33:09,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f1cc6a4c026e16348b40 HTTP/1.1" 200 None 2026-03-31 09:33:09,180 - lava_callback - ERROR - Error processing callback for node 69c8f1cc6a4c026e16348b40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50532 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:34:29,157 - lava_callback - INFO - Received callback for node 69c8f2176a4c026e16348b4f INFO: 172.19.0.1:50536 - "POST /node/69c8f2176a4c026e16348b4f HTTP/1.0" 202 Accepted 2026-03-31 09:34:29,163 - lava_callback - INFO - Starting async job submit for node 69c8f2176a4c026e16348b4f 2026-03-31 09:34:29,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:34:29,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2176a4c026e16348b4f HTTP/1.1" 200 None 2026-03-31 09:34:29,179 - lava_callback - DEBUG - Processing node 69c8f2176a4c026e16348b4f: baseline-arm64-baylibre 2026-03-31 09:34:29,179 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f2176a4c026e16348b4f 2026-03-31 09:34:29,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:34:29,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2176a4c026e16348b4f HTTP/1.1" 200 None 2026-03-31 09:34:29,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:34:29,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:34:29,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:34:29,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f2176a4c026e16348b4f HTTP/1.1" 200 None 2026-03-31 09:34:29,248 - lava_callback - ERROR - Error processing callback for node 69c8f2176a4c026e16348b4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35784 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:35:07,518 - lava_callback - INFO - Received callback for node 69cb912c2a64fa9aff9375da INFO: 172.19.0.1:35788 - "POST /node/69cb912c2a64fa9aff9375da HTTP/1.0" 202 Accepted 2026-03-31 09:35:07,525 - lava_callback - INFO - Starting async job submit for node 69cb912c2a64fa9aff9375da 2026-03-31 09:35:07,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:07,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb912c2a64fa9aff9375da HTTP/1.1" 200 None 2026-03-31 09:35:07,594 - lava_callback - DEBUG - Processing node 69cb912c2a64fa9aff9375da: baseline-arm 2026-03-31 09:35:08,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:35:08,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:35:08,443 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb912c2a64fa9aff9375da/log.txt.gz 2026-03-31 09:35:08,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:35:09,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:35:09,099 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb912c2a64fa9aff9375da/lava-logs.yaml 2026-03-31 09:35:09,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:35:09,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:35:09,294 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb912c2a64fa9aff9375da/lava_callback.json.gz DEBUG: Setup failed for 69cb912c2a64fa9aff9375da. Transitting job node result: incomplete -> fail 2026-03-31 09:35:09,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:09,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb912c2a64fa9aff9375da HTTP/1.1" 200 None 2026-03-31 09:35:09,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:09,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 09:35:09,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:09,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb912c2a64fa9aff9375da HTTP/1.1" 200 None 2026-03-31 09:35:09,566 - lava_callback - INFO - Completed processing callback for node 69cb912c2a64fa9aff9375da INFO: 172.19.0.1:46332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52306 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:35:38,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:38,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42742 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:35:49,253 - lava_callback - INFO - Received callback for node 69c8f2626a4c026e16348b5e INFO: 172.19.0.1:42754 - "POST /node/69c8f2626a4c026e16348b5e HTTP/1.0" 202 Accepted 2026-03-31 09:35:49,258 - lava_callback - INFO - Starting async job submit for node 69c8f2626a4c026e16348b5e 2026-03-31 09:35:49,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:49,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2626a4c026e16348b5e HTTP/1.1" 200 None 2026-03-31 09:35:49,274 - lava_callback - DEBUG - Processing node 69c8f2626a4c026e16348b5e: baseline-arm64-baylibre 2026-03-31 09:35:49,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f2626a4c026e16348b5e 2026-03-31 09:35:49,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:49,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2626a4c026e16348b5e HTTP/1.1" 200 None 2026-03-31 09:35:49,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:49,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:35:49,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:49,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f2626a4c026e16348b5e HTTP/1.1" 200 None 2026-03-31 09:35:49,341 - lava_callback - ERROR - Error processing callback for node 69c8f2626a4c026e16348b5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46516 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:36:49,237 - lava_callback - INFO - Received callback for node 69c8f2ad6a4c026e16348b6f INFO: 172.19.0.1:46524 - "POST /node/69c8f2ad6a4c026e16348b6f HTTP/1.0" 202 Accepted 2026-03-31 09:36:49,244 - lava_callback - INFO - Starting async job submit for node 69c8f2ad6a4c026e16348b6f 2026-03-31 09:36:49,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:36:49,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2ad6a4c026e16348b6f HTTP/1.1" 200 None 2026-03-31 09:36:49,261 - lava_callback - DEBUG - Processing node 69c8f2ad6a4c026e16348b6f: baseline-arm64-baylibre 2026-03-31 09:36:49,261 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f2ad6a4c026e16348b6f 2026-03-31 09:36:49,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:36:49,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2ad6a4c026e16348b6f HTTP/1.1" 200 None 2026-03-31 09:36:49,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:36:49,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:36:49,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:36:49,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f2ad6a4c026e16348b6f HTTP/1.1" 200 None 2026-03-31 09:36:49,325 - lava_callback - ERROR - Error processing callback for node 69c8f2ad6a4c026e16348b6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:32802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58290 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:38:28,950 - lava_callback - INFO - Received callback for node 69c8f2ff6a4c026e16348b7b INFO: 172.19.0.1:58292 - "POST /node/69c8f2ff6a4c026e16348b7b HTTP/1.0" 202 Accepted 2026-03-31 09:38:28,956 - lava_callback - INFO - Starting async job submit for node 69c8f2ff6a4c026e16348b7b 2026-03-31 09:38:28,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:38:28,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2ff6a4c026e16348b7b HTTP/1.1" 200 None 2026-03-31 09:38:28,973 - lava_callback - DEBUG - Processing node 69c8f2ff6a4c026e16348b7b: baseline-arm64-clabbe 2026-03-31 09:38:28,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f2ff6a4c026e16348b7b 2026-03-31 09:38:28,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:38:28,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2ff6a4c026e16348b7b HTTP/1.1" 200 None 2026-03-31 09:38:28,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:38:29,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:38:29,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:38:29,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f2ff6a4c026e16348b7b HTTP/1.1" 200 None 2026-03-31 09:38:29,034 - lava_callback - ERROR - Error processing callback for node 69c8f2ff6a4c026e16348b7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54524 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:39:48,974 - lava_callback - INFO - Received callback for node 69c8f34e6a4c026e16348b80 INFO: 172.19.0.1:54528 - "POST /node/69c8f34e6a4c026e16348b80 HTTP/1.0" 202 Accepted 2026-03-31 09:39:48,979 - lava_callback - INFO - Starting async job submit for node 69c8f34e6a4c026e16348b80 2026-03-31 09:39:48,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:39:48,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f34e6a4c026e16348b80 HTTP/1.1" 200 None 2026-03-31 09:39:48,997 - lava_callback - DEBUG - Processing node 69c8f34e6a4c026e16348b80: baseline-arm64-clabbe 2026-03-31 09:39:48,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f34e6a4c026e16348b80 2026-03-31 09:39:48,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:39:49,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f34e6a4c026e16348b80 HTTP/1.1" 200 None 2026-03-31 09:39:49,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:39:49,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:39:49,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:39:49,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f34e6a4c026e16348b80 HTTP/1.1" 200 None 2026-03-31 09:39:49,060 - lava_callback - ERROR - Error processing callback for node 69c8f34e6a4c026e16348b80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39458 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:40:49,043 - lava_callback - INFO - Received callback for node 69c8f39d6a4c026e16348b8c INFO: 172.19.0.1:39472 - "POST /node/69c8f39d6a4c026e16348b8c HTTP/1.0" 202 Accepted 2026-03-31 09:40:49,048 - lava_callback - INFO - Starting async job submit for node 69c8f39d6a4c026e16348b8c 2026-03-31 09:40:49,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:40:49,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f39d6a4c026e16348b8c HTTP/1.1" 200 None 2026-03-31 09:40:49,069 - lava_callback - DEBUG - Processing node 69c8f39d6a4c026e16348b8c: baseline-arm64-clabbe 2026-03-31 09:40:49,070 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f39d6a4c026e16348b8c 2026-03-31 09:40:49,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:40:49,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f39d6a4c026e16348b8c HTTP/1.1" 200 None 2026-03-31 09:40:49,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:40:49,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:40:49,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:40:49,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f39d6a4c026e16348b8c HTTP/1.1" 200 None 2026-03-31 09:40:49,145 - lava_callback - ERROR - Error processing callback for node 69c8f39d6a4c026e16348b8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41944 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:41:12,541 - lava_callback - INFO - Received callback for node 69cb96a82a64fa9aff93770c INFO: 172.19.0.1:36062 - "POST /node/69cb96a82a64fa9aff93770c HTTP/1.0" 202 Accepted 2026-03-31 09:41:12,547 - lava_callback - INFO - Starting async job submit for node 69cb96a82a64fa9aff93770c 2026-03-31 09:41:12,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:41:12,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb96a82a64fa9aff93770c HTTP/1.1" 200 None 2026-03-31 09:41:12,567 - lava_callback - DEBUG - Processing node 69cb96a82a64fa9aff93770c: baseline-nfs-arm64 2026-03-31 09:41:12,568 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:41:12,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:41:12,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:41:12,698 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb96a82a64fa9aff93770c/lava-logs.yaml 2026-03-31 09:41:12,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:41:12,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:41:12,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb96a82a64fa9aff93770c/lava_callback.json.gz 2026-03-31 09:41:12,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:41:12,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb96a82a64fa9aff93770c HTTP/1.1" 200 None 2026-03-31 09:41:12,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:41:12,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:41:12,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:41:12,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb96a82a64fa9aff93770c HTTP/1.1" 200 None 2026-03-31 09:41:12,893 - lava_callback - INFO - Completed processing callback for node 69cb96a82a64fa9aff93770c INFO: 172.19.0.1:36074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51990 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:41:38,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:41:38,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:48412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33016 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:42:09,117 - lava_callback - INFO - Received callback for node 69c8f3ea6a4c026e16348b92 INFO: 172.19.0.1:33020 - "POST /node/69c8f3ea6a4c026e16348b92 HTTP/1.0" 202 Accepted 2026-03-31 09:42:09,122 - lava_callback - INFO - Starting async job submit for node 69c8f3ea6a4c026e16348b92 2026-03-31 09:42:09,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:09,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f3ea6a4c026e16348b92 HTTP/1.1" 200 None 2026-03-31 09:42:09,143 - lava_callback - DEBUG - Processing node 69c8f3ea6a4c026e16348b92: baseline-arm64-clabbe 2026-03-31 09:42:09,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f3ea6a4c026e16348b92 2026-03-31 09:42:09,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:09,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f3ea6a4c026e16348b92 HTTP/1.1" 200 None 2026-03-31 09:42:09,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:09,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:42:09,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:09,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f3ea6a4c026e16348b92 HTTP/1.1" 200 None 2026-03-31 09:42:09,206 - lava_callback - ERROR - Error processing callback for node 69c8f3ea6a4c026e16348b92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32868 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:42:46,307 - lava_callback - INFO - Received callback for node 69cb954f2a64fa9aff9376d4 INFO: 172.19.0.1:60078 - "POST /node/69cb954f2a64fa9aff9376d4 HTTP/1.0" 202 Accepted 2026-03-31 09:42:46,315 - lava_callback - INFO - Starting async job submit for node 69cb954f2a64fa9aff9376d4 2026-03-31 09:42:46,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:46,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb954f2a64fa9aff9376d4 HTTP/1.1" 200 None 2026-03-31 09:42:46,403 - lava_callback - DEBUG - Processing node 69cb954f2a64fa9aff9376d4: baseline-arm64 INFO: 172.19.0.1:60094 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:42:46,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:42:46,936 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:42:46,938 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb954f2a64fa9aff9376d4/log.txt.gz 2026-03-31 09:42:47,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:42:47,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:42:47,370 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb954f2a64fa9aff9376d4/lava-logs.yaml 2026-03-31 09:42:47,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:42:47,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:42:47,524 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb954f2a64fa9aff9376d4/lava_callback.json.gz DEBUG: Setup failed for 69cb954f2a64fa9aff9376d4. Transitting job node result: incomplete -> fail 2026-03-31 09:42:47,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:47,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb954f2a64fa9aff9376d4 HTTP/1.1" 200 None 2026-03-31 09:42:47,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:47,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:42:47,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:47,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb954f2a64fa9aff9376d4 HTTP/1.1" 200 None 2026-03-31 09:42:47,736 - lava_callback - INFO - Completed processing callback for node 69cb954f2a64fa9aff9376d4 INFO: 172.19.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42894 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:43:08,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:43:08,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:48724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32972 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:43:29,144 - lava_callback - INFO - Received callback for node 69c8f4386a4c026e16348b9d INFO: 172.19.0.1:32988 - "POST /node/69c8f4386a4c026e16348b9d HTTP/1.0" 202 Accepted 2026-03-31 09:43:29,152 - lava_callback - INFO - Starting async job submit for node 69c8f4386a4c026e16348b9d 2026-03-31 09:43:29,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:43:29,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f4386a4c026e16348b9d HTTP/1.1" 200 None 2026-03-31 09:43:29,168 - lava_callback - DEBUG - Processing node 69c8f4386a4c026e16348b9d: baseline-arm64-clabbe 2026-03-31 09:43:29,169 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f4386a4c026e16348b9d 2026-03-31 09:43:29,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:43:29,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f4386a4c026e16348b9d HTTP/1.1" 200 None 2026-03-31 09:43:29,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:43:29,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:43:29,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:43:29,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f4386a4c026e16348b9d HTTP/1.1" 200 None 2026-03-31 09:43:29,243 - lava_callback - ERROR - Error processing callback for node 69c8f4386a4c026e16348b9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35868 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:44:49,237 - lava_callback - INFO - Received callback for node 69c8f4856a4c026e16348ba9 INFO: 172.19.0.1:35880 - "POST /node/69c8f4856a4c026e16348ba9 HTTP/1.0" 202 Accepted 2026-03-31 09:44:49,244 - lava_callback - INFO - Starting async job submit for node 69c8f4856a4c026e16348ba9 2026-03-31 09:44:49,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:44:49,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f4856a4c026e16348ba9 HTTP/1.1" 200 None 2026-03-31 09:44:49,266 - lava_callback - DEBUG - Processing node 69c8f4856a4c026e16348ba9: baseline-arm64-clabbe 2026-03-31 09:44:49,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f4856a4c026e16348ba9 2026-03-31 09:44:49,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:44:49,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f4856a4c026e16348ba9 HTTP/1.1" 200 None 2026-03-31 09:44:49,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:44:49,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:44:49,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:44:49,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f4856a4c026e16348ba9 HTTP/1.1" 200 None 2026-03-31 09:44:49,326 - lava_callback - ERROR - Error processing callback for node 69c8f4856a4c026e16348ba9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54006 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:48:29,991 - lava_callback - INFO - Received callback for node 69c8f5666a4c026e16348bc2 INFO: 172.19.0.1:46096 - "POST /node/69c8f5666a4c026e16348bc2 HTTP/1.0" 202 Accepted 2026-03-31 09:48:29,996 - lava_callback - INFO - Starting async job submit for node 69c8f5666a4c026e16348bc2 2026-03-31 09:48:29,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:48:30,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f5666a4c026e16348bc2 HTTP/1.1" 200 None 2026-03-31 09:48:30,012 - lava_callback - DEBUG - Processing node 69c8f5666a4c026e16348bc2: baseline-arm64-baylibre 2026-03-31 09:48:30,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f5666a4c026e16348bc2 2026-03-31 09:48:30,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:48:30,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f5666a4c026e16348bc2 HTTP/1.1" 200 None 2026-03-31 09:48:30,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:48:30,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:48:30,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:48:30,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f5666a4c026e16348bc2 HTTP/1.1" 200 None 2026-03-31 09:48:30,074 - lava_callback - ERROR - Error processing callback for node 69c8f5666a4c026e16348bc2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36014 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:49:12,958 - lava_callback - INFO - Received callback for node 69cb98662a64fa9aff93774c INFO: 172.19.0.1:47546 - "POST /node/69cb98662a64fa9aff93774c HTTP/1.0" 202 Accepted 2026-03-31 09:49:12,965 - lava_callback - INFO - Starting async job submit for node 69cb98662a64fa9aff93774c 2026-03-31 09:49:12,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:13,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb98662a64fa9aff93774c HTTP/1.1" 200 None 2026-03-31 09:49:13,009 - lava_callback - DEBUG - Processing node 69cb98662a64fa9aff93774c: baseline-arm64 2026-03-31 09:49:13,140 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:49:13,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:49:13,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:49:13,579 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb98662a64fa9aff93774c/lava-logs.yaml 2026-03-31 09:49:13,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:49:13,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:49:13,709 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb98662a64fa9aff93774c/lava_callback.json.gz 2026-03-31 09:49:13,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:13,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb98662a64fa9aff93774c HTTP/1.1" 200 None 2026-03-31 09:49:13,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:13,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:49:13,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:13,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb98662a64fa9aff93774c HTTP/1.1" 200 None 2026-03-31 09:49:13,861 - lava_callback - INFO - Completed processing callback for node 69cb98662a64fa9aff93774c INFO: 172.19.0.1:47562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47566 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:49:19,611 - lava_callback - INFO - Received callback for node 69cb94072a64fa9aff937670 INFO: 172.19.0.1:47572 - "POST /node/69cb94072a64fa9aff937670 HTTP/1.0" 202 Accepted 2026-03-31 09:49:19,657 - lava_callback - INFO - Starting async job submit for node 69cb94072a64fa9aff937670 2026-03-31 09:49:19,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:19,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94072a64fa9aff937670 HTTP/1.1" 200 None 2026-03-31 09:49:19,696 - lava_callback - DEBUG - Processing node 69cb94072a64fa9aff937670: baseline-arm-pengutronix 2026-03-31 09:49:19,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:49:19,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:49:19,930 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb94072a64fa9aff937670/log.txt.gz 2026-03-31 09:49:19,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:49:20,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:49:20,150 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb94072a64fa9aff937670/lava-logs.yaml 2026-03-31 09:49:20,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:49:20,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:49:20,275 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb94072a64fa9aff937670/lava_callback.json.gz DEBUG: Setup failed for 69cb94072a64fa9aff937670. Transitting job node result: incomplete -> fail 2026-03-31 09:49:20,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:20,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94072a64fa9aff937670 HTTP/1.1" 200 None 2026-03-31 09:49:20,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:20,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 09:49:20,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:20,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb94072a64fa9aff937670 HTTP/1.1" 200 None 2026-03-31 09:49:20,449 - lava_callback - INFO - Completed processing callback for node 69cb94072a64fa9aff937670 INFO: 172.19.0.1:35456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47872 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:49:38,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:38,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:47936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47952 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:49:50,002 - lava_callback - INFO - Received callback for node 69c8f5b26a4c026e16348bce INFO: 172.19.0.1:41364 - "POST /node/69c8f5b26a4c026e16348bce HTTP/1.0" 202 Accepted 2026-03-31 09:49:50,008 - lava_callback - INFO - Starting async job submit for node 69c8f5b26a4c026e16348bce 2026-03-31 09:49:50,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:50,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f5b26a4c026e16348bce HTTP/1.1" 200 None 2026-03-31 09:49:50,024 - lava_callback - DEBUG - Processing node 69c8f5b26a4c026e16348bce: baseline-arm64-baylibre 2026-03-31 09:49:50,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f5b26a4c026e16348bce 2026-03-31 09:49:50,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:50,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f5b26a4c026e16348bce HTTP/1.1" 200 None 2026-03-31 09:49:50,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:50,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:49:50,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:50,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f5b26a4c026e16348bce HTTP/1.1" 200 None 2026-03-31 09:49:50,079 - lava_callback - ERROR - Error processing callback for node 69c8f5b26a4c026e16348bce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48320 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:50:11,950 - lava_callback - INFO - Received callback for node 69cb96562a64fa9aff9376fe INFO: 172.19.0.1:59748 - "POST /node/69cb96562a64fa9aff9376fe HTTP/1.0" 202 Accepted 2026-03-31 09:50:11,956 - lava_callback - INFO - Starting async job submit for node 69cb96562a64fa9aff9376fe 2026-03-31 09:50:11,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:12,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb96562a64fa9aff9376fe HTTP/1.1" 200 None 2026-03-31 09:50:12,013 - lava_callback - DEBUG - Processing node 69cb96562a64fa9aff9376fe: baseline-arm64 2026-03-31 09:50:12,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:50:12,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:50:12,634 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb96562a64fa9aff9376fe/log.txt.gz 2026-03-31 09:50:12,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:50:13,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:50:13,096 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb96562a64fa9aff9376fe/lava-logs.yaml 2026-03-31 09:50:13,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:50:13,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:50:13,264 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb96562a64fa9aff9376fe/lava_callback.json.gz 2026-03-31 09:50:13,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:13,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb96562a64fa9aff9376fe HTTP/1.1" 200 None 2026-03-31 09:50:13,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:13,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:50:13,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:13,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb96562a64fa9aff9376fe HTTP/1.1" 200 None 2026-03-31 09:50:13,401 - lava_callback - INFO - Completed processing callback for node 69cb96562a64fa9aff9376fe INFO: 172.19.0.1:59758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57778 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:50:37,253 - lava_callback - INFO - Received callback for node 69cb98b02a64fa9aff93775b INFO: 172.19.0.1:57788 - "POST /node/69cb98b02a64fa9aff93775b HTTP/1.0" 202 Accepted 2026-03-31 09:50:37,260 - lava_callback - INFO - Starting async job submit for node 69cb98b02a64fa9aff93775b 2026-03-31 09:50:37,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:37,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb98b02a64fa9aff93775b HTTP/1.1" 200 None 2026-03-31 09:50:37,305 - lava_callback - DEBUG - Processing node 69cb98b02a64fa9aff93775b: baseline-arm64 2026-03-31 09:50:37,452 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:50:37,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:50:37,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:50:37,702 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb98b02a64fa9aff93775b/lava-logs.yaml 2026-03-31 09:50:37,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:50:37,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:50:37,839 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb98b02a64fa9aff93775b/lava_callback.json.gz 2026-03-31 09:50:37,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:37,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb98b02a64fa9aff93775b HTTP/1.1" 200 None 2026-03-31 09:50:37,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:37,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:50:37,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:37,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb98b02a64fa9aff93775b HTTP/1.1" 200 None 2026-03-31 09:50:37,992 - lava_callback - INFO - Completed processing callback for node 69cb98b02a64fa9aff93775b 2026-03-31 09:50:38,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:38,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:37562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43976 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:51:42,888 - lava_callback - INFO - Received callback for node 69cb94b32a64fa9aff9376a2 INFO: 172.19.0.1:33346 - "POST /node/69cb94b32a64fa9aff9376a2 HTTP/1.0" 202 Accepted 2026-03-31 09:51:42,895 - lava_callback - INFO - Starting async job submit for node 69cb94b32a64fa9aff9376a2 2026-03-31 09:51:42,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:51:42,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94b32a64fa9aff9376a2 HTTP/1.1" 200 None 2026-03-31 09:51:42,968 - lava_callback - DEBUG - Processing node 69cb94b32a64fa9aff9376a2: ltp-smoke 2026-03-31 09:51:43,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:51:43,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:51:43,604 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb94b32a64fa9aff9376a2/log.txt.gz 2026-03-31 09:51:43,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:51:44,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:51:44,147 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb94b32a64fa9aff9376a2/lava-logs.yaml 2026-03-31 09:51:44,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:51:44,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:51:44,334 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb94b32a64fa9aff9376a2/lava_callback.json.gz 2026-03-31 09:51:44,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:51:44,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94b32a64fa9aff9376a2 HTTP/1.1" 200 None 2026-03-31 09:51:44,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:51:44,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83462a64fa9aff936905 HTTP/1.1" 200 None 2026-03-31 09:51:44,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:51:44,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb94b32a64fa9aff9376a2 HTTP/1.1" 200 None 2026-03-31 09:51:44,696 - lava_callback - INFO - Completed processing callback for node 69cb94b32a64fa9aff9376a2 INFO: 172.19.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53912 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:52:08,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:52:08,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:38016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55434 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:52:29,566 - lava_callback - INFO - Received callback for node 69c8f6526a4c026e16348be7 INFO: 172.19.0.1:55446 - "POST /node/69c8f6526a4c026e16348be7 HTTP/1.0" 202 Accepted 2026-03-31 09:52:29,576 - lava_callback - INFO - Starting async job submit for node 69c8f6526a4c026e16348be7 2026-03-31 09:52:29,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:52:29,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f6526a4c026e16348be7 HTTP/1.1" 200 None 2026-03-31 09:52:29,603 - lava_callback - DEBUG - Processing node 69c8f6526a4c026e16348be7: baseline-arm64-clabbe 2026-03-31 09:52:29,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f6526a4c026e16348be7 2026-03-31 09:52:29,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:52:29,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f6526a4c026e16348be7 HTTP/1.1" 200 None 2026-03-31 09:52:29,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:52:29,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:52:29,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:52:29,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f6526a4c026e16348be7 HTTP/1.1" 200 None 2026-03-31 09:52:29,695 - lava_callback - ERROR - Error processing callback for node 69c8f6526a4c026e16348be7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47610 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:53:49,555 - lava_callback - INFO - Received callback for node 69c8f69f6a4c026e16348bf0 INFO: 172.19.0.1:47622 - "POST /node/69c8f69f6a4c026e16348bf0 HTTP/1.0" 202 Accepted 2026-03-31 09:53:49,561 - lava_callback - INFO - Starting async job submit for node 69c8f69f6a4c026e16348bf0 2026-03-31 09:53:49,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:53:49,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f69f6a4c026e16348bf0 HTTP/1.1" 200 None 2026-03-31 09:53:49,584 - lava_callback - DEBUG - Processing node 69c8f69f6a4c026e16348bf0: baseline-arm64-clabbe 2026-03-31 09:53:49,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f69f6a4c026e16348bf0 2026-03-31 09:53:49,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:53:49,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f69f6a4c026e16348bf0 HTTP/1.1" 200 None 2026-03-31 09:53:49,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:53:49,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:53:49,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:53:49,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f69f6a4c026e16348bf0 HTTP/1.1" 200 None 2026-03-31 09:53:49,647 - lava_callback - ERROR - Error processing callback for node 69c8f69f6a4c026e16348bf0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40750 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:55:09,653 - lava_callback - INFO - Received callback for node 69c8f6ec6a4c026e16348bf6 INFO: 172.19.0.1:40756 - "POST /node/69c8f6ec6a4c026e16348bf6 HTTP/1.0" 202 Accepted 2026-03-31 09:55:09,658 - lava_callback - INFO - Starting async job submit for node 69c8f6ec6a4c026e16348bf6 2026-03-31 09:55:09,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:55:09,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f6ec6a4c026e16348bf6 HTTP/1.1" 200 None 2026-03-31 09:55:09,675 - lava_callback - DEBUG - Processing node 69c8f6ec6a4c026e16348bf6: baseline-arm64-clabbe 2026-03-31 09:55:09,675 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f6ec6a4c026e16348bf6 2026-03-31 09:55:09,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:55:09,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f6ec6a4c026e16348bf6 HTTP/1.1" 200 None 2026-03-31 09:55:09,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:55:09,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:55:09,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:55:09,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f6ec6a4c026e16348bf6 HTTP/1.1" 200 None 2026-03-31 09:55:09,738 - lava_callback - ERROR - Error processing callback for node 69c8f6ec6a4c026e16348bf6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43046 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:56:29,710 - lava_callback - INFO - Received callback for node 69c8f73a6a4c026e16348c02 INFO: 172.19.0.1:58058 - "POST /node/69c8f73a6a4c026e16348c02 HTTP/1.0" 202 Accepted 2026-03-31 09:56:29,715 - lava_callback - INFO - Starting async job submit for node 69c8f73a6a4c026e16348c02 2026-03-31 09:56:29,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:56:29,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f73a6a4c026e16348c02 HTTP/1.1" 200 None 2026-03-31 09:56:29,731 - lava_callback - DEBUG - Processing node 69c8f73a6a4c026e16348c02: baseline-arm64-clabbe 2026-03-31 09:56:29,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f73a6a4c026e16348c02 2026-03-31 09:56:29,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:56:29,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f73a6a4c026e16348c02 HTTP/1.1" 200 None 2026-03-31 09:56:29,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:56:29,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:56:29,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:56:29,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f73a6a4c026e16348c02 HTTP/1.1" 200 None 2026-03-31 09:56:29,790 - lava_callback - ERROR - Error processing callback for node 69c8f73a6a4c026e16348c02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57096 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:57:18,626 - lava_callback - INFO - Received callback for node 69cb95c32a64fa9aff9376ec INFO: 172.19.0.1:57108 - "POST /node/69cb95c32a64fa9aff9376ec HTTP/1.0" 202 Accepted 2026-03-31 09:57:18,635 - lava_callback - INFO - Starting async job submit for node 69cb95c32a64fa9aff9376ec 2026-03-31 09:57:18,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:57:18,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb95c32a64fa9aff9376ec HTTP/1.1" 200 None 2026-03-31 09:57:18,698 - lava_callback - DEBUG - Processing node 69cb95c32a64fa9aff9376ec: baseline-nfs-arm64 2026-03-31 09:57:19,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:57:19,914 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:57:19,916 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb95c32a64fa9aff9376ec/log.txt.gz 2026-03-31 09:57:20,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:57:20,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:57:20,808 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb95c32a64fa9aff9376ec/lava-logs.yaml 2026-03-31 09:57:20,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:57:21,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:57:21,037 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb95c32a64fa9aff9376ec/lava_callback.json.gz DEBUG: Setup failed for 69cb95c32a64fa9aff9376ec. Transitting job node result: incomplete -> fail 2026-03-31 09:57:21,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:57:21,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb95c32a64fa9aff9376ec HTTP/1.1" 200 None 2026-03-31 09:57:21,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:57:21,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:57:21,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:57:21,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb95c32a64fa9aff9376ec HTTP/1.1" 200 None 2026-03-31 09:57:21,332 - lava_callback - INFO - Completed processing callback for node 69cb95c32a64fa9aff9376ec INFO: 172.19.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56846 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:57:38,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:57:38,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52856 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:58:49,827 - lava_callback - INFO - Received callback for node 69c8f7d56a4c026e16348c1a INFO: 172.19.0.1:34128 - "POST /node/69c8f7d56a4c026e16348c1a HTTP/1.0" 202 Accepted 2026-03-31 09:58:49,832 - lava_callback - INFO - Starting async job submit for node 69c8f7d56a4c026e16348c1a 2026-03-31 09:58:49,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:58:49,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f7d56a4c026e16348c1a HTTP/1.1" 200 None 2026-03-31 09:58:49,849 - lava_callback - DEBUG - Processing node 69c8f7d56a4c026e16348c1a: baseline-arm64-clabbe 2026-03-31 09:58:49,849 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f7d56a4c026e16348c1a 2026-03-31 09:58:49,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:58:49,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f7d56a4c026e16348c1a HTTP/1.1" 200 None 2026-03-31 09:58:49,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:58:49,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:58:49,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:58:49,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f7d56a4c026e16348c1a HTTP/1.1" 200 None 2026-03-31 09:58:49,901 - lava_callback - ERROR - Error processing callback for node 69c8f7d56a4c026e16348c1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56582 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:59:43,495 - lava_callback - INFO - Received callback for node 69cb97d72a64fa9aff93773c INFO: 172.19.0.1:43570 - "POST /node/69cb97d72a64fa9aff93773c HTTP/1.0" 202 Accepted 2026-03-31 09:59:43,501 - lava_callback - INFO - Starting async job submit for node 69cb97d72a64fa9aff93773c 2026-03-31 09:59:43,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:59:43,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb97d72a64fa9aff93773c HTTP/1.1" 200 None 2026-03-31 09:59:43,546 - lava_callback - DEBUG - Processing node 69cb97d72a64fa9aff93773c: baseline-nfs-arm64 2026-03-31 09:59:43,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:59:44,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:59:44,170 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb97d72a64fa9aff93773c/log.txt.gz 2026-03-31 09:59:44,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:59:44,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:59:44,717 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb97d72a64fa9aff93773c/lava-logs.yaml 2026-03-31 09:59:44,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:59:44,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:59:44,863 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb97d72a64fa9aff93773c/lava_callback.json.gz 2026-03-31 09:59:44,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:59:44,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb97d72a64fa9aff93773c HTTP/1.1" 200 None 2026-03-31 09:59:44,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:59:44,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:59:44,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:59:44,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb97d72a64fa9aff93773c HTTP/1.1" 200 None 2026-03-31 09:59:44,998 - lava_callback - INFO - Completed processing callback for node 69cb97d72a64fa9aff93773c INFO: 172.19.0.1:43574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40386 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:00:08,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:00:08,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:45896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48948 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:01:29,928 - lava_callback - INFO - Received callback for node 69c8f86e6a4c026e16348c2b INFO: 172.19.0.1:37278 - "POST /node/69c8f86e6a4c026e16348c2b HTTP/1.0" 202 Accepted 2026-03-31 10:01:29,934 - lava_callback - INFO - Starting async job submit for node 69c8f86e6a4c026e16348c2b 2026-03-31 10:01:29,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:01:29,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f86e6a4c026e16348c2b HTTP/1.1" 200 None 2026-03-31 10:01:29,952 - lava_callback - DEBUG - Processing node 69c8f86e6a4c026e16348c2b: baseline-arm64-clabbe 2026-03-31 10:01:29,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f86e6a4c026e16348c2b 2026-03-31 10:01:29,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:01:29,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f86e6a4c026e16348c2b HTTP/1.1" 200 None 2026-03-31 10:01:29,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:01:29,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 10:01:29,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:01:30,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f86e6a4c026e16348c2b HTTP/1.1" 200 None 2026-03-31 10:01:30,003 - lava_callback - ERROR - Error processing callback for node 69c8f86e6a4c026e16348c2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33402 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:07:12,376 - lava_callback - INFO - Received callback for node 69cb9cc62a64fa9aff9377fd INFO: 172.19.0.1:46768 - "POST /node/69cb9cc62a64fa9aff9377fd HTTP/1.0" 202 Accepted 2026-03-31 10:07:12,381 - lava_callback - INFO - Starting async job submit for node 69cb9cc62a64fa9aff9377fd 2026-03-31 10:07:12,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:12,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9cc62a64fa9aff9377fd HTTP/1.1" 200 None 2026-03-31 10:07:12,400 - lava_callback - DEBUG - Processing node 69cb9cc62a64fa9aff9377fd: baseline-nfs-arm64 2026-03-31 10:07:12,401 - lava_callback - WARNING - Failed to upload log 2026-03-31 10:07:12,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:07:12,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:07:12,555 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb9cc62a64fa9aff9377fd/lava-logs.yaml 2026-03-31 10:07:12,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:07:12,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:07:12,702 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb9cc62a64fa9aff9377fd/lava_callback.json.gz 2026-03-31 10:07:12,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:12,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9cc62a64fa9aff9377fd HTTP/1.1" 200 None 2026-03-31 10:07:12,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:12,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:07:12,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:12,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb9cc62a64fa9aff9377fd HTTP/1.1" 200 None 2026-03-31 10:07:12,772 - lava_callback - INFO - Completed processing callback for node 69cb9cc62a64fa9aff9377fd INFO: 172.19.0.1:46770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37686 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:07:31,092 - lava_callback - INFO - Received callback for node 69c8f9de6a4c026e16348c5d INFO: 172.19.0.1:51652 - "POST /node/69c8f9de6a4c026e16348c5d HTTP/1.0" 202 Accepted 2026-03-31 10:07:31,098 - lava_callback - INFO - Starting async job submit for node 69c8f9de6a4c026e16348c5d 2026-03-31 10:07:31,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:31,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f9de6a4c026e16348c5d HTTP/1.1" 200 None 2026-03-31 10:07:31,118 - lava_callback - DEBUG - Processing node 69c8f9de6a4c026e16348c5d: baseline-arm-baylibre 2026-03-31 10:07:31,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f9de6a4c026e16348c5d 2026-03-31 10:07:31,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:31,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f9de6a4c026e16348c5d HTTP/1.1" 200 None 2026-03-31 10:07:31,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:31,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:07:31,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:31,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f9de6a4c026e16348c5d HTTP/1.1" 200 None 2026-03-31 10:07:31,178 - lava_callback - ERROR - Error processing callback for node 69c8f9de6a4c026e16348c5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51662 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:07:38,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:38,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:46470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35610 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:08:10,980 - lava_callback - INFO - Received callback for node 69c8fa036a4c026e16348c69 INFO: 172.19.0.1:60280 - "POST /node/69c8fa036a4c026e16348c69 HTTP/1.0" 202 Accepted 2026-03-31 10:08:10,986 - lava_callback - INFO - Starting async job submit for node 69c8fa036a4c026e16348c69 2026-03-31 10:08:10,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:08:11,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa036a4c026e16348c69 HTTP/1.1" 200 None 2026-03-31 10:08:11,007 - lava_callback - DEBUG - Processing node 69c8fa036a4c026e16348c69: baseline-arm-baylibre 2026-03-31 10:08:11,007 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fa036a4c026e16348c69 2026-03-31 10:08:11,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:08:11,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa036a4c026e16348c69 HTTP/1.1" 200 None 2026-03-31 10:08:11,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:08:11,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:08:11,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:08:11,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fa036a4c026e16348c69 HTTP/1.1" 200 None 2026-03-31 10:08:11,060 - lava_callback - ERROR - Error processing callback for node 69c8fa036a4c026e16348c69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54848 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:10:11,155 - lava_callback - INFO - Received callback for node 69c8fa756a4c026e16348c78 INFO: 172.19.0.1:48692 - "POST /node/69c8fa756a4c026e16348c78 HTTP/1.0" 202 Accepted 2026-03-31 10:10:11,163 - lava_callback - INFO - Starting async job submit for node 69c8fa756a4c026e16348c78 2026-03-31 10:10:11,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:11,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa756a4c026e16348c78 HTTP/1.1" 200 None 2026-03-31 10:10:11,199 - lava_callback - DEBUG - Processing node 69c8fa756a4c026e16348c78: baseline-arm-baylibre 2026-03-31 10:10:11,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fa756a4c026e16348c78 2026-03-31 10:10:11,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:11,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa756a4c026e16348c78 HTTP/1.1" 200 None 2026-03-31 10:10:11,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:11,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:10:11,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:11,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fa756a4c026e16348c78 HTTP/1.1" 200 None 2026-03-31 10:10:11,274 - lava_callback - ERROR - Error processing callback for node 69c8fa756a4c026e16348c78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50416 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:10:51,169 - lava_callback - INFO - Received callback for node 69c8fa996a4c026e16348c85 INFO: 172.19.0.1:52778 - "POST /node/69c8fa996a4c026e16348c85 HTTP/1.0" 202 Accepted 2026-03-31 10:10:51,174 - lava_callback - INFO - Starting async job submit for node 69c8fa996a4c026e16348c85 2026-03-31 10:10:51,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:51,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa996a4c026e16348c85 HTTP/1.1" 200 None 2026-03-31 10:10:51,190 - lava_callback - DEBUG - Processing node 69c8fa996a4c026e16348c85: baseline-arm-baylibre 2026-03-31 10:10:51,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fa996a4c026e16348c85 2026-03-31 10:10:51,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:51,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa996a4c026e16348c85 HTTP/1.1" 200 None 2026-03-31 10:10:51,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:51,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:10:51,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:51,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fa996a4c026e16348c85 HTTP/1.1" 200 None 2026-03-31 10:10:51,242 - lava_callback - ERROR - Error processing callback for node 69c8fa996a4c026e16348c85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53580 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:11:31,213 - lava_callback - INFO - Received callback for node 69c8fac46a4c026e16348c8c INFO: 172.19.0.1:45080 - "POST /node/69c8fac46a4c026e16348c8c HTTP/1.0" 202 Accepted 2026-03-31 10:11:31,219 - lava_callback - INFO - Starting async job submit for node 69c8fac46a4c026e16348c8c 2026-03-31 10:11:31,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:11:31,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fac46a4c026e16348c8c HTTP/1.1" 200 None 2026-03-31 10:11:31,235 - lava_callback - DEBUG - Processing node 69c8fac46a4c026e16348c8c: baseline-arm-baylibre 2026-03-31 10:11:31,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fac46a4c026e16348c8c 2026-03-31 10:11:31,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:11:31,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fac46a4c026e16348c8c HTTP/1.1" 200 None 2026-03-31 10:11:31,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:11:31,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:11:31,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:11:31,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fac46a4c026e16348c8c HTTP/1.1" 200 None 2026-03-31 10:11:31,300 - lava_callback - ERROR - Error processing callback for node 69c8fac46a4c026e16348c8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34392 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:12:11,220 - lava_callback - INFO - Received callback for node 69c8fae86a4c026e16348c95 INFO: 172.19.0.1:37888 - "POST /node/69c8fae86a4c026e16348c95 HTTP/1.0" 202 Accepted 2026-03-31 10:12:11,229 - lava_callback - INFO - Starting async job submit for node 69c8fae86a4c026e16348c95 2026-03-31 10:12:11,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:12:11,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fae86a4c026e16348c95 HTTP/1.1" 200 None 2026-03-31 10:12:11,244 - lava_callback - DEBUG - Processing node 69c8fae86a4c026e16348c95: baseline-arm-baylibre 2026-03-31 10:12:11,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fae86a4c026e16348c95 2026-03-31 10:12:11,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:12:11,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fae86a4c026e16348c95 HTTP/1.1" 200 None 2026-03-31 10:12:11,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:12:11,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:12:11,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:12:11,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fae86a4c026e16348c95 HTTP/1.1" 200 None 2026-03-31 10:12:11,308 - lava_callback - ERROR - Error processing callback for node 69c8fae86a4c026e16348c95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54996 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:13:30,519 - lava_callback - INFO - Received callback for node 69c8fb346a4c026e16348caa INFO: 172.19.0.1:54920 - "POST /node/69c8fb346a4c026e16348caa HTTP/1.0" 202 Accepted 2026-03-31 10:13:30,526 - lava_callback - INFO - Starting async job submit for node 69c8fb346a4c026e16348caa 2026-03-31 10:13:30,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:30,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb346a4c026e16348caa HTTP/1.1" 200 None 2026-03-31 10:13:30,549 - lava_callback - DEBUG - Processing node 69c8fb346a4c026e16348caa: baseline-arm-clabbe 2026-03-31 10:13:30,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fb346a4c026e16348caa 2026-03-31 10:13:30,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:30,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb346a4c026e16348caa HTTP/1.1" 200 None 2026-03-31 10:13:30,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:30,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:13:30,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:30,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fb346a4c026e16348caa HTTP/1.1" 200 None 2026-03-31 10:13:30,616 - lava_callback - ERROR - Error processing callback for node 69c8fb346a4c026e16348caa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39244 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:13:50,434 - lava_callback - INFO - Received callback for node 69c8fb5a6a4c026e16348cb1 INFO: 172.19.0.1:57644 - "POST /node/69c8fb5a6a4c026e16348cb1 HTTP/1.0" 202 Accepted 2026-03-31 10:13:50,440 - lava_callback - INFO - Starting async job submit for node 69c8fb5a6a4c026e16348cb1 2026-03-31 10:13:50,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:50,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb5a6a4c026e16348cb1 HTTP/1.1" 200 None 2026-03-31 10:13:50,462 - lava_callback - DEBUG - Processing node 69c8fb5a6a4c026e16348cb1: baseline-arm-clabbe 2026-03-31 10:13:50,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fb5a6a4c026e16348cb1 2026-03-31 10:13:50,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:50,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb5a6a4c026e16348cb1 HTTP/1.1" 200 None 2026-03-31 10:13:50,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:50,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:13:50,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:50,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fb5a6a4c026e16348cb1 HTTP/1.1" 200 None 2026-03-31 10:13:50,545 - lava_callback - ERROR - Error processing callback for node 69c8fb5a6a4c026e16348cb1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55538 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:14:30,504 - lava_callback - INFO - Received callback for node 69c8fb7f6a4c026e16348cc0 INFO: 172.19.0.1:53426 - "POST /node/69c8fb7f6a4c026e16348cc0 HTTP/1.0" 202 Accepted 2026-03-31 10:14:30,510 - lava_callback - INFO - Starting async job submit for node 69c8fb7f6a4c026e16348cc0 2026-03-31 10:14:30,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:14:30,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb7f6a4c026e16348cc0 HTTP/1.1" 200 None 2026-03-31 10:14:30,532 - lava_callback - DEBUG - Processing node 69c8fb7f6a4c026e16348cc0: baseline-arm-clabbe 2026-03-31 10:14:30,532 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fb7f6a4c026e16348cc0 2026-03-31 10:14:30,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:14:30,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb7f6a4c026e16348cc0 HTTP/1.1" 200 None 2026-03-31 10:14:30,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:14:30,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:14:30,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:14:30,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fb7f6a4c026e16348cc0 HTTP/1.1" 200 None 2026-03-31 10:14:30,586 - lava_callback - ERROR - Error processing callback for node 69c8fb7f6a4c026e16348cc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60650 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:15:10,482 - lava_callback - INFO - Received callback for node 69c8fba56a4c026e16348ccc INFO: 172.19.0.1:40316 - "POST /node/69c8fba56a4c026e16348ccc HTTP/1.0" 202 Accepted 2026-03-31 10:15:10,488 - lava_callback - INFO - Starting async job submit for node 69c8fba56a4c026e16348ccc 2026-03-31 10:15:10,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:10,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fba56a4c026e16348ccc HTTP/1.1" 200 None 2026-03-31 10:15:10,516 - lava_callback - DEBUG - Processing node 69c8fba56a4c026e16348ccc: baseline-arm-clabbe 2026-03-31 10:15:10,516 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fba56a4c026e16348ccc 2026-03-31 10:15:10,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:10,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fba56a4c026e16348ccc HTTP/1.1" 200 None 2026-03-31 10:15:10,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:10,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:15:10,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:10,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fba56a4c026e16348ccc HTTP/1.1" 200 None 2026-03-31 10:15:10,566 - lava_callback - ERROR - Error processing callback for node 69c8fba56a4c026e16348ccc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57064 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:15:42,090 - lava_callback - INFO - Received callback for node 69cb9ac72a64fa9aff9377b6 INFO: 172.19.0.1:47218 - "POST /node/69cb9ac72a64fa9aff9377b6 HTTP/1.0" 202 Accepted 2026-03-31 10:15:42,097 - lava_callback - INFO - Starting async job submit for node 69cb9ac72a64fa9aff9377b6 2026-03-31 10:15:42,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:42,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9ac72a64fa9aff9377b6 HTTP/1.1" 200 None 2026-03-31 10:15:42,197 - lava_callback - DEBUG - Processing node 69cb9ac72a64fa9aff9377b6: baseline-arm 2026-03-31 10:15:42,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:15:43,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:15:43,050 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb9ac72a64fa9aff9377b6/log.txt.gz 2026-03-31 10:15:43,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:15:43,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:15:43,621 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb9ac72a64fa9aff9377b6/lava-logs.yaml 2026-03-31 10:15:43,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:15:43,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:15:43,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb9ac72a64fa9aff9377b6/lava_callback.json.gz DEBUG: Setup failed for 69cb9ac72a64fa9aff9377b6. Transitting job node result: incomplete -> fail 2026-03-31 10:15:43,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:43,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9ac72a64fa9aff9377b6 HTTP/1.1" 200 None 2026-03-31 10:15:43,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:43,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 10:15:43,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:43,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb9ac72a64fa9aff9377b6 HTTP/1.1" 200 None 2026-03-31 10:15:44,067 - lava_callback - INFO - Completed processing callback for node 69cb9ac72a64fa9aff9377b6 INFO: 172.19.0.1:47230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47242 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:15:50,533 - lava_callback - INFO - Received callback for node 69c8fbc96a4c026e16348cd7 INFO: 172.19.0.1:36514 - "POST /node/69c8fbc96a4c026e16348cd7 HTTP/1.0" 202 Accepted 2026-03-31 10:15:50,539 - lava_callback - INFO - Starting async job submit for node 69c8fbc96a4c026e16348cd7 2026-03-31 10:15:50,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:50,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fbc96a4c026e16348cd7 HTTP/1.1" 200 None 2026-03-31 10:15:50,560 - lava_callback - DEBUG - Processing node 69c8fbc96a4c026e16348cd7: baseline-arm-clabbe 2026-03-31 10:15:50,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fbc96a4c026e16348cd7 2026-03-31 10:15:50,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:50,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fbc96a4c026e16348cd7 HTTP/1.1" 200 None 2026-03-31 10:15:50,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:50,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:15:50,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:50,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fbc96a4c026e16348cd7 HTTP/1.1" 200 None 2026-03-31 10:15:50,612 - lava_callback - ERROR - Error processing callback for node 69c8fbc96a4c026e16348cd7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45764 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:16:08,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:08,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 10:16:13,128 - lava_callback - INFO - Received callback for node 69cb9dab2a64fa9aff93781f INFO: 172.19.0.1:34294 - "POST /node/69cb9dab2a64fa9aff93781f HTTP/1.0" 202 Accepted 2026-03-31 10:16:13,136 - lava_callback - INFO - Starting async job submit for node 69cb9dab2a64fa9aff93781f 2026-03-31 10:16:13,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:13,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9dab2a64fa9aff93781f HTTP/1.1" 200 None 2026-03-31 10:16:13,224 - lava_callback - DEBUG - Processing node 69cb9dab2a64fa9aff93781f: baseline-arm64 2026-03-31 10:16:13,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:16:13,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:16:13,834 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9dab2a64fa9aff93781f/log.txt.gz 2026-03-31 10:16:14,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:16:14,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:16:14,242 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9dab2a64fa9aff93781f/lava-logs.yaml 2026-03-31 10:16:14,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:16:14,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:16:14,403 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9dab2a64fa9aff93781f/lava_callback.json.gz DEBUG: Setup failed for 69cb9dab2a64fa9aff93781f. Transitting job node result: incomplete -> fail 2026-03-31 10:16:14,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:14,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9dab2a64fa9aff93781f HTTP/1.1" 200 None 2026-03-31 10:16:14,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:14,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:16:14,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:14,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb9dab2a64fa9aff93781f HTTP/1.1" 200 None 2026-03-31 10:16:14,605 - lava_callback - INFO - Completed processing callback for node 69cb9dab2a64fa9aff93781f INFO: 172.19.0.1:34300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39842 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:16:38,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:38,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36316 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:18:23,423 - lava_callback - INFO - Received callback for node 69cb9f382a64fa9aff937860 INFO: 172.19.0.1:34100 - "POST /node/69cb9f382a64fa9aff937860 HTTP/1.0" 202 Accepted 2026-03-31 10:18:23,430 - lava_callback - INFO - Starting async job submit for node 69cb9f382a64fa9aff937860 2026-03-31 10:18:23,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:23,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9f382a64fa9aff937860 HTTP/1.1" 200 None 2026-03-31 10:18:23,478 - lava_callback - DEBUG - Processing node 69cb9f382a64fa9aff937860: baseline-arm64 2026-03-31 10:18:23,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:18:23,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:18:23,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9f382a64fa9aff937860/log.txt.gz 2026-03-31 10:18:23,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:18:24,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:18:24,109 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9f382a64fa9aff937860/lava-logs.yaml 2026-03-31 10:18:24,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:18:24,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:18:24,260 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9f382a64fa9aff937860/lava_callback.json.gz 2026-03-31 10:18:24,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:24,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9f382a64fa9aff937860 HTTP/1.1" 200 None 2026-03-31 10:18:24,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:24,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:18:24,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:24,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb9f382a64fa9aff937860 HTTP/1.1" 200 None 2026-03-31 10:18:24,427 - lava_callback - INFO - Completed processing callback for node 69cb9f382a64fa9aff937860 INFO: 172.19.0.1:34108 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:18:31,691 - lava_callback - INFO - Received callback for node 69c8fc666a4c026e16348cf8 INFO: 172.19.0.1:46846 - "POST /node/69c8fc666a4c026e16348cf8 HTTP/1.0" 202 Accepted 2026-03-31 10:18:31,700 - lava_callback - INFO - Starting async job submit for node 69c8fc666a4c026e16348cf8 2026-03-31 10:18:31,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:31,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fc666a4c026e16348cf8 HTTP/1.1" 200 None 2026-03-31 10:18:31,717 - lava_callback - DEBUG - Processing node 69c8fc666a4c026e16348cf8: baseline-arm64-baylibre 2026-03-31 10:18:31,717 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fc666a4c026e16348cf8 2026-03-31 10:18:31,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:31,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fc666a4c026e16348cf8 HTTP/1.1" 200 None 2026-03-31 10:18:31,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:31,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:18:31,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:31,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fc666a4c026e16348cf8 HTTP/1.1" 200 None 2026-03-31 10:18:31,781 - lava_callback - ERROR - Error processing callback for node 69c8fc666a4c026e16348cf8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46852 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:18:38,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:38,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:41978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39072 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:19:51,876 - lava_callback - INFO - Received callback for node 69c8fcb26a4c026e16348cfe INFO: 172.19.0.1:59510 - "POST /node/69c8fcb26a4c026e16348cfe HTTP/1.0" 202 Accepted 2026-03-31 10:19:51,884 - lava_callback - INFO - Starting async job submit for node 69c8fcb26a4c026e16348cfe 2026-03-31 10:19:51,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:19:51,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fcb26a4c026e16348cfe HTTP/1.1" 200 None 2026-03-31 10:19:51,900 - lava_callback - DEBUG - Processing node 69c8fcb26a4c026e16348cfe: baseline-arm64-baylibre 2026-03-31 10:19:51,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fcb26a4c026e16348cfe 2026-03-31 10:19:51,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:19:51,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fcb26a4c026e16348cfe HTTP/1.1" 200 None 2026-03-31 10:19:51,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:19:51,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:19:51,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:19:51,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fcb26a4c026e16348cfe HTTP/1.1" 200 None 2026-03-31 10:19:51,962 - lava_callback - ERROR - Error processing callback for node 69c8fcb26a4c026e16348cfe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50052 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:20:51,656 - lava_callback - INFO - Received callback for node 69c8fd006a4c026e16348d08 INFO: 172.19.0.1:41302 - "POST /node/69c8fd006a4c026e16348d08 HTTP/1.0" 202 Accepted 2026-03-31 10:20:51,661 - lava_callback - INFO - Starting async job submit for node 69c8fd006a4c026e16348d08 2026-03-31 10:20:51,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:20:51,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fd006a4c026e16348d08 HTTP/1.1" 200 None 2026-03-31 10:20:51,681 - lava_callback - DEBUG - Processing node 69c8fd006a4c026e16348d08: baseline-arm64-baylibre 2026-03-31 10:20:51,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fd006a4c026e16348d08 2026-03-31 10:20:51,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:20:51,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fd006a4c026e16348d08 HTTP/1.1" 200 None 2026-03-31 10:20:51,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:20:51,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:20:51,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:20:51,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fd006a4c026e16348d08 HTTP/1.1" 200 None 2026-03-31 10:20:51,745 - lava_callback - ERROR - Error processing callback for node 69c8fd006a4c026e16348d08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50702 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:22:11,762 - lava_callback - INFO - Received callback for node 69c8fd4c6a4c026e16348d12 INFO: 172.19.0.1:34770 - "POST /node/69c8fd4c6a4c026e16348d12 HTTP/1.0" 202 Accepted 2026-03-31 10:22:11,767 - lava_callback - INFO - Starting async job submit for node 69c8fd4c6a4c026e16348d12 2026-03-31 10:22:11,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:11,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fd4c6a4c026e16348d12 HTTP/1.1" 200 None 2026-03-31 10:22:11,804 - lava_callback - DEBUG - Processing node 69c8fd4c6a4c026e16348d12: baseline-arm64-baylibre 2026-03-31 10:22:11,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fd4c6a4c026e16348d12 2026-03-31 10:22:11,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:11,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fd4c6a4c026e16348d12 HTTP/1.1" 200 None 2026-03-31 10:22:11,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:11,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:22:11,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:11,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fd4c6a4c026e16348d12 HTTP/1.1" 200 None 2026-03-31 10:22:11,861 - lava_callback - ERROR - Error processing callback for node 69c8fd4c6a4c026e16348d12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36000 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:22:58,125 - lava_callback - INFO - Received callback for node 69cba04b2a64fa9aff937886 INFO: 172.19.0.1:36008 - "POST /node/69cba04b2a64fa9aff937886 HTTP/1.0" 202 Accepted 2026-03-31 10:22:58,132 - lava_callback - INFO - Starting async job submit for node 69cba04b2a64fa9aff937886 2026-03-31 10:22:58,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:58,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba04b2a64fa9aff937886 HTTP/1.1" 200 None 2026-03-31 10:22:58,176 - lava_callback - DEBUG - Processing node 69cba04b2a64fa9aff937886: baseline-arm64 2026-03-31 10:22:58,310 - lava_callback - WARNING - Failed to upload log 2026-03-31 10:22:58,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:22:58,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:22:58,611 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba04b2a64fa9aff937886/lava-logs.yaml 2026-03-31 10:22:58,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:22:58,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:22:58,738 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba04b2a64fa9aff937886/lava_callback.json.gz 2026-03-31 10:22:58,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:58,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba04b2a64fa9aff937886 HTTP/1.1" 200 None 2026-03-31 10:22:58,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:58,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:22:58,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:58,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba04b2a64fa9aff937886 HTTP/1.1" 200 None 2026-03-31 10:22:58,896 - lava_callback - INFO - Completed processing callback for node 69cba04b2a64fa9aff937886 INFO: 172.19.0.1:36714 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:23:08,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:23:08,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:37992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36306 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:24:52,006 - lava_callback - INFO - Received callback for node 69c8fde46a4c026e16348d25 INFO: 172.19.0.1:54982 - "POST /node/69c8fde46a4c026e16348d25 HTTP/1.0" 202 Accepted 2026-03-31 10:24:52,011 - lava_callback - INFO - Starting async job submit for node 69c8fde46a4c026e16348d25 2026-03-31 10:24:52,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:24:52,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fde46a4c026e16348d25 HTTP/1.1" 200 None 2026-03-31 10:24:52,028 - lava_callback - DEBUG - Processing node 69c8fde46a4c026e16348d25: baseline-arm64-baylibre 2026-03-31 10:24:52,028 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fde46a4c026e16348d25 2026-03-31 10:24:52,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:24:52,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fde46a4c026e16348d25 HTTP/1.1" 200 None 2026-03-31 10:24:52,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:24:52,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:24:52,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:24:52,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fde46a4c026e16348d25 HTTP/1.1" 200 None 2026-03-31 10:24:52,080 - lava_callback - ERROR - Error processing callback for node 69c8fde46a4c026e16348d25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49432 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:27:31,040 - lava_callback - INFO - Received callback for node 69c8fe806a4c026e16348d3c INFO: 172.19.0.1:34052 - "POST /node/69c8fe806a4c026e16348d3c HTTP/1.0" 202 Accepted 2026-03-31 10:27:31,046 - lava_callback - INFO - Starting async job submit for node 69c8fe806a4c026e16348d3c 2026-03-31 10:27:31,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:27:31,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fe806a4c026e16348d3c HTTP/1.1" 200 None 2026-03-31 10:27:31,062 - lava_callback - DEBUG - Processing node 69c8fe806a4c026e16348d3c: baseline-arm64-clabbe 2026-03-31 10:27:31,062 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fe806a4c026e16348d3c 2026-03-31 10:27:31,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:27:31,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fe806a4c026e16348d3c HTTP/1.1" 200 None 2026-03-31 10:27:31,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:27:31,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:27:31,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:27:31,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fe806a4c026e16348d3c HTTP/1.1" 200 None 2026-03-31 10:27:31,121 - lava_callback - ERROR - Error processing callback for node 69c8fe806a4c026e16348d3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56590 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:28:31,114 - lava_callback - INFO - Received callback for node 69c8fecb6a4c026e16348d44 INFO: 172.19.0.1:46098 - "POST /node/69c8fecb6a4c026e16348d44 HTTP/1.0" 202 Accepted 2026-03-31 10:28:31,119 - lava_callback - INFO - Starting async job submit for node 69c8fecb6a4c026e16348d44 2026-03-31 10:28:31,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:28:31,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fecb6a4c026e16348d44 HTTP/1.1" 200 None 2026-03-31 10:28:31,140 - lava_callback - DEBUG - Processing node 69c8fecb6a4c026e16348d44: baseline-arm64-clabbe 2026-03-31 10:28:31,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fecb6a4c026e16348d44 2026-03-31 10:28:31,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:28:31,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fecb6a4c026e16348d44 HTTP/1.1" 200 None 2026-03-31 10:28:31,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:28:31,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:28:31,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:28:31,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fecb6a4c026e16348d44 HTTP/1.1" 200 None 2026-03-31 10:28:31,193 - lava_callback - ERROR - Error processing callback for node 69c8fecb6a4c026e16348d44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37322 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:29:51,179 - lava_callback - INFO - Received callback for node 69c8ff176a4c026e16348d57 INFO: 172.19.0.1:44062 - "POST /node/69c8ff176a4c026e16348d57 HTTP/1.0" 202 Accepted 2026-03-31 10:29:51,184 - lava_callback - INFO - Starting async job submit for node 69c8ff176a4c026e16348d57 2026-03-31 10:29:51,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:29:51,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ff176a4c026e16348d57 HTTP/1.1" 200 None 2026-03-31 10:29:51,200 - lava_callback - DEBUG - Processing node 69c8ff176a4c026e16348d57: baseline-arm64-clabbe 2026-03-31 10:29:51,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ff176a4c026e16348d57 2026-03-31 10:29:51,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:29:51,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ff176a4c026e16348d57 HTTP/1.1" 200 None 2026-03-31 10:29:51,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:29:51,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:29:51,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:29:51,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ff176a4c026e16348d57 HTTP/1.1" 200 None 2026-03-31 10:29:51,252 - lava_callback - ERROR - Error processing callback for node 69c8ff176a4c026e16348d57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47198 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:31:11,230 - lava_callback - INFO - Received callback for node 69c8ff646a4c026e16348d5c INFO: 172.19.0.1:42018 - "POST /node/69c8ff646a4c026e16348d5c HTTP/1.0" 202 Accepted 2026-03-31 10:31:11,236 - lava_callback - INFO - Starting async job submit for node 69c8ff646a4c026e16348d5c 2026-03-31 10:31:11,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:31:11,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ff646a4c026e16348d5c HTTP/1.1" 200 None 2026-03-31 10:31:11,262 - lava_callback - DEBUG - Processing node 69c8ff646a4c026e16348d5c: baseline-arm64-clabbe 2026-03-31 10:31:11,262 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ff646a4c026e16348d5c 2026-03-31 10:31:11,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:31:11,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ff646a4c026e16348d5c HTTP/1.1" 200 None 2026-03-31 10:31:11,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:31:11,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:31:11,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:31:11,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ff646a4c026e16348d5c HTTP/1.1" 200 None 2026-03-31 10:31:11,325 - lava_callback - ERROR - Error processing callback for node 69c8ff646a4c026e16348d5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52790 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:32:33,858 - lava_callback - INFO - Received callback for node 69c8ffb46a4c026e16348d66 INFO: 172.19.0.1:56866 - "POST /node/69c8ffb46a4c026e16348d66 HTTP/1.0" 202 Accepted 2026-03-31 10:32:33,864 - lava_callback - INFO - Starting async job submit for node 69c8ffb46a4c026e16348d66 2026-03-31 10:32:33,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:32:33,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ffb46a4c026e16348d66 HTTP/1.1" 200 None 2026-03-31 10:32:33,880 - lava_callback - DEBUG - Processing node 69c8ffb46a4c026e16348d66: baseline-arm64-clabbe 2026-03-31 10:32:33,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ffb46a4c026e16348d66 2026-03-31 10:32:33,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:32:33,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ffb46a4c026e16348d66 HTTP/1.1" 200 None 2026-03-31 10:32:33,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:32:33,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:32:33,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:32:33,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ffb46a4c026e16348d66 HTTP/1.1" 200 None 2026-03-31 10:32:33,949 - lava_callback - ERROR - Error processing callback for node 69c8ffb46a4c026e16348d66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33126 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33130 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:33:51,352 - lava_callback - INFO - Received callback for node 69c900026a4c026e16348d6f INFO: 172.19.0.1:34772 - "POST /node/69c900026a4c026e16348d6f HTTP/1.0" 202 Accepted 2026-03-31 10:33:51,358 - lava_callback - INFO - Starting async job submit for node 69c900026a4c026e16348d6f 2026-03-31 10:33:51,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:33:51,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c900026a4c026e16348d6f HTTP/1.1" 200 None 2026-03-31 10:33:51,376 - lava_callback - DEBUG - Processing node 69c900026a4c026e16348d6f: baseline-arm64-clabbe 2026-03-31 10:33:51,377 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c900026a4c026e16348d6f 2026-03-31 10:33:51,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:33:51,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c900026a4c026e16348d6f HTTP/1.1" 200 None 2026-03-31 10:33:51,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:33:51,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:33:51,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:33:51,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c900026a4c026e16348d6f HTTP/1.1" 200 None 2026-03-31 10:33:51,449 - lava_callback - ERROR - Error processing callback for node 69c900026a4c026e16348d6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37254 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:39:13,533 - lava_callback - INFO - Received callback for node 69cba0002a64fa9aff93787d INFO: 172.19.0.1:46750 - "POST /node/69cba0002a64fa9aff93787d HTTP/1.0" 202 Accepted 2026-03-31 10:39:13,540 - lava_callback - INFO - Starting async job submit for node 69cba0002a64fa9aff93787d 2026-03-31 10:39:13,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:39:13,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba0002a64fa9aff93787d HTTP/1.1" 200 None 2026-03-31 10:39:13,590 - lava_callback - DEBUG - Processing node 69cba0002a64fa9aff93787d: baseline-arm64 2026-03-31 10:39:14,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:39:14,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:39:14,232 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba0002a64fa9aff93787d/log.txt.gz 2026-03-31 10:39:14,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:39:14,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:39:14,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba0002a64fa9aff93787d/lava-logs.yaml 2026-03-31 10:39:14,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:39:14,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:39:14,978 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba0002a64fa9aff93787d/lava_callback.json.gz 2026-03-31 10:39:15,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:39:15,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba0002a64fa9aff93787d HTTP/1.1" 200 None 2026-03-31 10:39:15,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:39:15,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:39:15,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:39:15,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba0002a64fa9aff93787d HTTP/1.1" 200 None INFO: 172.19.0.1:46762 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 10:39:15,168 - lava_callback - INFO - Completed processing callback for node 69cba0002a64fa9aff93787d INFO: 172.19.0.1:46778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39256 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:39:38,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:39:38,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:47484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37514 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:41:04,209 - lava_callback - INFO - Received callback for node 69cba0982a64fa9aff937892 INFO: 172.19.0.1:56340 - "POST /node/69cba0982a64fa9aff937892 HTTP/1.0" 202 Accepted 2026-03-31 10:41:04,216 - lava_callback - INFO - Starting async job submit for node 69cba0982a64fa9aff937892 2026-03-31 10:41:04,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:04,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba0982a64fa9aff937892 HTTP/1.1" 200 None 2026-03-31 10:41:04,290 - lava_callback - DEBUG - Processing node 69cba0982a64fa9aff937892: ltp-smoke 2026-03-31 10:41:04,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:04,967 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:04,969 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba0982a64fa9aff937892/log.txt.gz 2026-03-31 10:41:05,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:05,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:05,593 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba0982a64fa9aff937892/lava-logs.yaml 2026-03-31 10:41:05,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:05,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:05,787 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba0982a64fa9aff937892/lava_callback.json.gz 2026-03-31 10:41:05,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:05,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba0982a64fa9aff937892 HTTP/1.1" 200 None 2026-03-31 10:41:05,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:05,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83462a64fa9aff936905 HTTP/1.1" 200 None 2026-03-31 10:41:05,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:05,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba0982a64fa9aff937892 HTTP/1.1" 200 None 2026-03-31 10:41:06,101 - lava_callback - INFO - Completed processing callback for node 69cba0982a64fa9aff937892 INFO: 172.19.0.1:56352 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:41:08,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:08,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:56974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52172 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:41:40,902 - lava_callback - INFO - Received callback for node 69cb9f912a64fa9aff93786e INFO: 172.19.0.1:33744 - "POST /node/69cb9f912a64fa9aff93786e HTTP/1.0" 202 Accepted 2026-03-31 10:41:40,951 - lava_callback - INFO - Starting async job submit for node 69cb9f912a64fa9aff93786e 2026-03-31 10:41:40,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:40,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9f912a64fa9aff93786e HTTP/1.1" 200 None 2026-03-31 10:41:40,992 - lava_callback - DEBUG - Processing node 69cb9f912a64fa9aff93786e: baseline-arm-pengutronix 2026-03-31 10:41:41,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:41,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:41,268 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb9f912a64fa9aff93786e/log.txt.gz 2026-03-31 10:41:41,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:41,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:41,523 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb9f912a64fa9aff93786e/lava-logs.yaml 2026-03-31 10:41:41,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:41,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:41,647 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb9f912a64fa9aff93786e/lava_callback.json.gz DEBUG: Setup failed for 69cb9f912a64fa9aff93786e. Transitting job node result: incomplete -> fail 2026-03-31 10:41:41,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:41,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9f912a64fa9aff93786e HTTP/1.1" 200 None 2026-03-31 10:41:41,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:41,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 10:41:41,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:41,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb9f912a64fa9aff93786e HTTP/1.1" 200 None 2026-03-31 10:41:41,830 - lava_callback - INFO - Completed processing callback for node 69cb9f912a64fa9aff93786e INFO: 172.19.0.1:33750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51958 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:42:02,449 - lava_callback - INFO - Received callback for node 69cba2772a64fa9aff9378ce INFO: 172.19.0.1:49576 - "POST /node/69cba2772a64fa9aff9378ce HTTP/1.0" 202 Accepted 2026-03-31 10:42:02,456 - lava_callback - INFO - Starting async job submit for node 69cba2772a64fa9aff9378ce 2026-03-31 10:42:02,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:02,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba2772a64fa9aff9378ce HTTP/1.1" 200 None 2026-03-31 10:42:02,500 - lava_callback - DEBUG - Processing node 69cba2772a64fa9aff9378ce: baseline-nfs-arm64 2026-03-31 10:42:03,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:03,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:03,180 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba2772a64fa9aff9378ce/log.txt.gz 2026-03-31 10:42:03,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:03,702 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:03,705 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba2772a64fa9aff9378ce/lava-logs.yaml 2026-03-31 10:42:03,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:03,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:03,910 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba2772a64fa9aff9378ce/lava_callback.json.gz 2026-03-31 10:42:03,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:03,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba2772a64fa9aff9378ce HTTP/1.1" 200 None 2026-03-31 10:42:03,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:04,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:42:04,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:04,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba2772a64fa9aff9378ce HTTP/1.1" 200 None 2026-03-31 10:42:04,083 - lava_callback - INFO - Completed processing callback for node 69cba2772a64fa9aff9378ce INFO: 172.19.0.1:49580 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:42:08,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:08,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:59838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38622 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:42:31,971 - lava_callback - INFO - Received callback for node 69cba33c2a64fa9aff9378e6 INFO: 172.19.0.1:54048 - "POST /node/69cba33c2a64fa9aff9378e6 HTTP/1.0" 202 Accepted 2026-03-31 10:42:31,979 - lava_callback - INFO - Starting async job submit for node 69cba33c2a64fa9aff9378e6 2026-03-31 10:42:32,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:32,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba33c2a64fa9aff9378e6 HTTP/1.1" 200 None 2026-03-31 10:42:32,030 - lava_callback - DEBUG - Processing node 69cba33c2a64fa9aff9378e6: baseline-arm64 2026-03-31 10:42:32,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:32,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:32,543 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba33c2a64fa9aff9378e6/log.txt.gz 2026-03-31 10:42:32,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:33,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:33,018 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba33c2a64fa9aff9378e6/lava-logs.yaml 2026-03-31 10:42:33,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:33,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:33,197 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba33c2a64fa9aff9378e6/lava_callback.json.gz DEBUG: Setup failed for 69cba33c2a64fa9aff9378e6. Transitting job node result: incomplete -> fail 2026-03-31 10:42:33,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:33,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba33c2a64fa9aff9378e6 HTTP/1.1" 200 None 2026-03-31 10:42:33,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:33,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:42:33,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:33,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba33c2a64fa9aff9378e6 HTTP/1.1" 200 None 2026-03-31 10:42:33,479 - lava_callback - INFO - Completed processing callback for node 69cba33c2a64fa9aff9378e6 INFO: 172.19.0.1:54060 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:42:38,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:38,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39966 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:46:11,918 - lava_callback - INFO - Received callback for node 69c902e66a4c026e16348def INFO: 172.19.0.1:39110 - "POST /node/69c902e66a4c026e16348def HTTP/1.0" 202 Accepted 2026-03-31 10:46:11,923 - lava_callback - INFO - Starting async job submit for node 69c902e66a4c026e16348def 2026-03-31 10:46:11,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:11,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c902e66a4c026e16348def HTTP/1.1" 200 None 2026-03-31 10:46:11,949 - lava_callback - DEBUG - Processing node 69c902e66a4c026e16348def: baseline-arm-clabbe 2026-03-31 10:46:11,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c902e66a4c026e16348def 2026-03-31 10:46:11,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:11,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c902e66a4c026e16348def HTTP/1.1" 200 None 2026-03-31 10:46:11,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:12,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 10:46:12,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:12,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c902e66a4c026e16348def HTTP/1.1" 200 None 2026-03-31 10:46:12,031 - lava_callback - ERROR - Error processing callback for node 69c902e66a4c026e16348def: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44758 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:46:51,881 - lava_callback - INFO - Received callback for node 69c9030c6a4c026e16348dfd INFO: 172.19.0.1:40188 - "POST /node/69c9030c6a4c026e16348dfd HTTP/1.0" 202 Accepted 2026-03-31 10:46:51,886 - lava_callback - INFO - Starting async job submit for node 69c9030c6a4c026e16348dfd 2026-03-31 10:46:51,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:51,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9030c6a4c026e16348dfd HTTP/1.1" 200 None 2026-03-31 10:46:51,903 - lava_callback - DEBUG - Processing node 69c9030c6a4c026e16348dfd: baseline-arm-clabbe 2026-03-31 10:46:51,903 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9030c6a4c026e16348dfd 2026-03-31 10:46:51,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:51,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9030c6a4c026e16348dfd HTTP/1.1" 200 None 2026-03-31 10:46:51,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:51,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 10:46:51,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:51,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9030c6a4c026e16348dfd HTTP/1.1" 200 None 2026-03-31 10:46:51,975 - lava_callback - ERROR - Error processing callback for node 69c9030c6a4c026e16348dfd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48970 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:47:31,930 - lava_callback - INFO - Received callback for node 69c9033b6a4c026e16348e08 INFO: 172.19.0.1:33360 - "POST /node/69c9033b6a4c026e16348e08 HTTP/1.0" 202 Accepted 2026-03-31 10:47:31,935 - lava_callback - INFO - Starting async job submit for node 69c9033b6a4c026e16348e08 2026-03-31 10:47:31,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:47:31,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9033b6a4c026e16348e08 HTTP/1.1" 200 None 2026-03-31 10:47:31,955 - lava_callback - DEBUG - Processing node 69c9033b6a4c026e16348e08: baseline-arm-clabbe 2026-03-31 10:47:31,955 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9033b6a4c026e16348e08 2026-03-31 10:47:31,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:47:31,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9033b6a4c026e16348e08 HTTP/1.1" 200 None 2026-03-31 10:47:31,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:47:32,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 10:47:32,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:47:32,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9033b6a4c026e16348e08 HTTP/1.1" 200 None 2026-03-31 10:47:32,033 - lava_callback - ERROR - Error processing callback for node 69c9033b6a4c026e16348e08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55282 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:48:12,045 - lava_callback - INFO - Received callback for node 69c903616a4c026e16348e15 INFO: 172.19.0.1:47734 - "POST /node/69c903616a4c026e16348e15 HTTP/1.0" 202 Accepted 2026-03-31 10:48:12,050 - lava_callback - INFO - Starting async job submit for node 69c903616a4c026e16348e15 2026-03-31 10:48:12,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:12,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c903616a4c026e16348e15 HTTP/1.1" 200 None 2026-03-31 10:48:12,088 - lava_callback - DEBUG - Processing node 69c903616a4c026e16348e15: baseline-arm-clabbe 2026-03-31 10:48:12,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c903616a4c026e16348e15 2026-03-31 10:48:12,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:12,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c903616a4c026e16348e15 HTTP/1.1" 200 None 2026-03-31 10:48:12,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:12,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 10:48:12,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:12,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c903616a4c026e16348e15 HTTP/1.1" 200 None 2026-03-31 10:48:12,142 - lava_callback - ERROR - Error processing callback for node 69c903616a4c026e16348e15: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40380 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:48:40,884 - lava_callback - INFO - Received callback for node 69cba1da2a64fa9aff9378b8 INFO: 172.19.0.1:33700 - "POST /node/69cba1da2a64fa9aff9378b8 HTTP/1.0" 202 Accepted 2026-03-31 10:48:40,891 - lava_callback - INFO - Starting async job submit for node 69cba1da2a64fa9aff9378b8 2026-03-31 10:48:40,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:40,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba1da2a64fa9aff9378b8 HTTP/1.1" 200 None 2026-03-31 10:48:40,954 - lava_callback - DEBUG - Processing node 69cba1da2a64fa9aff9378b8: baseline-nfs-arm64 2026-03-31 10:48:41,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:48:42,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:48:42,317 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba1da2a64fa9aff9378b8/log.txt.gz 2026-03-31 10:48:42,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:48:43,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:48:43,152 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba1da2a64fa9aff9378b8/lava-logs.yaml 2026-03-31 10:48:43,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:48:43,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:48:43,373 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba1da2a64fa9aff9378b8/lava_callback.json.gz DEBUG: Setup failed for 69cba1da2a64fa9aff9378b8. Transitting job node result: incomplete -> fail 2026-03-31 10:48:43,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:43,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba1da2a64fa9aff9378b8 HTTP/1.1" 200 None 2026-03-31 10:48:43,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:43,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:48:43,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:43,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba1da2a64fa9aff9378b8 HTTP/1.1" 200 None 2026-03-31 10:48:43,601 - lava_callback - INFO - Completed processing callback for node 69cba1da2a64fa9aff9378b8 INFO: 172.19.0.1:33702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47568 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:49:08,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:08,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:53304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53314 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:49:17,428 - lava_callback - INFO - Received callback for node 69cba4592a64fa9aff9378f1 INFO: 172.19.0.1:53320 - "POST /node/69cba4592a64fa9aff9378f1 HTTP/1.0" 202 Accepted 2026-03-31 10:49:17,435 - lava_callback - INFO - Starting async job submit for node 69cba4592a64fa9aff9378f1 2026-03-31 10:49:17,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:17,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4592a64fa9aff9378f1 HTTP/1.1" 200 None 2026-03-31 10:49:17,483 - lava_callback - DEBUG - Processing node 69cba4592a64fa9aff9378f1: baseline-arm64 2026-03-31 10:49:17,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:49:18,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:49:18,120 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba4592a64fa9aff9378f1/log.txt.gz 2026-03-31 10:49:18,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:49:18,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:49:18,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba4592a64fa9aff9378f1/lava-logs.yaml 2026-03-31 10:49:18,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:49:18,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:49:18,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba4592a64fa9aff9378f1/lava_callback.json.gz 2026-03-31 10:49:18,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:18,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4592a64fa9aff9378f1 HTTP/1.1" 200 None 2026-03-31 10:49:18,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:18,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:49:18,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:18,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba4592a64fa9aff9378f1 HTTP/1.1" 200 None 2026-03-31 10:49:18,921 - lava_callback - INFO - Completed processing callback for node 69cba4592a64fa9aff9378f1 INFO: 172.19.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43994 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:49:38,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:38,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:59208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45224 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:50:12,036 - lava_callback - INFO - Received callback for node 69c903ce6a4c026e16348e27 INFO: 172.19.0.1:37948 - "POST /node/69c903ce6a4c026e16348e27 HTTP/1.0" 202 Accepted 2026-03-31 10:50:12,041 - lava_callback - INFO - Starting async job submit for node 69c903ce6a4c026e16348e27 2026-03-31 10:50:12,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:12,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c903ce6a4c026e16348e27 HTTP/1.1" 200 None 2026-03-31 10:50:12,058 - lava_callback - DEBUG - Processing node 69c903ce6a4c026e16348e27: baseline-arm-clabbe 2026-03-31 10:50:12,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c903ce6a4c026e16348e27 2026-03-31 10:50:12,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:12,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c903ce6a4c026e16348e27 HTTP/1.1" 200 None 2026-03-31 10:50:12,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:12,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 10:50:12,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:12,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c903ce6a4c026e16348e27 HTTP/1.1" 200 None 2026-03-31 10:50:12,134 - lava_callback - ERROR - Error processing callback for node 69c903ce6a4c026e16348e27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37712 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:50:30,655 - lava_callback - INFO - Received callback for node 69cba3132a64fa9aff9378dd INFO: 172.19.0.1:52974 - "POST /node/69cba3132a64fa9aff9378dd HTTP/1.0" 202 Accepted 2026-03-31 10:50:30,662 - lava_callback - INFO - Starting async job submit for node 69cba3132a64fa9aff9378dd 2026-03-31 10:50:30,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:30,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba3132a64fa9aff9378dd HTTP/1.1" 200 None 2026-03-31 10:50:30,731 - lava_callback - DEBUG - Processing node 69cba3132a64fa9aff9378dd: baseline-arm 2026-03-31 10:50:31,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:50:31,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:50:31,538 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cba3132a64fa9aff9378dd/log.txt.gz 2026-03-31 10:50:31,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:50:32,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:50:32,341 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cba3132a64fa9aff9378dd/lava-logs.yaml 2026-03-31 10:50:32,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:50:32,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:50:32,528 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cba3132a64fa9aff9378dd/lava_callback.json.gz DEBUG: Setup failed for 69cba3132a64fa9aff9378dd. Transitting job node result: incomplete -> fail 2026-03-31 10:50:32,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:32,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba3132a64fa9aff9378dd HTTP/1.1" 200 None 2026-03-31 10:50:32,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:32,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 10:50:32,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:32,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba3132a64fa9aff9378dd HTTP/1.1" 200 None 2026-03-31 10:50:32,808 - lava_callback - INFO - Completed processing callback for node 69cba3132a64fa9aff9378dd INFO: 172.19.0.1:52986 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:50:38,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:38,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37136 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:53:05,656 - lava_callback - INFO - Received callback for node 69cba5142a64fa9aff937918 INFO: 172.19.0.1:40748 - "POST /node/69cba5142a64fa9aff937918 HTTP/1.0" 202 Accepted 2026-03-31 10:53:05,662 - lava_callback - INFO - Starting async job submit for node 69cba5142a64fa9aff937918 2026-03-31 10:53:05,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:53:05,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba5142a64fa9aff937918 HTTP/1.1" 200 None 2026-03-31 10:53:05,714 - lava_callback - DEBUG - Processing node 69cba5142a64fa9aff937918: baseline-nfs-arm64 2026-03-31 10:53:06,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:53:06,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:53:06,352 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba5142a64fa9aff937918/log.txt.gz INFO: 172.19.0.1:40756 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:53:06,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:53:07,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:53:07,020 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba5142a64fa9aff937918/lava-logs.yaml 2026-03-31 10:53:07,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:53:07,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:53:07,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba5142a64fa9aff937918/lava_callback.json.gz 2026-03-31 10:53:07,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:53:07,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba5142a64fa9aff937918 HTTP/1.1" 200 None 2026-03-31 10:53:07,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:53:07,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:53:07,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:53:07,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba5142a64fa9aff937918 HTTP/1.1" 200 None 2026-03-31 10:53:07,392 - lava_callback - INFO - Completed processing callback for node 69cba5142a64fa9aff937918 2026-03-31 10:53:08,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:53:09,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:37726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34366 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:57:33,740 - lava_callback - INFO - Received callback for node 69c9059a6a4c026e16348e53 INFO: 172.19.0.1:36362 - "POST /node/69c9059a6a4c026e16348e53 HTTP/1.0" 202 Accepted 2026-03-31 10:57:33,745 - lava_callback - INFO - Starting async job submit for node 69c9059a6a4c026e16348e53 2026-03-31 10:57:33,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:57:33,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9059a6a4c026e16348e53 HTTP/1.1" 200 None 2026-03-31 10:57:33,762 - lava_callback - DEBUG - Processing node 69c9059a6a4c026e16348e53: baseline-arm64-baylibre 2026-03-31 10:57:33,763 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9059a6a4c026e16348e53 2026-03-31 10:57:33,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:57:33,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9059a6a4c026e16348e53 HTTP/1.1" 200 None 2026-03-31 10:57:33,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:57:33,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 10:57:33,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:57:33,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9059a6a4c026e16348e53 HTTP/1.1" 200 None 2026-03-31 10:57:33,829 - lava_callback - ERROR - Error processing callback for node 69c9059a6a4c026e16348e53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44714 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:58:44,727 - lava_callback - INFO - Received callback for node 69cba4c32a64fa9aff9378fe INFO: 172.19.0.1:50942 - "POST /node/69cba4c32a64fa9aff9378fe HTTP/1.0" 202 Accepted 2026-03-31 10:58:44,734 - lava_callback - INFO - Starting async job submit for node 69cba4c32a64fa9aff9378fe 2026-03-31 10:58:44,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:44,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4c32a64fa9aff9378fe HTTP/1.1" 200 None 2026-03-31 10:58:44,812 - lava_callback - DEBUG - Processing node 69cba4c32a64fa9aff9378fe: ltp-smoke 2026-03-31 10:58:45,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:58:45,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:58:45,454 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba4c32a64fa9aff9378fe/log.txt.gz 2026-03-31 10:58:45,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:58:45,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:58:45,958 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba4c32a64fa9aff9378fe/lava-logs.yaml 2026-03-31 10:58:45,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:58:46,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:58:46,130 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba4c32a64fa9aff9378fe/lava_callback.json.gz 2026-03-31 10:58:46,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:46,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4c32a64fa9aff9378fe HTTP/1.1" 200 None 2026-03-31 10:58:46,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:46,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83462a64fa9aff936905 HTTP/1.1" 200 None 2026-03-31 10:58:46,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:46,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba4c32a64fa9aff9378fe HTTP/1.1" 200 None 2026-03-31 10:58:46,457 - lava_callback - INFO - Completed processing callback for node 69cba4c32a64fa9aff9378fe INFO: 172.19.0.1:50952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50962 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:58:53,798 - lava_callback - INFO - Received callback for node 69c905e86a4c026e16348e65 INFO: 172.19.0.1:48918 - "POST /node/69c905e86a4c026e16348e65 HTTP/1.0" 202 Accepted 2026-03-31 10:58:53,807 - lava_callback - INFO - Starting async job submit for node 69c905e86a4c026e16348e65 2026-03-31 10:58:53,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:53,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c905e86a4c026e16348e65 HTTP/1.1" 200 None 2026-03-31 10:58:53,832 - lava_callback - DEBUG - Processing node 69c905e86a4c026e16348e65: baseline-arm64-baylibre 2026-03-31 10:58:53,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c905e86a4c026e16348e65 2026-03-31 10:58:53,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:53,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c905e86a4c026e16348e65 HTTP/1.1" 200 None 2026-03-31 10:58:53,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:53,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 10:58:53,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:53,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c905e86a4c026e16348e65 HTTP/1.1" 200 None 2026-03-31 10:58:53,908 - lava_callback - ERROR - Error processing callback for node 69c905e86a4c026e16348e65: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60728 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:59:09,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:59:09,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:50704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50122 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:00:53,891 - lava_callback - INFO - Received callback for node 69c906516a4c026e16348e74 INFO: 172.19.0.1:46600 - "POST /node/69c906516a4c026e16348e74 HTTP/1.0" 202 Accepted 2026-03-31 11:00:53,897 - lava_callback - INFO - Starting async job submit for node 69c906516a4c026e16348e74 2026-03-31 11:00:53,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:00:53,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c906516a4c026e16348e74 HTTP/1.1" 200 None 2026-03-31 11:00:53,927 - lava_callback - DEBUG - Processing node 69c906516a4c026e16348e74: baseline-arm64-baylibre 2026-03-31 11:00:53,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c906516a4c026e16348e74 2026-03-31 11:00:53,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:00:53,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c906516a4c026e16348e74 HTTP/1.1" 200 None 2026-03-31 11:00:53,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:00:53,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:00:53,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:00:53,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c906516a4c026e16348e74 HTTP/1.1" 200 None 2026-03-31 11:00:53,995 - lava_callback - ERROR - Error processing callback for node 69c906516a4c026e16348e74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57218 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:01:22,040 - lava_callback - INFO - Received callback for node 69cba4f22a64fa9aff93790b INFO: 172.19.0.1:48122 - "POST /node/69cba4f22a64fa9aff93790b HTTP/1.0" 202 Accepted 2026-03-31 11:01:22,089 - lava_callback - INFO - Starting async job submit for node 69cba4f22a64fa9aff93790b 2026-03-31 11:01:22,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:01:22,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4f22a64fa9aff93790b HTTP/1.1" 200 None 2026-03-31 11:01:22,143 - lava_callback - DEBUG - Processing node 69cba4f22a64fa9aff93790b: baseline-arm-pengutronix 2026-03-31 11:01:22,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:01:22,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:01:22,409 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cba4f22a64fa9aff93790b/log.txt.gz 2026-03-31 11:01:22,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:01:22,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:01:22,660 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cba4f22a64fa9aff93790b/lava-logs.yaml 2026-03-31 11:01:22,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:01:22,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:01:22,802 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cba4f22a64fa9aff93790b/lava_callback.json.gz DEBUG: Setup failed for 69cba4f22a64fa9aff93790b. Transitting job node result: incomplete -> fail 2026-03-31 11:01:22,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:01:22,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4f22a64fa9aff93790b HTTP/1.1" 200 None 2026-03-31 11:01:22,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:01:22,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 11:01:22,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:01:22,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba4f22a64fa9aff93790b HTTP/1.1" 200 None 2026-03-31 11:01:23,005 - lava_callback - INFO - Completed processing callback for node 69cba4f22a64fa9aff93790b INFO: 172.19.0.1:48138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46960 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:01:39,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:01:39,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:32896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56140 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:03:14,007 - lava_callback - INFO - Received callback for node 69c906e86a4c026e16348e8b INFO: 172.19.0.1:45236 - "POST /node/69c906e86a4c026e16348e8b HTTP/1.0" 202 Accepted 2026-03-31 11:03:14,013 - lava_callback - INFO - Starting async job submit for node 69c906e86a4c026e16348e8b 2026-03-31 11:03:14,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:03:14,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c906e86a4c026e16348e8b HTTP/1.1" 200 None 2026-03-31 11:03:14,040 - lava_callback - DEBUG - Processing node 69c906e86a4c026e16348e8b: baseline-arm64-baylibre 2026-03-31 11:03:14,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c906e86a4c026e16348e8b 2026-03-31 11:03:14,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:03:14,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c906e86a4c026e16348e8b HTTP/1.1" 200 None 2026-03-31 11:03:14,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:03:14,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:03:14,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:03:14,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c906e86a4c026e16348e8b HTTP/1.1" 200 None 2026-03-31 11:03:14,107 - lava_callback - ERROR - Error processing callback for node 69c906e86a4c026e16348e8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58550 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:08:34,309 - lava_callback - INFO - Received callback for node 69c908236a4c026e16348eba INFO: 172.19.0.1:50476 - "POST /node/69c908236a4c026e16348eba HTTP/1.0" 202 Accepted 2026-03-31 11:08:34,315 - lava_callback - INFO - Starting async job submit for node 69c908236a4c026e16348eba 2026-03-31 11:08:34,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:08:34,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c908236a4c026e16348eba HTTP/1.1" 200 None 2026-03-31 11:08:34,331 - lava_callback - DEBUG - Processing node 69c908236a4c026e16348eba: baseline-arm64-baylibre 2026-03-31 11:08:34,331 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c908236a4c026e16348eba 2026-03-31 11:08:34,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:08:34,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c908236a4c026e16348eba HTTP/1.1" 200 None 2026-03-31 11:08:34,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:08:34,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:08:34,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:08:34,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c908236a4c026e16348eba HTTP/1.1" 200 None 2026-03-31 11:08:34,387 - lava_callback - ERROR - Error processing callback for node 69c908236a4c026e16348eba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45908 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:09:01,550 - lava_callback - INFO - Received callback for node 69cba6942a64fa9aff93793e INFO: 172.19.0.1:57724 - "POST /node/69cba6942a64fa9aff93793e HTTP/1.0" 202 Accepted 2026-03-31 11:09:01,557 - lava_callback - INFO - Starting async job submit for node 69cba6942a64fa9aff93793e 2026-03-31 11:09:01,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:09:01,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba6942a64fa9aff93793e HTTP/1.1" 200 None 2026-03-31 11:09:01,620 - lava_callback - DEBUG - Processing node 69cba6942a64fa9aff93793e: baseline-nfs-arm64 2026-03-31 11:09:02,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:09:02,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:09:02,812 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba6942a64fa9aff93793e/log.txt.gz 2026-03-31 11:09:03,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:09:03,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:09:03,648 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba6942a64fa9aff93793e/lava-logs.yaml 2026-03-31 11:09:03,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:09:03,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:09:03,865 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba6942a64fa9aff93793e/lava_callback.json.gz DEBUG: Setup failed for 69cba6942a64fa9aff93793e. Transitting job node result: incomplete -> fail 2026-03-31 11:09:03,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:09:03,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba6942a64fa9aff93793e HTTP/1.1" 200 None 2026-03-31 11:09:03,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:09:03,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 11:09:03,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:09:04,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba6942a64fa9aff93793e HTTP/1.1" 200 None 2026-03-31 11:09:04,102 - lava_callback - INFO - Completed processing callback for node 69cba6942a64fa9aff93793e INFO: 172.19.0.1:57726 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:09:09,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:09:09,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:58400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58956 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:11:14,495 - lava_callback - INFO - Received callback for node 69c908be6a4c026e16348ed2 INFO: 172.19.0.1:32994 - "POST /node/69c908be6a4c026e16348ed2 HTTP/1.0" 202 Accepted 2026-03-31 11:11:14,500 - lava_callback - INFO - Starting async job submit for node 69c908be6a4c026e16348ed2 2026-03-31 11:11:14,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:11:14,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c908be6a4c026e16348ed2 HTTP/1.1" 200 None 2026-03-31 11:11:14,525 - lava_callback - DEBUG - Processing node 69c908be6a4c026e16348ed2: baseline-arm64-baylibre 2026-03-31 11:11:14,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c908be6a4c026e16348ed2 2026-03-31 11:11:14,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:11:14,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c908be6a4c026e16348ed2 HTTP/1.1" 200 None 2026-03-31 11:11:14,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:11:14,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:11:14,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:11:14,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c908be6a4c026e16348ed2 HTTP/1.1" 200 None 2026-03-31 11:11:14,628 - lava_callback - ERROR - Error processing callback for node 69c908be6a4c026e16348ed2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:32996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43274 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:12:34,510 - lava_callback - INFO - Received callback for node 69c9091e6a4c026e16348edc INFO: 172.19.0.1:60974 - "POST /node/69c9091e6a4c026e16348edc HTTP/1.0" 202 Accepted 2026-03-31 11:12:34,515 - lava_callback - INFO - Starting async job submit for node 69c9091e6a4c026e16348edc 2026-03-31 11:12:34,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:12:34,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9091e6a4c026e16348edc HTTP/1.1" 200 None 2026-03-31 11:12:34,538 - lava_callback - DEBUG - Processing node 69c9091e6a4c026e16348edc: baseline-arm64-baylibre 2026-03-31 11:12:34,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9091e6a4c026e16348edc 2026-03-31 11:12:34,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:12:34,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9091e6a4c026e16348edc HTTP/1.1" 200 None 2026-03-31 11:12:34,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:12:34,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:12:34,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:12:34,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9091e6a4c026e16348edc HTTP/1.1" 200 None 2026-03-31 11:12:34,622 - lava_callback - ERROR - Error processing callback for node 69c9091e6a4c026e16348edc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39572 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:14:13,103 - lava_callback - INFO - Received callback for node 69c909726a4c026e16348ee9 INFO: 172.19.0.1:42008 - "POST /node/69c909726a4c026e16348ee9 HTTP/1.0" 202 Accepted 2026-03-31 11:14:13,110 - lava_callback - INFO - Starting async job submit for node 69c909726a4c026e16348ee9 2026-03-31 11:14:13,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:14:13,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c909726a4c026e16348ee9 HTTP/1.1" 200 None 2026-03-31 11:14:13,126 - lava_callback - DEBUG - Processing node 69c909726a4c026e16348ee9: baseline-arm64-clabbe 2026-03-31 11:14:13,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c909726a4c026e16348ee9 2026-03-31 11:14:13,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:14:13,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c909726a4c026e16348ee9 HTTP/1.1" 200 None 2026-03-31 11:14:13,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:14:13,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:14:13,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:14:13,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c909726a4c026e16348ee9 HTTP/1.1" 200 None 2026-03-31 11:14:13,188 - lava_callback - ERROR - Error processing callback for node 69c909726a4c026e16348ee9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46656 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:15:14,760 - lava_callback - INFO - Received callback for node 69c909b86a4c026e16348ef7 INFO: 172.19.0.1:58230 - "POST /node/69c909b86a4c026e16348ef7 HTTP/1.0" 202 Accepted 2026-03-31 11:15:14,766 - lava_callback - INFO - Starting async job submit for node 69c909b86a4c026e16348ef7 2026-03-31 11:15:14,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:15:14,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c909b86a4c026e16348ef7 HTTP/1.1" 200 None 2026-03-31 11:15:14,784 - lava_callback - DEBUG - Processing node 69c909b86a4c026e16348ef7: baseline-arm64-baylibre 2026-03-31 11:15:14,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c909b86a4c026e16348ef7 2026-03-31 11:15:14,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:15:14,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c909b86a4c026e16348ef7 HTTP/1.1" 200 None 2026-03-31 11:15:14,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:15:14,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:15:14,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:15:14,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c909b86a4c026e16348ef7 HTTP/1.1" 200 None 2026-03-31 11:15:14,857 - lava_callback - ERROR - Error processing callback for node 69c909b86a4c026e16348ef7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52208 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:16:33,239 - lava_callback - INFO - Received callback for node 69c90a0a6a4c026e16348f09 INFO: 172.19.0.1:57062 - "POST /node/69c90a0a6a4c026e16348f09 HTTP/1.0" 202 Accepted 2026-03-31 11:16:33,244 - lava_callback - INFO - Starting async job submit for node 69c90a0a6a4c026e16348f09 2026-03-31 11:16:33,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:16:33,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a0a6a4c026e16348f09 HTTP/1.1" 200 None 2026-03-31 11:16:33,263 - lava_callback - DEBUG - Processing node 69c90a0a6a4c026e16348f09: baseline-arm64-clabbe 2026-03-31 11:16:33,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90a0a6a4c026e16348f09 2026-03-31 11:16:33,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:16:33,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a0a6a4c026e16348f09 HTTP/1.1" 200 None 2026-03-31 11:16:33,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:16:33,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:16:33,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:16:33,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90a0a6a4c026e16348f09 HTTP/1.1" 200 None 2026-03-31 11:16:33,316 - lava_callback - ERROR - Error processing callback for node 69c90a0a6a4c026e16348f09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52786 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:17:53,228 - lava_callback - INFO - Received callback for node 69c90a576a4c026e16348f16 INFO: 172.19.0.1:35206 - "POST /node/69c90a576a4c026e16348f16 HTTP/1.0" 202 Accepted 2026-03-31 11:17:53,234 - lava_callback - INFO - Starting async job submit for node 69c90a576a4c026e16348f16 2026-03-31 11:17:53,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:17:53,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a576a4c026e16348f16 HTTP/1.1" 200 None 2026-03-31 11:17:53,256 - lava_callback - DEBUG - Processing node 69c90a576a4c026e16348f16: baseline-arm64-clabbe 2026-03-31 11:17:53,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90a576a4c026e16348f16 2026-03-31 11:17:53,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:17:53,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a576a4c026e16348f16 HTTP/1.1" 200 None 2026-03-31 11:17:53,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:17:53,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:17:53,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:17:53,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90a576a4c026e16348f16 HTTP/1.1" 200 None 2026-03-31 11:17:53,317 - lava_callback - ERROR - Error processing callback for node 69c90a576a4c026e16348f16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54408 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:19:15,025 - lava_callback - INFO - Received callback for node 69c90a9f6a4c026e16348f20 INFO: 172.19.0.1:51910 - "POST /node/69c90a9f6a4c026e16348f20 HTTP/1.0" 202 Accepted 2026-03-31 11:19:15,030 - lava_callback - INFO - Starting async job submit for node 69c90a9f6a4c026e16348f20 2026-03-31 11:19:15,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:19:15,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a9f6a4c026e16348f20 HTTP/1.1" 200 None 2026-03-31 11:19:15,049 - lava_callback - DEBUG - Processing node 69c90a9f6a4c026e16348f20: baseline-arm64-baylibre 2026-03-31 11:19:15,050 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90a9f6a4c026e16348f20 2026-03-31 11:19:15,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:19:15,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a9f6a4c026e16348f20 HTTP/1.1" 200 None 2026-03-31 11:19:15,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:19:15,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:19:15,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:19:15,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90a9f6a4c026e16348f20 HTTP/1.1" 200 None 2026-03-31 11:19:15,120 - lava_callback - ERROR - Error processing callback for node 69c90a9f6a4c026e16348f20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47164 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:20:33,327 - lava_callback - INFO - Received callback for node 69c90af06a4c026e16348f30 INFO: 172.19.0.1:49188 - "POST /node/69c90af06a4c026e16348f30 HTTP/1.0" 202 Accepted 2026-03-31 11:20:33,333 - lava_callback - INFO - Starting async job submit for node 69c90af06a4c026e16348f30 2026-03-31 11:20:33,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:20:33,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90af06a4c026e16348f30 HTTP/1.1" 200 None 2026-03-31 11:20:33,349 - lava_callback - DEBUG - Processing node 69c90af06a4c026e16348f30: baseline-arm64-clabbe 2026-03-31 11:20:33,349 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90af06a4c026e16348f30 2026-03-31 11:20:33,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:20:33,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90af06a4c026e16348f30 HTTP/1.1" 200 None 2026-03-31 11:20:33,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:20:33,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:20:33,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:20:33,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90af06a4c026e16348f30 HTTP/1.1" 200 None 2026-03-31 11:20:33,410 - lava_callback - ERROR - Error processing callback for node 69c90af06a4c026e16348f30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46150 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:21:53,400 - lava_callback - INFO - Received callback for node 69c90b3d6a4c026e16348f3c INFO: 172.19.0.1:58520 - "POST /node/69c90b3d6a4c026e16348f3c HTTP/1.0" 202 Accepted 2026-03-31 11:21:53,405 - lava_callback - INFO - Starting async job submit for node 69c90b3d6a4c026e16348f3c 2026-03-31 11:21:53,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:21:53,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90b3d6a4c026e16348f3c HTTP/1.1" 200 None 2026-03-31 11:21:53,444 - lava_callback - DEBUG - Processing node 69c90b3d6a4c026e16348f3c: baseline-arm64-clabbe 2026-03-31 11:21:53,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90b3d6a4c026e16348f3c 2026-03-31 11:21:53,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:21:53,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90b3d6a4c026e16348f3c HTTP/1.1" 200 None 2026-03-31 11:21:53,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:21:53,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:21:53,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:21:53,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90b3d6a4c026e16348f3c HTTP/1.1" 200 None 2026-03-31 11:21:53,525 - lava_callback - ERROR - Error processing callback for node 69c90b3d6a4c026e16348f3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42722 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:23:13,628 - lava_callback - INFO - Received callback for node 69c90b8b6a4c026e16348f4b INFO: 172.19.0.1:33052 - "POST /node/69c90b8b6a4c026e16348f4b HTTP/1.0" 202 Accepted 2026-03-31 11:23:13,634 - lava_callback - INFO - Starting async job submit for node 69c90b8b6a4c026e16348f4b 2026-03-31 11:23:13,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:23:13,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90b8b6a4c026e16348f4b HTTP/1.1" 200 None 2026-03-31 11:23:13,664 - lava_callback - DEBUG - Processing node 69c90b8b6a4c026e16348f4b: baseline-arm64-clabbe 2026-03-31 11:23:13,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90b8b6a4c026e16348f4b 2026-03-31 11:23:13,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:23:13,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90b8b6a4c026e16348f4b HTTP/1.1" 200 None 2026-03-31 11:23:13,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:23:13,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:23:13,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:23:13,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90b8b6a4c026e16348f4b HTTP/1.1" 200 None 2026-03-31 11:23:13,755 - lava_callback - ERROR - Error processing callback for node 69c90b8b6a4c026e16348f4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47538 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:24:13,454 - lava_callback - INFO - Received callback for node 69c90bd86a4c026e16348f57 INFO: 172.19.0.1:41740 - "POST /node/69c90bd86a4c026e16348f57 HTTP/1.0" 202 Accepted 2026-03-31 11:24:13,459 - lava_callback - INFO - Starting async job submit for node 69c90bd86a4c026e16348f57 2026-03-31 11:24:13,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:24:13,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90bd86a4c026e16348f57 HTTP/1.1" 200 None 2026-03-31 11:24:13,476 - lava_callback - DEBUG - Processing node 69c90bd86a4c026e16348f57: baseline-arm64-clabbe 2026-03-31 11:24:13,476 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90bd86a4c026e16348f57 2026-03-31 11:24:13,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:24:13,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90bd86a4c026e16348f57 HTTP/1.1" 200 None 2026-03-31 11:24:13,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:24:13,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:24:13,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:24:13,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90bd86a4c026e16348f57 HTTP/1.1" 200 None 2026-03-31 11:24:13,536 - lava_callback - ERROR - Error processing callback for node 69c90bd86a4c026e16348f57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46824 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:25:33,573 - lava_callback - INFO - Received callback for node 69c90c256a4c026e16348f61 INFO: 172.19.0.1:38340 - "POST /node/69c90c256a4c026e16348f61 HTTP/1.0" 202 Accepted 2026-03-31 11:25:33,578 - lava_callback - INFO - Starting async job submit for node 69c90c256a4c026e16348f61 2026-03-31 11:25:33,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:25:33,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90c256a4c026e16348f61 HTTP/1.1" 200 None 2026-03-31 11:25:33,599 - lava_callback - DEBUG - Processing node 69c90c256a4c026e16348f61: baseline-arm64-clabbe 2026-03-31 11:25:33,599 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90c256a4c026e16348f61 2026-03-31 11:25:33,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:25:33,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90c256a4c026e16348f61 HTTP/1.1" 200 None 2026-03-31 11:25:33,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:25:33,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:25:33,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:25:33,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90c256a4c026e16348f61 HTTP/1.1" 200 None 2026-03-31 11:25:33,654 - lava_callback - ERROR - Error processing callback for node 69c90c256a4c026e16348f61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51936 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:26:53,627 - lava_callback - INFO - Received callback for node 69c90c726a4c026e16348f6f INFO: 172.19.0.1:37954 - "POST /node/69c90c726a4c026e16348f6f HTTP/1.0" 202 Accepted 2026-03-31 11:26:53,634 - lava_callback - INFO - Starting async job submit for node 69c90c726a4c026e16348f6f 2026-03-31 11:26:53,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:26:53,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90c726a4c026e16348f6f HTTP/1.1" 200 None 2026-03-31 11:26:53,652 - lava_callback - DEBUG - Processing node 69c90c726a4c026e16348f6f: baseline-arm64-clabbe 2026-03-31 11:26:53,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90c726a4c026e16348f6f 2026-03-31 11:26:53,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:26:53,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90c726a4c026e16348f6f HTTP/1.1" 200 None 2026-03-31 11:26:53,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:26:53,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:26:53,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:26:53,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90c726a4c026e16348f6f HTTP/1.1" 200 None 2026-03-31 11:26:53,728 - lava_callback - ERROR - Error processing callback for node 69c90c726a4c026e16348f6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60536 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:28:13,641 - lava_callback - INFO - Received callback for node 69c90cbf6a4c026e16348f78 INFO: 172.19.0.1:37832 - "POST /node/69c90cbf6a4c026e16348f78 HTTP/1.0" 202 Accepted 2026-03-31 11:28:13,647 - lava_callback - INFO - Starting async job submit for node 69c90cbf6a4c026e16348f78 2026-03-31 11:28:13,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:28:13,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90cbf6a4c026e16348f78 HTTP/1.1" 200 None 2026-03-31 11:28:13,663 - lava_callback - DEBUG - Processing node 69c90cbf6a4c026e16348f78: baseline-arm64-clabbe 2026-03-31 11:28:13,663 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90cbf6a4c026e16348f78 2026-03-31 11:28:13,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:28:13,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90cbf6a4c026e16348f78 HTTP/1.1" 200 None 2026-03-31 11:28:13,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:28:13,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:28:13,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:28:13,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90cbf6a4c026e16348f78 HTTP/1.1" 200 None 2026-03-31 11:28:13,725 - lava_callback - ERROR - Error processing callback for node 69c90cbf6a4c026e16348f78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38100 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:29:33,667 - lava_callback - INFO - Received callback for node 69c90d0a6a4c026e16348f86 INFO: 172.19.0.1:46382 - "POST /node/69c90d0a6a4c026e16348f86 HTTP/1.0" 202 Accepted 2026-03-31 11:29:33,673 - lava_callback - INFO - Starting async job submit for node 69c90d0a6a4c026e16348f86 2026-03-31 11:29:33,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:29:33,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90d0a6a4c026e16348f86 HTTP/1.1" 200 None 2026-03-31 11:29:33,697 - lava_callback - DEBUG - Processing node 69c90d0a6a4c026e16348f86: baseline-arm64-clabbe 2026-03-31 11:29:33,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90d0a6a4c026e16348f86 2026-03-31 11:29:33,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:29:33,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90d0a6a4c026e16348f86 HTTP/1.1" 200 None 2026-03-31 11:29:33,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:29:33,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:29:33,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:29:33,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90d0a6a4c026e16348f86 HTTP/1.1" 200 None 2026-03-31 11:29:33,761 - lava_callback - ERROR - Error processing callback for node 69c90d0a6a4c026e16348f86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48708 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:30:53,720 - lava_callback - INFO - Received callback for node 69c90d586a4c026e16348f8f INFO: 172.19.0.1:54524 - "POST /node/69c90d586a4c026e16348f8f HTTP/1.0" 202 Accepted 2026-03-31 11:30:53,725 - lava_callback - INFO - Starting async job submit for node 69c90d586a4c026e16348f8f 2026-03-31 11:30:53,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:30:53,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90d586a4c026e16348f8f HTTP/1.1" 200 None 2026-03-31 11:30:53,744 - lava_callback - DEBUG - Processing node 69c90d586a4c026e16348f8f: baseline-arm64-clabbe 2026-03-31 11:30:53,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90d586a4c026e16348f8f 2026-03-31 11:30:53,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:30:53,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90d586a4c026e16348f8f HTTP/1.1" 200 None 2026-03-31 11:30:53,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:30:53,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:30:53,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:30:53,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90d586a4c026e16348f8f HTTP/1.1" 200 None 2026-03-31 11:30:53,819 - lava_callback - ERROR - Error processing callback for node 69c90d586a4c026e16348f8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57186 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:31:53,758 - lava_callback - INFO - Received callback for node 69c90da56a4c026e16348f9e INFO: 172.19.0.1:45350 - "POST /node/69c90da56a4c026e16348f9e HTTP/1.0" 202 Accepted 2026-03-31 11:31:53,763 - lava_callback - INFO - Starting async job submit for node 69c90da56a4c026e16348f9e 2026-03-31 11:31:53,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:31:53,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90da56a4c026e16348f9e HTTP/1.1" 200 None 2026-03-31 11:31:53,780 - lava_callback - DEBUG - Processing node 69c90da56a4c026e16348f9e: baseline-arm64-clabbe 2026-03-31 11:31:53,780 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90da56a4c026e16348f9e 2026-03-31 11:31:53,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:31:53,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90da56a4c026e16348f9e HTTP/1.1" 200 None 2026-03-31 11:31:53,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:31:53,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:31:53,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:31:53,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90da56a4c026e16348f9e HTTP/1.1" 200 None 2026-03-31 11:31:53,857 - lava_callback - ERROR - Error processing callback for node 69c90da56a4c026e16348f9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39772 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:33:13,812 - lava_callback - INFO - Received callback for node 69c90df26a4c026e16348faa INFO: 172.19.0.1:41602 - "POST /node/69c90df26a4c026e16348faa HTTP/1.0" 202 Accepted 2026-03-31 11:33:13,817 - lava_callback - INFO - Starting async job submit for node 69c90df26a4c026e16348faa 2026-03-31 11:33:13,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:33:13,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90df26a4c026e16348faa HTTP/1.1" 200 None 2026-03-31 11:33:13,833 - lava_callback - DEBUG - Processing node 69c90df26a4c026e16348faa: baseline-arm64-clabbe 2026-03-31 11:33:13,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90df26a4c026e16348faa 2026-03-31 11:33:13,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:33:13,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90df26a4c026e16348faa HTTP/1.1" 200 None 2026-03-31 11:33:13,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:33:13,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:33:13,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:33:13,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90df26a4c026e16348faa HTTP/1.1" 200 None 2026-03-31 11:33:13,894 - lava_callback - ERROR - Error processing callback for node 69c90df26a4c026e16348faa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43708 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:34:33,874 - lava_callback - INFO - Received callback for node 69c90e3c6a4c026e16348fb7 INFO: 172.19.0.1:39752 - "POST /node/69c90e3c6a4c026e16348fb7 HTTP/1.0" 202 Accepted 2026-03-31 11:34:33,883 - lava_callback - INFO - Starting async job submit for node 69c90e3c6a4c026e16348fb7 2026-03-31 11:34:33,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:34:33,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90e3c6a4c026e16348fb7 HTTP/1.1" 200 None 2026-03-31 11:34:33,900 - lava_callback - DEBUG - Processing node 69c90e3c6a4c026e16348fb7: baseline-arm64-clabbe 2026-03-31 11:34:33,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90e3c6a4c026e16348fb7 2026-03-31 11:34:33,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:34:33,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90e3c6a4c026e16348fb7 HTTP/1.1" 200 None 2026-03-31 11:34:33,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:34:33,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:34:33,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:34:33,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90e3c6a4c026e16348fb7 HTTP/1.1" 200 None 2026-03-31 11:34:33,961 - lava_callback - ERROR - Error processing callback for node 69c90e3c6a4c026e16348fb7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52792 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:35:53,930 - lava_callback - INFO - Received callback for node 69c90e886a4c026e16348fc5 INFO: 172.19.0.1:54156 - "POST /node/69c90e886a4c026e16348fc5 HTTP/1.0" 202 Accepted 2026-03-31 11:35:53,935 - lava_callback - INFO - Starting async job submit for node 69c90e886a4c026e16348fc5 2026-03-31 11:35:53,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:35:53,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90e886a4c026e16348fc5 HTTP/1.1" 200 None 2026-03-31 11:35:53,952 - lava_callback - DEBUG - Processing node 69c90e886a4c026e16348fc5: baseline-arm64-clabbe 2026-03-31 11:35:53,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90e886a4c026e16348fc5 2026-03-31 11:35:53,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:35:53,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90e886a4c026e16348fc5 HTTP/1.1" 200 None 2026-03-31 11:35:53,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:35:53,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:35:53,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:35:54,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90e886a4c026e16348fc5 HTTP/1.1" 200 None 2026-03-31 11:35:54,009 - lava_callback - ERROR - Error processing callback for node 69c90e886a4c026e16348fc5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56350 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:36:55,917 - lava_callback - INFO - Received callback for node 69c90ecf6a4c026e16348fcf INFO: 172.19.0.1:58672 - "POST /node/69c90ecf6a4c026e16348fcf HTTP/1.0" 202 Accepted 2026-03-31 11:36:55,922 - lava_callback - INFO - Starting async job submit for node 69c90ecf6a4c026e16348fcf 2026-03-31 11:36:55,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:36:55,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90ecf6a4c026e16348fcf HTTP/1.1" 200 None 2026-03-31 11:36:55,938 - lava_callback - DEBUG - Processing node 69c90ecf6a4c026e16348fcf: baseline-arm64-baylibre 2026-03-31 11:36:55,939 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90ecf6a4c026e16348fcf 2026-03-31 11:36:55,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:36:55,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90ecf6a4c026e16348fcf HTTP/1.1" 200 None 2026-03-31 11:36:55,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:36:55,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:36:55,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:36:55,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90ecf6a4c026e16348fcf HTTP/1.1" 200 None 2026-03-31 11:36:56,000 - lava_callback - ERROR - Error processing callback for node 69c90ecf6a4c026e16348fcf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52772 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:38:14,042 - lava_callback - INFO - Received callback for node 69c90f206a4c026e16348fde INFO: 172.19.0.1:33634 - "POST /node/69c90f206a4c026e16348fde HTTP/1.0" 202 Accepted 2026-03-31 11:38:14,049 - lava_callback - INFO - Starting async job submit for node 69c90f206a4c026e16348fde 2026-03-31 11:38:14,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:38:14,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90f206a4c026e16348fde HTTP/1.1" 200 None 2026-03-31 11:38:14,091 - lava_callback - DEBUG - Processing node 69c90f206a4c026e16348fde: baseline-arm64-clabbe 2026-03-31 11:38:14,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90f206a4c026e16348fde 2026-03-31 11:38:14,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:38:14,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90f206a4c026e16348fde HTTP/1.1" 200 None 2026-03-31 11:38:14,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:38:14,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:38:14,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:38:14,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90f206a4c026e16348fde HTTP/1.1" 200 None 2026-03-31 11:38:14,152 - lava_callback - ERROR - Error processing callback for node 69c90f206a4c026e16348fde: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56960 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57232 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:39:34,079 - lava_callback - INFO - Received callback for node 69c90f6c6a4c026e16348fe9 INFO: 172.19.0.1:40122 - "POST /node/69c90f6c6a4c026e16348fe9 HTTP/1.0" 202 Accepted 2026-03-31 11:39:34,089 - lava_callback - INFO - Starting async job submit for node 69c90f6c6a4c026e16348fe9 2026-03-31 11:39:34,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:39:34,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90f6c6a4c026e16348fe9 HTTP/1.1" 200 None 2026-03-31 11:39:34,114 - lava_callback - DEBUG - Processing node 69c90f6c6a4c026e16348fe9: baseline-arm64-clabbe 2026-03-31 11:39:34,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90f6c6a4c026e16348fe9 2026-03-31 11:39:34,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:39:34,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90f6c6a4c026e16348fe9 HTTP/1.1" 200 None 2026-03-31 11:39:34,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:39:34,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:39:34,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:39:34,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90f6c6a4c026e16348fe9 HTTP/1.1" 200 None 2026-03-31 11:39:34,190 - lava_callback - ERROR - Error processing callback for node 69c90f6c6a4c026e16348fe9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34848 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:40:54,177 - lava_callback - INFO - Received callback for node 69c90fb86a4c026e16348ff9 INFO: 172.19.0.1:45438 - "POST /node/69c90fb86a4c026e16348ff9 HTTP/1.0" 202 Accepted 2026-03-31 11:40:54,183 - lava_callback - INFO - Starting async job submit for node 69c90fb86a4c026e16348ff9 2026-03-31 11:40:54,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:40:54,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90fb86a4c026e16348ff9 HTTP/1.1" 200 None 2026-03-31 11:40:54,199 - lava_callback - DEBUG - Processing node 69c90fb86a4c026e16348ff9: baseline-arm64-clabbe 2026-03-31 11:40:54,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90fb86a4c026e16348ff9 2026-03-31 11:40:54,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:40:54,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90fb86a4c026e16348ff9 HTTP/1.1" 200 None 2026-03-31 11:40:54,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:40:54,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:40:54,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:40:54,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90fb86a4c026e16348ff9 HTTP/1.1" 200 None 2026-03-31 11:40:54,251 - lava_callback - ERROR - Error processing callback for node 69c90fb86a4c026e16348ff9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51628 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:42:14,200 - lava_callback - INFO - Received callback for node 69c910026a4c026e16349003 INFO: 172.19.0.1:51518 - "POST /node/69c910026a4c026e16349003 HTTP/1.0" 202 Accepted 2026-03-31 11:42:14,209 - lava_callback - INFO - Starting async job submit for node 69c910026a4c026e16349003 2026-03-31 11:42:14,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:42:14,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910026a4c026e16349003 HTTP/1.1" 200 None 2026-03-31 11:42:14,225 - lava_callback - DEBUG - Processing node 69c910026a4c026e16349003: baseline-arm64-clabbe 2026-03-31 11:42:14,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c910026a4c026e16349003 2026-03-31 11:42:14,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:42:14,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910026a4c026e16349003 HTTP/1.1" 200 None 2026-03-31 11:42:14,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:42:14,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:42:14,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:42:14,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c910026a4c026e16349003 HTTP/1.1" 200 None 2026-03-31 11:42:14,300 - lava_callback - ERROR - Error processing callback for node 69c910026a4c026e16349003: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52650 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:43:14,200 - lava_callback - INFO - Received callback for node 69c9104d6a4c026e16349010 INFO: 172.19.0.1:47674 - "POST /node/69c9104d6a4c026e16349010 HTTP/1.0" 202 Accepted 2026-03-31 11:43:14,208 - lava_callback - INFO - Starting async job submit for node 69c9104d6a4c026e16349010 2026-03-31 11:43:14,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:43:14,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9104d6a4c026e16349010 HTTP/1.1" 200 None 2026-03-31 11:43:14,225 - lava_callback - DEBUG - Processing node 69c9104d6a4c026e16349010: baseline-arm64-clabbe 2026-03-31 11:43:14,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9104d6a4c026e16349010 2026-03-31 11:43:14,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:43:14,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9104d6a4c026e16349010 HTTP/1.1" 200 None 2026-03-31 11:43:14,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:43:14,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:43:14,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:43:14,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9104d6a4c026e16349010 HTTP/1.1" 200 None 2026-03-31 11:43:14,280 - lava_callback - ERROR - Error processing callback for node 69c9104d6a4c026e16349010: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36052 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:44:34,255 - lava_callback - INFO - Received callback for node 69c910986a4c026e1634901b INFO: 172.19.0.1:33056 - "POST /node/69c910986a4c026e1634901b HTTP/1.0" 202 Accepted 2026-03-31 11:44:34,260 - lava_callback - INFO - Starting async job submit for node 69c910986a4c026e1634901b 2026-03-31 11:44:34,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:44:34,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910986a4c026e1634901b HTTP/1.1" 200 None 2026-03-31 11:44:34,278 - lava_callback - DEBUG - Processing node 69c910986a4c026e1634901b: baseline-arm64-clabbe 2026-03-31 11:44:34,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c910986a4c026e1634901b 2026-03-31 11:44:34,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:44:34,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910986a4c026e1634901b HTTP/1.1" 200 None 2026-03-31 11:44:34,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:44:34,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:44:34,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:44:34,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c910986a4c026e1634901b HTTP/1.1" 200 None 2026-03-31 11:44:34,344 - lava_callback - ERROR - Error processing callback for node 69c910986a4c026e1634901b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37616 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:45:54,322 - lava_callback - INFO - Received callback for node 69c910e46a4c026e1634902a INFO: 172.19.0.1:55956 - "POST /node/69c910e46a4c026e1634902a HTTP/1.0" 202 Accepted 2026-03-31 11:45:54,328 - lava_callback - INFO - Starting async job submit for node 69c910e46a4c026e1634902a 2026-03-31 11:45:54,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:45:54,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910e46a4c026e1634902a HTTP/1.1" 200 None 2026-03-31 11:45:54,343 - lava_callback - DEBUG - Processing node 69c910e46a4c026e1634902a: baseline-arm64-clabbe 2026-03-31 11:45:54,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c910e46a4c026e1634902a 2026-03-31 11:45:54,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:45:54,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910e46a4c026e1634902a HTTP/1.1" 200 None 2026-03-31 11:45:54,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:45:54,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:45:54,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:45:54,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c910e46a4c026e1634902a HTTP/1.1" 200 None 2026-03-31 11:45:54,408 - lava_callback - ERROR - Error processing callback for node 69c910e46a4c026e1634902a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34192 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:47:14,409 - lava_callback - INFO - Received callback for node 69c911306a4c026e16349034 INFO: 172.19.0.1:49590 - "POST /node/69c911306a4c026e16349034 HTTP/1.0" 202 Accepted 2026-03-31 11:47:14,415 - lava_callback - INFO - Starting async job submit for node 69c911306a4c026e16349034 2026-03-31 11:47:14,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:47:14,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c911306a4c026e16349034 HTTP/1.1" 200 None 2026-03-31 11:47:14,431 - lava_callback - DEBUG - Processing node 69c911306a4c026e16349034: baseline-arm64-clabbe 2026-03-31 11:47:14,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c911306a4c026e16349034 2026-03-31 11:47:14,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:47:14,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c911306a4c026e16349034 HTTP/1.1" 200 None 2026-03-31 11:47:14,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:47:14,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:47:14,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:47:14,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c911306a4c026e16349034 HTTP/1.1" 200 None 2026-03-31 11:47:14,491 - lava_callback - ERROR - Error processing callback for node 69c911306a4c026e16349034: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42716 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:02:28,469 - lava_callback - INFO - Received callback for node 69cb86a72a64fa9aff936f0c INFO: 172.19.0.1:42720 - "POST /node/69cb86a72a64fa9aff936f0c HTTP/1.0" 202 Accepted 2026-03-31 12:02:28,475 - lava_callback - INFO - Starting async job submit for node 69cb86a72a64fa9aff936f0c 2026-03-31 12:02:28,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:02:28,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a72a64fa9aff936f0c HTTP/1.1" 200 None 2026-03-31 12:02:28,530 - lava_callback - DEBUG - Processing node 69cb86a72a64fa9aff936f0c: baseline-x86 2026-03-31 12:02:28,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 12:02:28,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 12:02:28,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a72a64fa9aff936f0c/log.txt.gz 2026-03-31 12:02:29,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 12:02:29,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 12:02:29,377 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a72a64fa9aff936f0c/lava-logs.yaml 2026-03-31 12:02:29,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 12:02:29,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 12:02:29,534 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a72a64fa9aff936f0c/lava_callback.json.gz 2026-03-31 12:02:29,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:02:29,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a72a64fa9aff936f0c HTTP/1.1" 200 None 2026-03-31 12:02:29,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:02:29,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83602a64fa9aff936925 HTTP/1.1" 200 None 2026-03-31 12:02:29,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:02:29,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb86a72a64fa9aff936f0c HTTP/1.1" 200 None 2026-03-31 12:02:29,616 - lava_callback - INFO - Completed processing callback for node 69cb86a72a64fa9aff936f0c INFO: 172.19.0.1:34026 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:02:39,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:02:39,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:60410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48604 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59588 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:36:39,221 - lava_callback - INFO - Received callback for node 69c91cce6a4c026e1634906b INFO: 172.19.0.1:59590 - "POST /node/69c91cce6a4c026e1634906b HTTP/1.0" 202 Accepted 2026-03-31 12:36:39,227 - lava_callback - INFO - Starting async job submit for node 69c91cce6a4c026e1634906b 2026-03-31 12:36:39,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:36:39,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91cce6a4c026e1634906b HTTP/1.1" 200 None 2026-03-31 12:36:39,245 - lava_callback - DEBUG - Processing node 69c91cce6a4c026e1634906b: baseline-arm-baylibre 2026-03-31 12:36:39,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91cce6a4c026e1634906b 2026-03-31 12:36:39,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:36:39,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91cce6a4c026e1634906b HTTP/1.1" 200 None 2026-03-31 12:36:39,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:36:39,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:36:39,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:36:39,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91cce6a4c026e1634906b HTTP/1.1" 200 None 2026-03-31 12:36:39,325 - lava_callback - ERROR - Error processing callback for node 69c91cce6a4c026e1634906b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36470 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:37:19,149 - lava_callback - INFO - Received callback for node 69c91cf36a4c026e1634907a INFO: 172.19.0.1:36482 - "POST /node/69c91cf36a4c026e1634907a HTTP/1.0" 202 Accepted 2026-03-31 12:37:19,155 - lava_callback - INFO - Starting async job submit for node 69c91cf36a4c026e1634907a 2026-03-31 12:37:19,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:19,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91cf36a4c026e1634907a HTTP/1.1" 200 None 2026-03-31 12:37:19,171 - lava_callback - DEBUG - Processing node 69c91cf36a4c026e1634907a: baseline-arm-baylibre 2026-03-31 12:37:19,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91cf36a4c026e1634907a 2026-03-31 12:37:19,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:19,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91cf36a4c026e1634907a HTTP/1.1" 200 None 2026-03-31 12:37:19,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:19,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:37:19,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:19,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91cf36a4c026e1634907a HTTP/1.1" 200 None 2026-03-31 12:37:19,243 - lava_callback - ERROR - Error processing callback for node 69c91cf36a4c026e1634907a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53386 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:37:59,265 - lava_callback - INFO - Received callback for node 69c91d176a4c026e16349087 INFO: 172.19.0.1:53390 - "POST /node/69c91d176a4c026e16349087 HTTP/1.0" 202 Accepted 2026-03-31 12:37:59,270 - lava_callback - INFO - Starting async job submit for node 69c91d176a4c026e16349087 2026-03-31 12:37:59,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:59,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d176a4c026e16349087 HTTP/1.1" 200 None 2026-03-31 12:37:59,287 - lava_callback - DEBUG - Processing node 69c91d176a4c026e16349087: baseline-arm-baylibre 2026-03-31 12:37:59,288 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91d176a4c026e16349087 2026-03-31 12:37:59,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:59,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d176a4c026e16349087 HTTP/1.1" 200 None 2026-03-31 12:37:59,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:59,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:37:59,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:59,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91d176a4c026e16349087 HTTP/1.1" 200 None 2026-03-31 12:37:59,351 - lava_callback - ERROR - Error processing callback for node 69c91d176a4c026e16349087: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51062 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:38:39,245 - lava_callback - INFO - Received callback for node 69c91d3c6a4c026e16349090 INFO: 172.19.0.1:51074 - "POST /node/69c91d3c6a4c026e16349090 HTTP/1.0" 202 Accepted 2026-03-31 12:38:39,254 - lava_callback - INFO - Starting async job submit for node 69c91d3c6a4c026e16349090 2026-03-31 12:38:39,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:38:39,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d3c6a4c026e16349090 HTTP/1.1" 200 None 2026-03-31 12:38:39,273 - lava_callback - DEBUG - Processing node 69c91d3c6a4c026e16349090: baseline-arm-baylibre 2026-03-31 12:38:39,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91d3c6a4c026e16349090 2026-03-31 12:38:39,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:38:39,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d3c6a4c026e16349090 HTTP/1.1" 200 None 2026-03-31 12:38:39,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:38:39,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:38:39,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:38:39,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91d3c6a4c026e16349090 HTTP/1.1" 200 None 2026-03-31 12:38:39,334 - lava_callback - ERROR - Error processing callback for node 69c91d3c6a4c026e16349090: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38820 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:39:19,305 - lava_callback - INFO - Received callback for node 69c91d646a4c026e1634909b INFO: 172.19.0.1:38830 - "POST /node/69c91d646a4c026e1634909b HTTP/1.0" 202 Accepted 2026-03-31 12:39:19,310 - lava_callback - INFO - Starting async job submit for node 69c91d646a4c026e1634909b 2026-03-31 12:39:19,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:19,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d646a4c026e1634909b HTTP/1.1" 200 None 2026-03-31 12:39:19,326 - lava_callback - DEBUG - Processing node 69c91d646a4c026e1634909b: baseline-arm-baylibre 2026-03-31 12:39:19,326 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91d646a4c026e1634909b 2026-03-31 12:39:19,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:19,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d646a4c026e1634909b HTTP/1.1" 200 None 2026-03-31 12:39:19,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:19,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:39:19,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:19,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91d646a4c026e1634909b HTTP/1.1" 200 None 2026-03-31 12:39:19,386 - lava_callback - ERROR - Error processing callback for node 69c91d646a4c026e1634909b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33012 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:39:39,305 - lava_callback - INFO - Received callback for node 69c91d886a4c026e163490ad INFO: 172.19.0.1:33020 - "POST /node/69c91d886a4c026e163490ad HTTP/1.0" 202 Accepted 2026-03-31 12:39:39,311 - lava_callback - INFO - Starting async job submit for node 69c91d886a4c026e163490ad 2026-03-31 12:39:39,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:39,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d886a4c026e163490ad HTTP/1.1" 200 None 2026-03-31 12:39:39,327 - lava_callback - DEBUG - Processing node 69c91d886a4c026e163490ad: baseline-arm-baylibre 2026-03-31 12:39:39,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91d886a4c026e163490ad 2026-03-31 12:39:39,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:39,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d886a4c026e163490ad HTTP/1.1" 200 None 2026-03-31 12:39:39,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:39,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:39:39,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:39,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91d886a4c026e163490ad HTTP/1.1" 200 None 2026-03-31 12:39:39,410 - lava_callback - ERROR - Error processing callback for node 69c91d886a4c026e163490ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60332 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:40:19,322 - lava_callback - INFO - Received callback for node 69c91dad6a4c026e163490b4 INFO: 172.19.0.1:60338 - "POST /node/69c91dad6a4c026e163490b4 HTTP/1.0" 202 Accepted 2026-03-31 12:40:19,331 - lava_callback - INFO - Starting async job submit for node 69c91dad6a4c026e163490b4 2026-03-31 12:40:19,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:19,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91dad6a4c026e163490b4 HTTP/1.1" 200 None 2026-03-31 12:40:19,353 - lava_callback - DEBUG - Processing node 69c91dad6a4c026e163490b4: baseline-arm-baylibre 2026-03-31 12:40:19,353 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91dad6a4c026e163490b4 2026-03-31 12:40:19,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:19,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91dad6a4c026e163490b4 HTTP/1.1" 200 None 2026-03-31 12:40:19,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:19,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:40:19,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:19,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91dad6a4c026e163490b4 HTTP/1.1" 200 None 2026-03-31 12:40:19,468 - lava_callback - ERROR - Error processing callback for node 69c91dad6a4c026e163490b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50104 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:40:59,390 - lava_callback - INFO - Received callback for node 69c91dd36a4c026e163490c3 INFO: 172.19.0.1:50114 - "POST /node/69c91dd36a4c026e163490c3 HTTP/1.0" 202 Accepted 2026-03-31 12:40:59,395 - lava_callback - INFO - Starting async job submit for node 69c91dd36a4c026e163490c3 2026-03-31 12:40:59,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:59,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91dd36a4c026e163490c3 HTTP/1.1" 200 None 2026-03-31 12:40:59,412 - lava_callback - DEBUG - Processing node 69c91dd36a4c026e163490c3: baseline-arm-baylibre 2026-03-31 12:40:59,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91dd36a4c026e163490c3 2026-03-31 12:40:59,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:59,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91dd36a4c026e163490c3 HTTP/1.1" 200 None 2026-03-31 12:40:59,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:59,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:40:59,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:59,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91dd36a4c026e163490c3 HTTP/1.1" 200 None 2026-03-31 12:40:59,479 - lava_callback - ERROR - Error processing callback for node 69c91dd36a4c026e163490c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44688 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:41:36,961 - lava_callback - INFO - Received callback for node 69c91df96a4c026e163490d0 INFO: 172.19.0.1:44704 - "POST /node/69c91df96a4c026e163490d0 HTTP/1.0" 202 Accepted 2026-03-31 12:41:36,970 - lava_callback - INFO - Starting async job submit for node 69c91df96a4c026e163490d0 2026-03-31 12:41:36,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:41:36,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91df96a4c026e163490d0 HTTP/1.1" 200 None 2026-03-31 12:41:36,988 - lava_callback - DEBUG - Processing node 69c91df96a4c026e163490d0: baseline-arm-clabbe 2026-03-31 12:41:36,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91df96a4c026e163490d0 2026-03-31 12:41:36,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:41:37,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91df96a4c026e163490d0 HTTP/1.1" 200 None 2026-03-31 12:41:37,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:41:37,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:41:37,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:41:37,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91df96a4c026e163490d0 HTTP/1.1" 200 None 2026-03-31 12:41:37,046 - lava_callback - ERROR - Error processing callback for node 69c91df96a4c026e163490d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46570 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:42:19,548 - lava_callback - INFO - Received callback for node 69c91e1c6a4c026e163490dc INFO: 172.19.0.1:46582 - "POST /node/69c91e1c6a4c026e163490dc HTTP/1.0" 202 Accepted 2026-03-31 12:42:19,554 - lava_callback - INFO - Starting async job submit for node 69c91e1c6a4c026e163490dc 2026-03-31 12:42:19,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:19,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e1c6a4c026e163490dc HTTP/1.1" 200 None 2026-03-31 12:42:19,571 - lava_callback - DEBUG - Processing node 69c91e1c6a4c026e163490dc: baseline-arm-baylibre 2026-03-31 12:42:19,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91e1c6a4c026e163490dc 2026-03-31 12:42:19,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:19,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e1c6a4c026e163490dc HTTP/1.1" 200 None 2026-03-31 12:42:19,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:19,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:42:19,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:19,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91e1c6a4c026e163490dc HTTP/1.1" 200 None 2026-03-31 12:42:19,637 - lava_callback - ERROR - Error processing callback for node 69c91e1c6a4c026e163490dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43344 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:42:59,464 - lava_callback - INFO - Received callback for node 69c91e406a4c026e163490ed INFO: 172.19.0.1:43346 - "POST /node/69c91e406a4c026e163490ed HTTP/1.0" 202 Accepted 2026-03-31 12:42:59,470 - lava_callback - INFO - Starting async job submit for node 69c91e406a4c026e163490ed 2026-03-31 12:42:59,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:59,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e406a4c026e163490ed HTTP/1.1" 200 None 2026-03-31 12:42:59,486 - lava_callback - DEBUG - Processing node 69c91e406a4c026e163490ed: baseline-arm-baylibre 2026-03-31 12:42:59,486 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91e406a4c026e163490ed 2026-03-31 12:42:59,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:59,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e406a4c026e163490ed HTTP/1.1" 200 None 2026-03-31 12:42:59,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:59,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:42:59,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:59,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91e406a4c026e163490ed HTTP/1.1" 200 None 2026-03-31 12:42:59,537 - lava_callback - ERROR - Error processing callback for node 69c91e406a4c026e163490ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55140 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:43:19,494 - lava_callback - INFO - Received callback for node 69c91e646a4c026e163490f4 INFO: 172.19.0.1:55156 - "POST /node/69c91e646a4c026e163490f4 HTTP/1.0" 202 Accepted 2026-03-31 12:43:19,500 - lava_callback - INFO - Starting async job submit for node 69c91e646a4c026e163490f4 2026-03-31 12:43:19,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:19,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e646a4c026e163490f4 HTTP/1.1" 200 None 2026-03-31 12:43:19,538 - lava_callback - DEBUG - Processing node 69c91e646a4c026e163490f4: baseline-arm-baylibre 2026-03-31 12:43:19,539 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91e646a4c026e163490f4 2026-03-31 12:43:19,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:19,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e646a4c026e163490f4 HTTP/1.1" 200 None 2026-03-31 12:43:19,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:19,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:43:19,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:19,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91e646a4c026e163490f4 HTTP/1.1" 200 None 2026-03-31 12:43:19,600 - lava_callback - ERROR - Error processing callback for node 69c91e646a4c026e163490f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42336 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:43:59,560 - lava_callback - INFO - Received callback for node 69c91e886a4c026e16349106 INFO: 172.19.0.1:42352 - "POST /node/69c91e886a4c026e16349106 HTTP/1.0" 202 Accepted 2026-03-31 12:43:59,566 - lava_callback - INFO - Starting async job submit for node 69c91e886a4c026e16349106 2026-03-31 12:43:59,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:59,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e886a4c026e16349106 HTTP/1.1" 200 None 2026-03-31 12:43:59,583 - lava_callback - DEBUG - Processing node 69c91e886a4c026e16349106: baseline-arm-baylibre 2026-03-31 12:43:59,583 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91e886a4c026e16349106 2026-03-31 12:43:59,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:59,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e886a4c026e16349106 HTTP/1.1" 200 None 2026-03-31 12:43:59,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:59,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:43:59,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:59,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91e886a4c026e16349106 HTTP/1.1" 200 None 2026-03-31 12:43:59,662 - lava_callback - ERROR - Error processing callback for node 69c91e886a4c026e16349106: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48734 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:44:37,132 - lava_callback - INFO - Received callback for node 69c91eae6a4c026e16349114 INFO: 172.19.0.1:48740 - "POST /node/69c91eae6a4c026e16349114 HTTP/1.0" 202 Accepted 2026-03-31 12:44:37,137 - lava_callback - INFO - Starting async job submit for node 69c91eae6a4c026e16349114 2026-03-31 12:44:37,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:44:37,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91eae6a4c026e16349114 HTTP/1.1" 200 None 2026-03-31 12:44:37,177 - lava_callback - DEBUG - Processing node 69c91eae6a4c026e16349114: baseline-arm-clabbe 2026-03-31 12:44:37,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91eae6a4c026e16349114 2026-03-31 12:44:37,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:44:37,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91eae6a4c026e16349114 HTTP/1.1" 200 None 2026-03-31 12:44:37,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:44:37,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:44:37,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:44:37,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91eae6a4c026e16349114 HTTP/1.1" 200 None 2026-03-31 12:44:37,408 - lava_callback - ERROR - Error processing callback for node 69c91eae6a4c026e16349114: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55364 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:45:19,627 - lava_callback - INFO - Received callback for node 69c91ed16a4c026e16349120 INFO: 172.19.0.1:55372 - "POST /node/69c91ed16a4c026e16349120 HTTP/1.0" 202 Accepted 2026-03-31 12:45:19,632 - lava_callback - INFO - Starting async job submit for node 69c91ed16a4c026e16349120 2026-03-31 12:45:19,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:45:19,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91ed16a4c026e16349120 HTTP/1.1" 200 None 2026-03-31 12:45:19,649 - lava_callback - DEBUG - Processing node 69c91ed16a4c026e16349120: baseline-arm-baylibre 2026-03-31 12:45:19,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91ed16a4c026e16349120 2026-03-31 12:45:19,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:45:19,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91ed16a4c026e16349120 HTTP/1.1" 200 None 2026-03-31 12:45:19,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:45:19,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:45:19,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:45:19,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91ed16a4c026e16349120 HTTP/1.1" 200 None 2026-03-31 12:45:19,725 - lava_callback - ERROR - Error processing callback for node 69c91ed16a4c026e16349120: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52112 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:47:16,937 - lava_callback - INFO - Received callback for node 69c91f426a4c026e16349149 INFO: 172.19.0.1:52128 - "POST /node/69c91f426a4c026e16349149 HTTP/1.0" 202 Accepted 2026-03-31 12:47:16,942 - lava_callback - INFO - Starting async job submit for node 69c91f426a4c026e16349149 2026-03-31 12:47:16,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:16,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91f426a4c026e16349149 HTTP/1.1" 200 None 2026-03-31 12:47:16,959 - lava_callback - DEBUG - Processing node 69c91f426a4c026e16349149: baseline-arm-clabbe 2026-03-31 12:47:16,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91f426a4c026e16349149 2026-03-31 12:47:16,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:16,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91f426a4c026e16349149 HTTP/1.1" 200 None 2026-03-31 12:47:16,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:16,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:47:16,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:17,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91f426a4c026e16349149 HTTP/1.1" 200 None 2026-03-31 12:47:17,013 - lava_callback - ERROR - Error processing callback for node 69c91f426a4c026e16349149: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46438 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:47:39,955 - lava_callback - INFO - Received callback for node 69c91f646a4c026e16349159 INFO: 172.19.0.1:50714 - "POST /node/69c91f646a4c026e16349159 HTTP/1.0" 202 Accepted 2026-03-31 12:47:39,960 - lava_callback - INFO - Starting async job submit for node 69c91f646a4c026e16349159 2026-03-31 12:47:39,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:39,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91f646a4c026e16349159 HTTP/1.1" 200 None 2026-03-31 12:47:39,978 - lava_callback - DEBUG - Processing node 69c91f646a4c026e16349159: baseline-arm-baylibre 2026-03-31 12:47:39,978 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91f646a4c026e16349159 2026-03-31 12:47:39,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:39,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91f646a4c026e16349159 HTTP/1.1" 200 None 2026-03-31 12:47:39,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:40,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:47:40,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:40,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91f646a4c026e16349159 HTTP/1.1" 200 None 2026-03-31 12:47:40,052 - lava_callback - ERROR - Error processing callback for node 69c91f646a4c026e16349159: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60886 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:48:59,843 - lava_callback - INFO - Received callback for node 69c91fac6a4c026e16349171 INFO: 172.19.0.1:34344 - "POST /node/69c91fac6a4c026e16349171 HTTP/1.0" 202 Accepted 2026-03-31 12:48:59,848 - lava_callback - INFO - Starting async job submit for node 69c91fac6a4c026e16349171 2026-03-31 12:48:59,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:48:59,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91fac6a4c026e16349171 HTTP/1.1" 200 None 2026-03-31 12:48:59,864 - lava_callback - DEBUG - Processing node 69c91fac6a4c026e16349171: baseline-arm-baylibre 2026-03-31 12:48:59,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91fac6a4c026e16349171 2026-03-31 12:48:59,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:48:59,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91fac6a4c026e16349171 HTTP/1.1" 200 None 2026-03-31 12:48:59,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:48:59,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:48:59,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:48:59,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91fac6a4c026e16349171 HTTP/1.1" 200 None 2026-03-31 12:48:59,924 - lava_callback - ERROR - Error processing callback for node 69c91fac6a4c026e16349171: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41204 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:49:36,968 - lava_callback - INFO - Received callback for node 69c91fd26a4c026e1634917c INFO: 172.19.0.1:41214 - "POST /node/69c91fd26a4c026e1634917c HTTP/1.0" 202 Accepted 2026-03-31 12:49:36,974 - lava_callback - INFO - Starting async job submit for node 69c91fd26a4c026e1634917c 2026-03-31 12:49:36,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:49:36,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91fd26a4c026e1634917c HTTP/1.1" 200 None 2026-03-31 12:49:36,991 - lava_callback - DEBUG - Processing node 69c91fd26a4c026e1634917c: baseline-arm-clabbe 2026-03-31 12:49:36,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91fd26a4c026e1634917c 2026-03-31 12:49:36,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:49:37,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91fd26a4c026e1634917c HTTP/1.1" 200 None 2026-03-31 12:49:37,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:49:37,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:49:37,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:49:37,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91fd26a4c026e1634917c HTTP/1.1" 200 None 2026-03-31 12:49:37,070 - lava_callback - ERROR - Error processing callback for node 69c91fd26a4c026e1634917c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:32838 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:50:16,915 - lava_callback - INFO - Received callback for node 69c91ff66a4c026e1634918e INFO: 172.19.0.1:32848 - "POST /node/69c91ff66a4c026e1634918e HTTP/1.0" 202 Accepted 2026-03-31 12:50:16,920 - lava_callback - INFO - Starting async job submit for node 69c91ff66a4c026e1634918e 2026-03-31 12:50:16,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:50:16,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91ff66a4c026e1634918e HTTP/1.1" 200 None 2026-03-31 12:50:16,936 - lava_callback - DEBUG - Processing node 69c91ff66a4c026e1634918e: baseline-arm-clabbe 2026-03-31 12:50:16,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91ff66a4c026e1634918e 2026-03-31 12:50:16,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:50:16,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91ff66a4c026e1634918e HTTP/1.1" 200 None 2026-03-31 12:50:16,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:50:16,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:50:16,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:50:16,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91ff66a4c026e1634918e HTTP/1.1" 200 None 2026-03-31 12:50:16,989 - lava_callback - ERROR - Error processing callback for node 69c91ff66a4c026e1634918e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37464 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:51:57,264 - lava_callback - INFO - Received callback for node 69c920606a4c026e163491bc INFO: 172.19.0.1:37480 - "POST /node/69c920606a4c026e163491bc HTTP/1.0" 202 Accepted 2026-03-31 12:51:57,269 - lava_callback - INFO - Starting async job submit for node 69c920606a4c026e163491bc 2026-03-31 12:51:57,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:51:57,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920606a4c026e163491bc HTTP/1.1" 200 None 2026-03-31 12:51:57,285 - lava_callback - DEBUG - Processing node 69c920606a4c026e163491bc: baseline-arm-clabbe 2026-03-31 12:51:57,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c920606a4c026e163491bc 2026-03-31 12:51:57,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:51:57,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920606a4c026e163491bc HTTP/1.1" 200 None 2026-03-31 12:51:57,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:51:57,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:51:57,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:51:57,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c920606a4c026e163491bc HTTP/1.1" 200 None 2026-03-31 12:51:57,366 - lava_callback - ERROR - Error processing callback for node 69c920606a4c026e163491bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59328 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:52:37,132 - lava_callback - INFO - Received callback for node 69c920866a4c026e163491c7 INFO: 172.19.0.1:59336 - "POST /node/69c920866a4c026e163491c7 HTTP/1.0" 202 Accepted 2026-03-31 12:52:37,138 - lava_callback - INFO - Starting async job submit for node 69c920866a4c026e163491c7 2026-03-31 12:52:37,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:52:37,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920866a4c026e163491c7 HTTP/1.1" 200 None 2026-03-31 12:52:37,175 - lava_callback - DEBUG - Processing node 69c920866a4c026e163491c7: baseline-arm-clabbe 2026-03-31 12:52:37,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c920866a4c026e163491c7 2026-03-31 12:52:37,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:52:37,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920866a4c026e163491c7 HTTP/1.1" 200 None 2026-03-31 12:52:37,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:52:37,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:52:37,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:52:37,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c920866a4c026e163491c7 HTTP/1.1" 200 None 2026-03-31 12:52:37,226 - lava_callback - ERROR - Error processing callback for node 69c920866a4c026e163491c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34714 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:53:17,182 - lava_callback - INFO - Received callback for node 69c920ab6a4c026e163491da INFO: 172.19.0.1:34726 - "POST /node/69c920ab6a4c026e163491da HTTP/1.0" 202 Accepted 2026-03-31 12:53:17,188 - lava_callback - INFO - Starting async job submit for node 69c920ab6a4c026e163491da 2026-03-31 12:53:17,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:17,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920ab6a4c026e163491da HTTP/1.1" 200 None 2026-03-31 12:53:17,204 - lava_callback - DEBUG - Processing node 69c920ab6a4c026e163491da: baseline-arm-clabbe 2026-03-31 12:53:17,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c920ab6a4c026e163491da 2026-03-31 12:53:17,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:17,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920ab6a4c026e163491da HTTP/1.1" 200 None 2026-03-31 12:53:17,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:17,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:53:17,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:17,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c920ab6a4c026e163491da HTTP/1.1" 200 None 2026-03-31 12:53:17,266 - lava_callback - ERROR - Error processing callback for node 69c920ab6a4c026e163491da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35672 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:53:57,164 - lava_callback - INFO - Received callback for node 69c920d06a4c026e163491eb INFO: 172.19.0.1:35682 - "POST /node/69c920d06a4c026e163491eb HTTP/1.0" 202 Accepted 2026-03-31 12:53:57,170 - lava_callback - INFO - Starting async job submit for node 69c920d06a4c026e163491eb 2026-03-31 12:53:57,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:57,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920d06a4c026e163491eb HTTP/1.1" 200 None 2026-03-31 12:53:57,185 - lava_callback - DEBUG - Processing node 69c920d06a4c026e163491eb: baseline-arm-clabbe 2026-03-31 12:53:57,186 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c920d06a4c026e163491eb 2026-03-31 12:53:57,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:57,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920d06a4c026e163491eb HTTP/1.1" 200 None 2026-03-31 12:53:57,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:57,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:53:57,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:57,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c920d06a4c026e163491eb HTTP/1.1" 200 None 2026-03-31 12:53:57,244 - lava_callback - ERROR - Error processing callback for node 69c920d06a4c026e163491eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54074 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:54:17,220 - lava_callback - INFO - Received callback for node 69c920f56a4c026e163491f7 INFO: 172.19.0.1:54084 - "POST /node/69c920f56a4c026e163491f7 HTTP/1.0" 202 Accepted 2026-03-31 12:54:17,226 - lava_callback - INFO - Starting async job submit for node 69c920f56a4c026e163491f7 2026-03-31 12:54:17,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:17,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920f56a4c026e163491f7 HTTP/1.1" 200 None 2026-03-31 12:54:17,241 - lava_callback - DEBUG - Processing node 69c920f56a4c026e163491f7: baseline-arm-clabbe 2026-03-31 12:54:17,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c920f56a4c026e163491f7 2026-03-31 12:54:17,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:17,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920f56a4c026e163491f7 HTTP/1.1" 200 None 2026-03-31 12:54:17,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:17,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:54:17,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:17,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c920f56a4c026e163491f7 HTTP/1.1" 200 None 2026-03-31 12:54:17,304 - lava_callback - ERROR - Error processing callback for node 69c920f56a4c026e163491f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35650 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:54:57,223 - lava_callback - INFO - Received callback for node 69c921196a4c026e16349203 INFO: 172.19.0.1:35654 - "POST /node/69c921196a4c026e16349203 HTTP/1.0" 202 Accepted 2026-03-31 12:54:57,228 - lava_callback - INFO - Starting async job submit for node 69c921196a4c026e16349203 2026-03-31 12:54:57,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:57,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921196a4c026e16349203 HTTP/1.1" 200 None 2026-03-31 12:54:57,245 - lava_callback - DEBUG - Processing node 69c921196a4c026e16349203: baseline-arm-clabbe 2026-03-31 12:54:57,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921196a4c026e16349203 2026-03-31 12:54:57,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:57,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921196a4c026e16349203 HTTP/1.1" 200 None 2026-03-31 12:54:57,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:57,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:54:57,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:57,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921196a4c026e16349203 HTTP/1.1" 200 None 2026-03-31 12:54:57,306 - lava_callback - ERROR - Error processing callback for node 69c921196a4c026e16349203: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59846 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:55:37,281 - lava_callback - INFO - Received callback for node 69c9213f6a4c026e1634920f INFO: 172.19.0.1:59862 - "POST /node/69c9213f6a4c026e1634920f HTTP/1.0" 202 Accepted 2026-03-31 12:55:37,287 - lava_callback - INFO - Starting async job submit for node 69c9213f6a4c026e1634920f 2026-03-31 12:55:37,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:55:37,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9213f6a4c026e1634920f HTTP/1.1" 200 None 2026-03-31 12:55:37,306 - lava_callback - DEBUG - Processing node 69c9213f6a4c026e1634920f: baseline-arm-clabbe 2026-03-31 12:55:37,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9213f6a4c026e1634920f 2026-03-31 12:55:37,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:55:37,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9213f6a4c026e1634920f HTTP/1.1" 200 None 2026-03-31 12:55:37,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:55:37,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:55:37,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:55:37,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9213f6a4c026e1634920f HTTP/1.1" 200 None 2026-03-31 12:55:37,376 - lava_callback - ERROR - Error processing callback for node 69c9213f6a4c026e1634920f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47300 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:56:17,279 - lava_callback - INFO - Received callback for node 69c921636a4c026e1634921e INFO: 172.19.0.1:47306 - "POST /node/69c921636a4c026e1634921e HTTP/1.0" 202 Accepted 2026-03-31 12:56:17,284 - lava_callback - INFO - Starting async job submit for node 69c921636a4c026e1634921e 2026-03-31 12:56:17,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:17,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921636a4c026e1634921e HTTP/1.1" 200 None 2026-03-31 12:56:17,300 - lava_callback - DEBUG - Processing node 69c921636a4c026e1634921e: baseline-arm-clabbe 2026-03-31 12:56:17,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921636a4c026e1634921e 2026-03-31 12:56:17,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:17,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921636a4c026e1634921e HTTP/1.1" 200 None 2026-03-31 12:56:17,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:17,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:56:17,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:17,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921636a4c026e1634921e HTTP/1.1" 200 None 2026-03-31 12:56:17,374 - lava_callback - ERROR - Error processing callback for node 69c921636a4c026e1634921e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56448 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:56:57,333 - lava_callback - INFO - Received callback for node 69c921886a4c026e1634922f INFO: 172.19.0.1:56458 - "POST /node/69c921886a4c026e1634922f HTTP/1.0" 202 Accepted 2026-03-31 12:56:57,339 - lava_callback - INFO - Starting async job submit for node 69c921886a4c026e1634922f 2026-03-31 12:56:57,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:57,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921886a4c026e1634922f HTTP/1.1" 200 None 2026-03-31 12:56:57,355 - lava_callback - DEBUG - Processing node 69c921886a4c026e1634922f: baseline-arm-clabbe 2026-03-31 12:56:57,355 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921886a4c026e1634922f 2026-03-31 12:56:57,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:57,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921886a4c026e1634922f HTTP/1.1" 200 None 2026-03-31 12:56:57,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:57,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:56:57,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:57,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921886a4c026e1634922f HTTP/1.1" 200 None 2026-03-31 12:56:57,423 - lava_callback - ERROR - Error processing callback for node 69c921886a4c026e1634922f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53510 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:57:37,325 - lava_callback - INFO - Received callback for node 69c921ad6a4c026e16349238 INFO: 172.19.0.1:53522 - "POST /node/69c921ad6a4c026e16349238 HTTP/1.0" 202 Accepted 2026-03-31 12:57:37,330 - lava_callback - INFO - Starting async job submit for node 69c921ad6a4c026e16349238 2026-03-31 12:57:37,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:37,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921ad6a4c026e16349238 HTTP/1.1" 200 None 2026-03-31 12:57:37,349 - lava_callback - DEBUG - Processing node 69c921ad6a4c026e16349238: baseline-arm-clabbe 2026-03-31 12:57:37,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921ad6a4c026e16349238 2026-03-31 12:57:37,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:37,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921ad6a4c026e16349238 HTTP/1.1" 200 None 2026-03-31 12:57:37,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:37,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:57:37,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:37,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921ad6a4c026e16349238 HTTP/1.1" 200 None 2026-03-31 12:57:37,406 - lava_callback - ERROR - Error processing callback for node 69c921ad6a4c026e16349238: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59390 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:57:57,363 - lava_callback - INFO - Received callback for node 69c921d26a4c026e16349245 INFO: 172.19.0.1:59394 - "POST /node/69c921d26a4c026e16349245 HTTP/1.0" 202 Accepted 2026-03-31 12:57:57,368 - lava_callback - INFO - Starting async job submit for node 69c921d26a4c026e16349245 2026-03-31 12:57:57,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:57,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921d26a4c026e16349245 HTTP/1.1" 200 None 2026-03-31 12:57:57,385 - lava_callback - DEBUG - Processing node 69c921d26a4c026e16349245: baseline-arm-clabbe 2026-03-31 12:57:57,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921d26a4c026e16349245 2026-03-31 12:57:57,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:57,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921d26a4c026e16349245 HTTP/1.1" 200 None 2026-03-31 12:57:57,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:57,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:57:57,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:57,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921d26a4c026e16349245 HTTP/1.1" 200 None 2026-03-31 12:57:57,453 - lava_callback - ERROR - Error processing callback for node 69c921d26a4c026e16349245: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38724 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:58:37,342 - lava_callback - INFO - Received callback for node 69c921f86a4c026e16349253 INFO: 172.19.0.1:38736 - "POST /node/69c921f86a4c026e16349253 HTTP/1.0" 202 Accepted 2026-03-31 12:58:37,348 - lava_callback - INFO - Starting async job submit for node 69c921f86a4c026e16349253 2026-03-31 12:58:37,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:58:37,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921f86a4c026e16349253 HTTP/1.1" 200 None 2026-03-31 12:58:37,364 - lava_callback - DEBUG - Processing node 69c921f86a4c026e16349253: baseline-arm-clabbe 2026-03-31 12:58:37,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921f86a4c026e16349253 2026-03-31 12:58:37,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:58:37,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921f86a4c026e16349253 HTTP/1.1" 200 None 2026-03-31 12:58:37,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:58:37,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:58:37,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:58:37,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921f86a4c026e16349253 HTTP/1.1" 200 None 2026-03-31 12:58:37,421 - lava_callback - ERROR - Error processing callback for node 69c921f86a4c026e16349253: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49972 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:59:17,461 - lava_callback - INFO - Received callback for node 69c9221c6a4c026e16349261 INFO: 172.19.0.1:49980 - "POST /node/69c9221c6a4c026e16349261 HTTP/1.0" 202 Accepted 2026-03-31 12:59:17,466 - lava_callback - INFO - Starting async job submit for node 69c9221c6a4c026e16349261 2026-03-31 12:59:17,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:17,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9221c6a4c026e16349261 HTTP/1.1" 200 None 2026-03-31 12:59:17,482 - lava_callback - DEBUG - Processing node 69c9221c6a4c026e16349261: baseline-arm-clabbe 2026-03-31 12:59:17,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9221c6a4c026e16349261 2026-03-31 12:59:17,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:17,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9221c6a4c026e16349261 HTTP/1.1" 200 None 2026-03-31 12:59:17,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:17,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:59:17,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:17,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9221c6a4c026e16349261 HTTP/1.1" 200 None 2026-03-31 12:59:17,538 - lava_callback - ERROR - Error processing callback for node 69c9221c6a4c026e16349261: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34810 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:59:57,442 - lava_callback - INFO - Received callback for node 69c922416a4c026e1634926c INFO: 172.19.0.1:34820 - "POST /node/69c922416a4c026e1634926c HTTP/1.0" 202 Accepted 2026-03-31 12:59:57,448 - lava_callback - INFO - Starting async job submit for node 69c922416a4c026e1634926c 2026-03-31 12:59:57,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:57,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c922416a4c026e1634926c HTTP/1.1" 200 None 2026-03-31 12:59:57,466 - lava_callback - DEBUG - Processing node 69c922416a4c026e1634926c: baseline-arm-clabbe 2026-03-31 12:59:57,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c922416a4c026e1634926c 2026-03-31 12:59:57,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:57,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c922416a4c026e1634926c HTTP/1.1" 200 None 2026-03-31 12:59:57,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:57,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:59:57,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:57,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c922416a4c026e1634926c HTTP/1.1" 200 None 2026-03-31 12:59:57,526 - lava_callback - ERROR - Error processing callback for node 69c922416a4c026e1634926c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53020 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:00:37,535 - lava_callback - INFO - Received callback for node 69c922666a4c026e16349276 INFO: 172.19.0.1:53036 - "POST /node/69c922666a4c026e16349276 HTTP/1.0" 202 Accepted 2026-03-31 13:00:37,540 - lava_callback - INFO - Starting async job submit for node 69c922666a4c026e16349276 2026-03-31 13:00:37,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:00:37,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c922666a4c026e16349276 HTTP/1.1" 200 None 2026-03-31 13:00:37,556 - lava_callback - DEBUG - Processing node 69c922666a4c026e16349276: baseline-arm-clabbe 2026-03-31 13:00:37,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c922666a4c026e16349276 2026-03-31 13:00:37,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:00:37,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c922666a4c026e16349276 HTTP/1.1" 200 None 2026-03-31 13:00:37,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:00:37,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 13:00:37,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:00:37,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c922666a4c026e16349276 HTTP/1.1" 200 None 2026-03-31 13:00:37,613 - lava_callback - ERROR - Error processing callback for node 69c922666a4c026e16349276: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55284 - "GET /node/698d49bf820eff0a56c15c2b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57404 - "GET /node/698a9b49d9a1cfa777a24e88%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50982 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:07:41,010 - lava_callback - INFO - Received callback for node 69c9241b6a4c026e1634929f INFO: 172.19.0.1:43146 - "POST /node/69c9241b6a4c026e1634929f HTTP/1.0" 202 Accepted 2026-03-31 13:07:41,019 - lava_callback - INFO - Starting async job submit for node 69c9241b6a4c026e1634929f 2026-03-31 13:07:41,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:07:41,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9241b6a4c026e1634929f HTTP/1.1" 200 None 2026-03-31 13:07:41,058 - lava_callback - DEBUG - Processing node 69c9241b6a4c026e1634929f: baseline-arm64-baylibre 2026-03-31 13:07:41,059 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9241b6a4c026e1634929f 2026-03-31 13:07:41,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:07:41,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9241b6a4c026e1634929f HTTP/1.1" 200 None 2026-03-31 13:07:41,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:07:41,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:07:41,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:07:41,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9241b6a4c026e1634929f HTTP/1.1" 200 None 2026-03-31 13:07:41,129 - lava_callback - ERROR - Error processing callback for node 69c9241b6a4c026e1634929f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38104 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:09:00,902 - lava_callback - INFO - Received callback for node 69c924686a4c026e163492a7 INFO: 172.19.0.1:56912 - "POST /node/69c924686a4c026e163492a7 HTTP/1.0" 202 Accepted 2026-03-31 13:09:00,908 - lava_callback - INFO - Starting async job submit for node 69c924686a4c026e163492a7 2026-03-31 13:09:00,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:09:00,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c924686a4c026e163492a7 HTTP/1.1" 200 None 2026-03-31 13:09:00,925 - lava_callback - DEBUG - Processing node 69c924686a4c026e163492a7: baseline-arm64-baylibre 2026-03-31 13:09:00,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c924686a4c026e163492a7 2026-03-31 13:09:00,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:09:00,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c924686a4c026e163492a7 HTTP/1.1" 200 None 2026-03-31 13:09:00,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:09:00,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:09:00,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:09:00,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c924686a4c026e163492a7 HTTP/1.1" 200 None 2026-03-31 13:09:00,979 - lava_callback - ERROR - Error processing callback for node 69c924686a4c026e163492a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42134 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:10:20,982 - lava_callback - INFO - Received callback for node 69c924b46a4c026e163492b7 INFO: 172.19.0.1:46112 - "POST /node/69c924b46a4c026e163492b7 HTTP/1.0" 202 Accepted 2026-03-31 13:10:20,988 - lava_callback - INFO - Starting async job submit for node 69c924b46a4c026e163492b7 2026-03-31 13:10:20,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:10:21,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c924b46a4c026e163492b7 HTTP/1.1" 200 None 2026-03-31 13:10:21,018 - lava_callback - DEBUG - Processing node 69c924b46a4c026e163492b7: baseline-arm64-baylibre 2026-03-31 13:10:21,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c924b46a4c026e163492b7 2026-03-31 13:10:21,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:10:21,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c924b46a4c026e163492b7 HTTP/1.1" 200 None 2026-03-31 13:10:21,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:10:21,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:10:21,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:10:21,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c924b46a4c026e163492b7 HTTP/1.1" 200 None 2026-03-31 13:10:21,070 - lava_callback - ERROR - Error processing callback for node 69c924b46a4c026e163492b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32858 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:11:41,131 - lava_callback - INFO - Received callback for node 69c925026a4c026e163492bd INFO: 172.19.0.1:59342 - "POST /node/69c925026a4c026e163492bd HTTP/1.0" 202 Accepted 2026-03-31 13:11:41,136 - lava_callback - INFO - Starting async job submit for node 69c925026a4c026e163492bd 2026-03-31 13:11:41,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:11:41,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c925026a4c026e163492bd HTTP/1.1" 200 None 2026-03-31 13:11:41,152 - lava_callback - DEBUG - Processing node 69c925026a4c026e163492bd: baseline-arm64-baylibre 2026-03-31 13:11:41,152 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c925026a4c026e163492bd 2026-03-31 13:11:41,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:11:41,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c925026a4c026e163492bd HTTP/1.1" 200 None 2026-03-31 13:11:41,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:11:41,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:11:41,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:11:41,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c925026a4c026e163492bd HTTP/1.1" 200 None 2026-03-31 13:11:41,236 - lava_callback - ERROR - Error processing callback for node 69c925026a4c026e163492bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47132 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:13:01,135 - lava_callback - INFO - Received callback for node 69c9254f6a4c026e163492cf INFO: 172.19.0.1:38540 - "POST /node/69c9254f6a4c026e163492cf HTTP/1.0" 202 Accepted 2026-03-31 13:13:01,141 - lava_callback - INFO - Starting async job submit for node 69c9254f6a4c026e163492cf 2026-03-31 13:13:01,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:13:01,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9254f6a4c026e163492cf HTTP/1.1" 200 None 2026-03-31 13:13:01,156 - lava_callback - DEBUG - Processing node 69c9254f6a4c026e163492cf: baseline-arm64-baylibre 2026-03-31 13:13:01,157 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9254f6a4c026e163492cf 2026-03-31 13:13:01,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:13:01,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9254f6a4c026e163492cf HTTP/1.1" 200 None 2026-03-31 13:13:01,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:13:01,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:13:01,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:13:01,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9254f6a4c026e163492cf HTTP/1.1" 200 None 2026-03-31 13:13:01,217 - lava_callback - ERROR - Error processing callback for node 69c9254f6a4c026e163492cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34010 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:14:21,194 - lava_callback - INFO - Received callback for node 69c9259c6a4c026e163492d9 INFO: 172.19.0.1:33140 - "POST /node/69c9259c6a4c026e163492d9 HTTP/1.0" 202 Accepted 2026-03-31 13:14:21,199 - lava_callback - INFO - Starting async job submit for node 69c9259c6a4c026e163492d9 2026-03-31 13:14:21,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:14:21,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9259c6a4c026e163492d9 HTTP/1.1" 200 None 2026-03-31 13:14:21,216 - lava_callback - DEBUG - Processing node 69c9259c6a4c026e163492d9: baseline-arm64-baylibre 2026-03-31 13:14:21,217 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9259c6a4c026e163492d9 2026-03-31 13:14:21,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:14:21,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9259c6a4c026e163492d9 HTTP/1.1" 200 None 2026-03-31 13:14:21,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:14:21,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:14:21,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:14:21,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9259c6a4c026e163492d9 HTTP/1.1" 200 None 2026-03-31 13:14:21,268 - lava_callback - ERROR - Error processing callback for node 69c9259c6a4c026e163492d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52776 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:16:41,298 - lava_callback - INFO - Received callback for node 69c926336a4c026e163492f0 INFO: 172.19.0.1:50650 - "POST /node/69c926336a4c026e163492f0 HTTP/1.0" 202 Accepted 2026-03-31 13:16:41,303 - lava_callback - INFO - Starting async job submit for node 69c926336a4c026e163492f0 2026-03-31 13:16:41,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:16:41,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c926336a4c026e163492f0 HTTP/1.1" 200 None 2026-03-31 13:16:41,319 - lava_callback - DEBUG - Processing node 69c926336a4c026e163492f0: baseline-arm64-baylibre 2026-03-31 13:16:41,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c926336a4c026e163492f0 2026-03-31 13:16:41,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:16:41,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c926336a4c026e163492f0 HTTP/1.1" 200 None 2026-03-31 13:16:41,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:16:41,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:16:41,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:16:41,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c926336a4c026e163492f0 HTTP/1.1" 200 None 2026-03-31 13:16:41,381 - lava_callback - ERROR - Error processing callback for node 69c926336a4c026e163492f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41608 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:18:01,365 - lava_callback - INFO - Received callback for node 69c9267c6a4c026e16349301 INFO: 172.19.0.1:59300 - "POST /node/69c9267c6a4c026e16349301 HTTP/1.0" 202 Accepted 2026-03-31 13:18:01,371 - lava_callback - INFO - Starting async job submit for node 69c9267c6a4c026e16349301 2026-03-31 13:18:01,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:18:01,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9267c6a4c026e16349301 HTTP/1.1" 200 None 2026-03-31 13:18:01,388 - lava_callback - DEBUG - Processing node 69c9267c6a4c026e16349301: baseline-arm64-baylibre 2026-03-31 13:18:01,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9267c6a4c026e16349301 2026-03-31 13:18:01,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:18:01,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9267c6a4c026e16349301 HTTP/1.1" 200 None 2026-03-31 13:18:01,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:18:01,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:18:01,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:18:01,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9267c6a4c026e16349301 HTTP/1.1" 200 None 2026-03-31 13:18:01,468 - lava_callback - ERROR - Error processing callback for node 69c9267c6a4c026e16349301: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53748 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:19:18,345 - lava_callback - INFO - Received callback for node 69c926cf6a4c026e16349313 INFO: 172.19.0.1:53752 - "POST /node/69c926cf6a4c026e16349313 HTTP/1.0" 202 Accepted 2026-03-31 13:19:18,351 - lava_callback - INFO - Starting async job submit for node 69c926cf6a4c026e16349313 2026-03-31 13:19:18,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:19:18,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c926cf6a4c026e16349313 HTTP/1.1" 200 None 2026-03-31 13:19:18,377 - lava_callback - DEBUG - Processing node 69c926cf6a4c026e16349313: baseline-arm64-clabbe 2026-03-31 13:19:18,377 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c926cf6a4c026e16349313 2026-03-31 13:19:18,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:19:18,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c926cf6a4c026e16349313 HTTP/1.1" 200 None 2026-03-31 13:19:18,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:19:18,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:19:18,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:19:18,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c926cf6a4c026e16349313 HTTP/1.1" 200 None 2026-03-31 13:19:18,454 - lava_callback - ERROR - Error processing callback for node 69c926cf6a4c026e16349313: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43494 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:20:38,381 - lava_callback - INFO - Received callback for node 69c9271c6a4c026e1634931e INFO: 172.19.0.1:43502 - "POST /node/69c9271c6a4c026e1634931e HTTP/1.0" 202 Accepted 2026-03-31 13:20:38,386 - lava_callback - INFO - Starting async job submit for node 69c9271c6a4c026e1634931e 2026-03-31 13:20:38,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:20:38,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9271c6a4c026e1634931e HTTP/1.1" 200 None 2026-03-31 13:20:38,414 - lava_callback - DEBUG - Processing node 69c9271c6a4c026e1634931e: baseline-arm64-clabbe 2026-03-31 13:20:38,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9271c6a4c026e1634931e 2026-03-31 13:20:38,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:20:38,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9271c6a4c026e1634931e HTTP/1.1" 200 None 2026-03-31 13:20:38,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:20:38,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:20:38,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:20:38,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9271c6a4c026e1634931e HTTP/1.1" 200 None 2026-03-31 13:20:38,466 - lava_callback - ERROR - Error processing callback for node 69c9271c6a4c026e1634931e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56458 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:21:58,479 - lava_callback - INFO - Received callback for node 69c927696a4c026e16349330 INFO: 172.19.0.1:56460 - "POST /node/69c927696a4c026e16349330 HTTP/1.0" 202 Accepted 2026-03-31 13:21:58,486 - lava_callback - INFO - Starting async job submit for node 69c927696a4c026e16349330 2026-03-31 13:21:58,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:21:58,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c927696a4c026e16349330 HTTP/1.1" 200 None 2026-03-31 13:21:58,505 - lava_callback - DEBUG - Processing node 69c927696a4c026e16349330: baseline-arm64-clabbe 2026-03-31 13:21:58,506 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c927696a4c026e16349330 2026-03-31 13:21:58,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:21:58,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c927696a4c026e16349330 HTTP/1.1" 200 None 2026-03-31 13:21:58,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:21:58,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:21:58,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:21:58,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c927696a4c026e16349330 HTTP/1.1" 200 None 2026-03-31 13:21:58,565 - lava_callback - ERROR - Error processing callback for node 69c927696a4c026e16349330: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38654 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:23:18,485 - lava_callback - INFO - Received callback for node 69c927b66a4c026e1634933b INFO: 172.19.0.1:38668 - "POST /node/69c927b66a4c026e1634933b HTTP/1.0" 202 Accepted 2026-03-31 13:23:18,491 - lava_callback - INFO - Starting async job submit for node 69c927b66a4c026e1634933b 2026-03-31 13:23:18,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:23:18,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c927b66a4c026e1634933b HTTP/1.1" 200 None 2026-03-31 13:23:18,509 - lava_callback - DEBUG - Processing node 69c927b66a4c026e1634933b: baseline-arm64-clabbe 2026-03-31 13:23:18,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c927b66a4c026e1634933b 2026-03-31 13:23:18,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:23:18,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c927b66a4c026e1634933b HTTP/1.1" 200 None 2026-03-31 13:23:18,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:23:18,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:23:18,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:23:18,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c927b66a4c026e1634933b HTTP/1.1" 200 None 2026-03-31 13:23:18,589 - lava_callback - ERROR - Error processing callback for node 69c927b66a4c026e1634933b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56910 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59778 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:24:38,505 - lava_callback - INFO - Received callback for node 69c928036a4c026e1634934d INFO: 172.19.0.1:59784 - "POST /node/69c928036a4c026e1634934d HTTP/1.0" 202 Accepted 2026-03-31 13:24:38,510 - lava_callback - INFO - Starting async job submit for node 69c928036a4c026e1634934d 2026-03-31 13:24:38,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:24:38,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c928036a4c026e1634934d HTTP/1.1" 200 None 2026-03-31 13:24:38,528 - lava_callback - DEBUG - Processing node 69c928036a4c026e1634934d: baseline-arm64-clabbe 2026-03-31 13:24:38,528 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c928036a4c026e1634934d 2026-03-31 13:24:38,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:24:38,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c928036a4c026e1634934d HTTP/1.1" 200 None 2026-03-31 13:24:38,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:24:38,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:24:38,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:24:38,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c928036a4c026e1634934d HTTP/1.1" 200 None 2026-03-31 13:24:38,589 - lava_callback - ERROR - Error processing callback for node 69c928036a4c026e1634934d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53410 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:25:41,898 - lava_callback - INFO - Received callback for node 69c9284d6a4c026e16349357 INFO: 172.19.0.1:57192 - "POST /node/69c9284d6a4c026e16349357 HTTP/1.0" 202 Accepted 2026-03-31 13:25:41,904 - lava_callback - INFO - Starting async job submit for node 69c9284d6a4c026e16349357 2026-03-31 13:25:41,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:25:41,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9284d6a4c026e16349357 HTTP/1.1" 200 None 2026-03-31 13:25:41,920 - lava_callback - DEBUG - Processing node 69c9284d6a4c026e16349357: baseline-arm64-baylibre 2026-03-31 13:25:41,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9284d6a4c026e16349357 2026-03-31 13:25:41,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:25:41,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9284d6a4c026e16349357 HTTP/1.1" 200 None 2026-03-31 13:25:41,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:25:41,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:25:41,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:25:42,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9284d6a4c026e16349357 HTTP/1.1" 200 None 2026-03-31 13:25:42,008 - lava_callback - ERROR - Error processing callback for node 69c9284d6a4c026e16349357: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33866 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:26:58,630 - lava_callback - INFO - Received callback for node 69c9289e6a4c026e16349365 INFO: 172.19.0.1:33880 - "POST /node/69c9289e6a4c026e16349365 HTTP/1.0" 202 Accepted 2026-03-31 13:26:58,635 - lava_callback - INFO - Starting async job submit for node 69c9289e6a4c026e16349365 2026-03-31 13:26:58,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:26:58,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9289e6a4c026e16349365 HTTP/1.1" 200 None 2026-03-31 13:26:58,653 - lava_callback - DEBUG - Processing node 69c9289e6a4c026e16349365: baseline-arm64-clabbe 2026-03-31 13:26:58,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9289e6a4c026e16349365 2026-03-31 13:26:58,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:26:58,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9289e6a4c026e16349365 HTTP/1.1" 200 None 2026-03-31 13:26:58,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:26:58,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:26:58,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:26:58,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9289e6a4c026e16349365 HTTP/1.1" 200 None 2026-03-31 13:26:58,716 - lava_callback - ERROR - Error processing callback for node 69c9289e6a4c026e16349365: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43492 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:28:18,671 - lava_callback - INFO - Received callback for node 69c928ea6a4c026e16349376 INFO: 172.19.0.1:43494 - "POST /node/69c928ea6a4c026e16349376 HTTP/1.0" 202 Accepted 2026-03-31 13:28:18,677 - lava_callback - INFO - Starting async job submit for node 69c928ea6a4c026e16349376 2026-03-31 13:28:18,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:28:18,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c928ea6a4c026e16349376 HTTP/1.1" 200 None 2026-03-31 13:28:18,698 - lava_callback - DEBUG - Processing node 69c928ea6a4c026e16349376: baseline-arm64-clabbe 2026-03-31 13:28:18,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c928ea6a4c026e16349376 2026-03-31 13:28:18,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:28:18,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c928ea6a4c026e16349376 HTTP/1.1" 200 None 2026-03-31 13:28:18,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:28:18,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:28:18,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:28:18,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c928ea6a4c026e16349376 HTTP/1.1" 200 None 2026-03-31 13:28:18,767 - lava_callback - ERROR - Error processing callback for node 69c928ea6a4c026e16349376: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47674 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:29:38,707 - lava_callback - INFO - Received callback for node 69c929366a4c026e16349380 INFO: 172.19.0.1:47682 - "POST /node/69c929366a4c026e16349380 HTTP/1.0" 202 Accepted 2026-03-31 13:29:38,712 - lava_callback - INFO - Starting async job submit for node 69c929366a4c026e16349380 2026-03-31 13:29:38,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:29:38,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929366a4c026e16349380 HTTP/1.1" 200 None 2026-03-31 13:29:38,728 - lava_callback - DEBUG - Processing node 69c929366a4c026e16349380: baseline-arm64-clabbe 2026-03-31 13:29:38,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c929366a4c026e16349380 2026-03-31 13:29:38,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:29:38,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929366a4c026e16349380 HTTP/1.1" 200 None 2026-03-31 13:29:38,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:29:38,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:29:38,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:29:38,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c929366a4c026e16349380 HTTP/1.1" 200 None 2026-03-31 13:29:38,790 - lava_callback - ERROR - Error processing callback for node 69c929366a4c026e16349380: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58860 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:30:58,719 - lava_callback - INFO - Received callback for node 69c929836a4c026e1634938f INFO: 172.19.0.1:58874 - "POST /node/69c929836a4c026e1634938f HTTP/1.0" 202 Accepted 2026-03-31 13:30:58,725 - lava_callback - INFO - Starting async job submit for node 69c929836a4c026e1634938f 2026-03-31 13:30:58,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:30:58,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929836a4c026e1634938f HTTP/1.1" 200 None 2026-03-31 13:30:58,747 - lava_callback - DEBUG - Processing node 69c929836a4c026e1634938f: baseline-arm64-clabbe 2026-03-31 13:30:58,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c929836a4c026e1634938f 2026-03-31 13:30:58,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:30:58,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929836a4c026e1634938f HTTP/1.1" 200 None 2026-03-31 13:30:58,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:30:58,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:30:58,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:30:58,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c929836a4c026e1634938f HTTP/1.1" 200 None 2026-03-31 13:30:58,808 - lava_callback - ERROR - Error processing callback for node 69c929836a4c026e1634938f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45646 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:32:18,820 - lava_callback - INFO - Received callback for node 69c929d16a4c026e16349397 INFO: 172.19.0.1:45660 - "POST /node/69c929d16a4c026e16349397 HTTP/1.0" 202 Accepted 2026-03-31 13:32:18,825 - lava_callback - INFO - Starting async job submit for node 69c929d16a4c026e16349397 2026-03-31 13:32:18,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:32:18,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929d16a4c026e16349397 HTTP/1.1" 200 None 2026-03-31 13:32:18,843 - lava_callback - DEBUG - Processing node 69c929d16a4c026e16349397: baseline-arm64-clabbe 2026-03-31 13:32:18,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c929d16a4c026e16349397 2026-03-31 13:32:18,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:32:18,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929d16a4c026e16349397 HTTP/1.1" 200 None 2026-03-31 13:32:18,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:32:18,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:32:18,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:32:18,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c929d16a4c026e16349397 HTTP/1.1" 200 None 2026-03-31 13:32:18,917 - lava_callback - ERROR - Error processing callback for node 69c929d16a4c026e16349397: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33088 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:33:38,857 - lava_callback - INFO - Received callback for node 69c92a1f6a4c026e163493a6 INFO: 172.19.0.1:33090 - "POST /node/69c92a1f6a4c026e163493a6 HTTP/1.0" 202 Accepted 2026-03-31 13:33:38,862 - lava_callback - INFO - Starting async job submit for node 69c92a1f6a4c026e163493a6 2026-03-31 13:33:38,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:33:38,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92a1f6a4c026e163493a6 HTTP/1.1" 200 None 2026-03-31 13:33:38,879 - lava_callback - DEBUG - Processing node 69c92a1f6a4c026e163493a6: baseline-arm64-clabbe 2026-03-31 13:33:38,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92a1f6a4c026e163493a6 2026-03-31 13:33:38,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:33:38,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92a1f6a4c026e163493a6 HTTP/1.1" 200 None 2026-03-31 13:33:38,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:33:38,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:33:38,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:33:38,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92a1f6a4c026e163493a6 HTTP/1.1" 200 None 2026-03-31 13:33:38,930 - lava_callback - ERROR - Error processing callback for node 69c92a1f6a4c026e163493a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39334 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:34:58,865 - lava_callback - INFO - Received callback for node 69c92a6d6a4c026e163493ae INFO: 172.19.0.1:39338 - "POST /node/69c92a6d6a4c026e163493ae HTTP/1.0" 202 Accepted 2026-03-31 13:34:58,871 - lava_callback - INFO - Starting async job submit for node 69c92a6d6a4c026e163493ae 2026-03-31 13:34:58,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:34:58,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92a6d6a4c026e163493ae HTTP/1.1" 200 None 2026-03-31 13:34:58,898 - lava_callback - DEBUG - Processing node 69c92a6d6a4c026e163493ae: baseline-arm64-clabbe 2026-03-31 13:34:58,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92a6d6a4c026e163493ae 2026-03-31 13:34:58,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:34:58,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92a6d6a4c026e163493ae HTTP/1.1" 200 None 2026-03-31 13:34:58,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:34:58,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:34:58,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:34:58,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92a6d6a4c026e163493ae HTTP/1.1" 200 None 2026-03-31 13:34:58,956 - lava_callback - ERROR - Error processing callback for node 69c92a6d6a4c026e163493ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44828 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:35:58,935 - lava_callback - INFO - Received callback for node 69c92aba6a4c026e163493b8 INFO: 172.19.0.1:44844 - "POST /node/69c92aba6a4c026e163493b8 HTTP/1.0" 202 Accepted 2026-03-31 13:35:58,941 - lava_callback - INFO - Starting async job submit for node 69c92aba6a4c026e163493b8 2026-03-31 13:35:58,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:35:58,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92aba6a4c026e163493b8 HTTP/1.1" 200 None 2026-03-31 13:35:58,967 - lava_callback - DEBUG - Processing node 69c92aba6a4c026e163493b8: baseline-arm64-clabbe 2026-03-31 13:35:58,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92aba6a4c026e163493b8 2026-03-31 13:35:58,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:35:58,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92aba6a4c026e163493b8 HTTP/1.1" 200 None 2026-03-31 13:35:58,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:35:58,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:35:58,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:35:59,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92aba6a4c026e163493b8 HTTP/1.1" 200 None 2026-03-31 13:35:59,026 - lava_callback - ERROR - Error processing callback for node 69c92aba6a4c026e163493b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50812 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:37:18,921 - lava_callback - INFO - Received callback for node 69c92b076a4c026e163493c4 INFO: 172.19.0.1:50828 - "POST /node/69c92b076a4c026e163493c4 HTTP/1.0" 202 Accepted 2026-03-31 13:37:18,931 - lava_callback - INFO - Starting async job submit for node 69c92b076a4c026e163493c4 2026-03-31 13:37:18,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:37:18,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92b076a4c026e163493c4 HTTP/1.1" 200 None 2026-03-31 13:37:18,973 - lava_callback - DEBUG - Processing node 69c92b076a4c026e163493c4: baseline-arm64-clabbe 2026-03-31 13:37:18,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92b076a4c026e163493c4 2026-03-31 13:37:18,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:37:18,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92b076a4c026e163493c4 HTTP/1.1" 200 None 2026-03-31 13:37:18,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:37:19,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:37:19,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:37:19,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92b076a4c026e163493c4 HTTP/1.1" 200 None 2026-03-31 13:37:19,026 - lava_callback - ERROR - Error processing callback for node 69c92b076a4c026e163493c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48876 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:38:39,080 - lava_callback - INFO - Received callback for node 69c92b546a4c026e163493cd INFO: 172.19.0.1:48878 - "POST /node/69c92b546a4c026e163493cd HTTP/1.0" 202 Accepted 2026-03-31 13:38:39,085 - lava_callback - INFO - Starting async job submit for node 69c92b546a4c026e163493cd 2026-03-31 13:38:39,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:38:39,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92b546a4c026e163493cd HTTP/1.1" 200 None 2026-03-31 13:38:39,101 - lava_callback - DEBUG - Processing node 69c92b546a4c026e163493cd: baseline-arm64-clabbe 2026-03-31 13:38:39,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92b546a4c026e163493cd 2026-03-31 13:38:39,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:38:39,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92b546a4c026e163493cd HTTP/1.1" 200 None 2026-03-31 13:38:39,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:38:39,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:38:39,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:38:39,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92b546a4c026e163493cd HTTP/1.1" 200 None 2026-03-31 13:38:39,199 - lava_callback - ERROR - Error processing callback for node 69c92b546a4c026e163493cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59906 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:39:59,122 - lava_callback - INFO - Received callback for node 69c92ba16a4c026e163493da INFO: 172.19.0.1:59908 - "POST /node/69c92ba16a4c026e163493da HTTP/1.0" 202 Accepted 2026-03-31 13:39:59,130 - lava_callback - INFO - Starting async job submit for node 69c92ba16a4c026e163493da 2026-03-31 13:39:59,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:39:59,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ba16a4c026e163493da HTTP/1.1" 200 None 2026-03-31 13:39:59,174 - lava_callback - DEBUG - Processing node 69c92ba16a4c026e163493da: baseline-arm64-clabbe 2026-03-31 13:39:59,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92ba16a4c026e163493da 2026-03-31 13:39:59,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:39:59,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ba16a4c026e163493da HTTP/1.1" 200 None 2026-03-31 13:39:59,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:39:59,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:39:59,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:39:59,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92ba16a4c026e163493da HTTP/1.1" 200 None 2026-03-31 13:39:59,273 - lava_callback - ERROR - Error processing callback for node 69c92ba16a4c026e163493da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35732 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:41:19,191 - lava_callback - INFO - Received callback for node 69c92bee6a4c026e163493e1 INFO: 172.19.0.1:35746 - "POST /node/69c92bee6a4c026e163493e1 HTTP/1.0" 202 Accepted 2026-03-31 13:41:19,197 - lava_callback - INFO - Starting async job submit for node 69c92bee6a4c026e163493e1 2026-03-31 13:41:19,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:41:19,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92bee6a4c026e163493e1 HTTP/1.1" 200 None 2026-03-31 13:41:19,213 - lava_callback - DEBUG - Processing node 69c92bee6a4c026e163493e1: baseline-arm64-clabbe 2026-03-31 13:41:19,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92bee6a4c026e163493e1 2026-03-31 13:41:19,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:41:19,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92bee6a4c026e163493e1 HTTP/1.1" 200 None 2026-03-31 13:41:19,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:41:19,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:41:19,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:41:19,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92bee6a4c026e163493e1 HTTP/1.1" 200 None 2026-03-31 13:41:19,285 - lava_callback - ERROR - Error processing callback for node 69c92bee6a4c026e163493e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53046 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:42:39,257 - lava_callback - INFO - Received callback for node 69c92c3d6a4c026e163493ef INFO: 172.19.0.1:53052 - "POST /node/69c92c3d6a4c026e163493ef HTTP/1.0" 202 Accepted 2026-03-31 13:42:39,263 - lava_callback - INFO - Starting async job submit for node 69c92c3d6a4c026e163493ef 2026-03-31 13:42:39,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:42:39,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92c3d6a4c026e163493ef HTTP/1.1" 200 None 2026-03-31 13:42:39,279 - lava_callback - DEBUG - Processing node 69c92c3d6a4c026e163493ef: baseline-arm64-clabbe 2026-03-31 13:42:39,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92c3d6a4c026e163493ef 2026-03-31 13:42:39,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:42:39,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92c3d6a4c026e163493ef HTTP/1.1" 200 None 2026-03-31 13:42:39,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:42:39,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:42:39,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:42:39,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92c3d6a4c026e163493ef HTTP/1.1" 200 None 2026-03-31 13:42:39,353 - lava_callback - ERROR - Error processing callback for node 69c92c3d6a4c026e163493ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39064 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:48:43,074 - lava_callback - INFO - Received callback for node 69c92db76a4c026e1634942b INFO: 172.19.0.1:43514 - "POST /node/69c92db76a4c026e1634942b HTTP/1.0" 202 Accepted 2026-03-31 13:48:43,079 - lava_callback - INFO - Starting async job submit for node 69c92db76a4c026e1634942b 2026-03-31 13:48:43,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:48:43,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92db76a4c026e1634942b HTTP/1.1" 200 None 2026-03-31 13:48:43,096 - lava_callback - DEBUG - Processing node 69c92db76a4c026e1634942b: baseline-arm64-baylibre 2026-03-31 13:48:43,096 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92db76a4c026e1634942b 2026-03-31 13:48:43,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:48:43,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92db76a4c026e1634942b HTTP/1.1" 200 None 2026-03-31 13:48:43,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:48:43,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:48:43,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:48:43,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92db76a4c026e1634942b HTTP/1.1" 200 None 2026-03-31 13:48:43,155 - lava_callback - ERROR - Error processing callback for node 69c92db76a4c026e1634942b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47576 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:50:19,592 - lava_callback - INFO - Received callback for node 69c92e0a6a4c026e16349435 INFO: 172.19.0.1:47578 - "POST /node/69c92e0a6a4c026e16349435 HTTP/1.0" 202 Accepted 2026-03-31 13:50:19,598 - lava_callback - INFO - Starting async job submit for node 69c92e0a6a4c026e16349435 2026-03-31 13:50:19,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:50:19,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92e0a6a4c026e16349435 HTTP/1.1" 200 None 2026-03-31 13:50:19,614 - lava_callback - DEBUG - Processing node 69c92e0a6a4c026e16349435: baseline-arm64-clabbe 2026-03-31 13:50:19,615 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92e0a6a4c026e16349435 2026-03-31 13:50:19,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:50:19,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92e0a6a4c026e16349435 HTTP/1.1" 200 None 2026-03-31 13:50:19,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:50:19,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:50:19,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:50:19,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92e0a6a4c026e16349435 HTTP/1.1" 200 None 2026-03-31 13:50:19,698 - lava_callback - ERROR - Error processing callback for node 69c92e0a6a4c026e16349435: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51820 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:51:59,640 - lava_callback - INFO - Received callback for node 69c92e7d6a4c026e16349441 INFO: 172.19.0.1:51824 - "POST /node/69c92e7d6a4c026e16349441 HTTP/1.0" 202 Accepted 2026-03-31 13:51:59,646 - lava_callback - INFO - Starting async job submit for node 69c92e7d6a4c026e16349441 2026-03-31 13:51:59,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:51:59,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92e7d6a4c026e16349441 HTTP/1.1" 200 None 2026-03-31 13:51:59,662 - lava_callback - DEBUG - Processing node 69c92e7d6a4c026e16349441: baseline-arm64-clabbe 2026-03-31 13:51:59,662 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92e7d6a4c026e16349441 2026-03-31 13:51:59,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:51:59,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92e7d6a4c026e16349441 HTTP/1.1" 200 None 2026-03-31 13:51:59,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:51:59,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:51:59,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:51:59,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92e7d6a4c026e16349441 HTTP/1.1" 200 None 2026-03-31 13:51:59,719 - lava_callback - ERROR - Error processing callback for node 69c92e7d6a4c026e16349441: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36506 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:53:19,666 - lava_callback - INFO - Received callback for node 69c92ec96a4c026e1634944f INFO: 172.19.0.1:36510 - "POST /node/69c92ec96a4c026e1634944f HTTP/1.0" 202 Accepted 2026-03-31 13:53:19,672 - lava_callback - INFO - Starting async job submit for node 69c92ec96a4c026e1634944f 2026-03-31 13:53:19,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:53:19,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ec96a4c026e1634944f HTTP/1.1" 200 None 2026-03-31 13:53:19,689 - lava_callback - DEBUG - Processing node 69c92ec96a4c026e1634944f: baseline-arm64-clabbe 2026-03-31 13:53:19,689 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92ec96a4c026e1634944f 2026-03-31 13:53:19,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:53:19,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ec96a4c026e1634944f HTTP/1.1" 200 None 2026-03-31 13:53:19,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:53:19,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:53:19,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:53:19,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92ec96a4c026e1634944f HTTP/1.1" 200 None 2026-03-31 13:53:19,749 - lava_callback - ERROR - Error processing callback for node 69c92ec96a4c026e1634944f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59128 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.19.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40010 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:54:39,614 - lava_callback - INFO - Received callback for node 69c92f156a4c026e1634945a INFO: 172.19.0.1:40020 - "POST /node/69c92f156a4c026e1634945a HTTP/1.0" 202 Accepted 2026-03-31 13:54:39,620 - lava_callback - INFO - Starting async job submit for node 69c92f156a4c026e1634945a 2026-03-31 13:54:39,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:54:39,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92f156a4c026e1634945a HTTP/1.1" 200 None 2026-03-31 13:54:39,637 - lava_callback - DEBUG - Processing node 69c92f156a4c026e1634945a: baseline-arm64-clabbe 2026-03-31 13:54:39,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92f156a4c026e1634945a 2026-03-31 13:54:39,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:54:39,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92f156a4c026e1634945a HTTP/1.1" 200 None 2026-03-31 13:54:39,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:54:39,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:54:39,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:54:39,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92f156a4c026e1634945a HTTP/1.1" 200 None 2026-03-31 13:54:39,708 - lava_callback - ERROR - Error processing callback for node 69c92f156a4c026e1634945a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37268 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:56:59,910 - lava_callback - INFO - Received callback for node 69c92fa96a4c026e16349468 INFO: 172.19.0.1:40336 - "POST /node/69c92fa96a4c026e16349468 HTTP/1.0" 202 Accepted 2026-03-31 13:56:59,916 - lava_callback - INFO - Starting async job submit for node 69c92fa96a4c026e16349468 2026-03-31 13:56:59,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:56:59,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92fa96a4c026e16349468 HTTP/1.1" 200 None 2026-03-31 13:56:59,945 - lava_callback - DEBUG - Processing node 69c92fa96a4c026e16349468: baseline-arm64-clabbe 2026-03-31 13:56:59,945 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92fa96a4c026e16349468 2026-03-31 13:56:59,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:56:59,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92fa96a4c026e16349468 HTTP/1.1" 200 None 2026-03-31 13:56:59,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:56:59,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:56:59,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:57:00,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92fa96a4c026e16349468 HTTP/1.1" 200 None 2026-03-31 13:57:00,026 - lava_callback - ERROR - Error processing callback for node 69c92fa96a4c026e16349468: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46002 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:58:19,899 - lava_callback - INFO - Received callback for node 69c92ff66a4c026e16349477 INFO: 172.19.0.1:51232 - "POST /node/69c92ff66a4c026e16349477 HTTP/1.0" 202 Accepted 2026-03-31 13:58:19,905 - lava_callback - INFO - Starting async job submit for node 69c92ff66a4c026e16349477 2026-03-31 13:58:19,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:58:19,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ff66a4c026e16349477 HTTP/1.1" 200 None 2026-03-31 13:58:19,921 - lava_callback - DEBUG - Processing node 69c92ff66a4c026e16349477: baseline-arm64-clabbe 2026-03-31 13:58:19,921 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92ff66a4c026e16349477 2026-03-31 13:58:19,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:58:19,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ff66a4c026e16349477 HTTP/1.1" 200 None 2026-03-31 13:58:19,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:58:19,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:58:19,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:58:19,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92ff66a4c026e16349477 HTTP/1.1" 200 None 2026-03-31 13:58:19,989 - lava_callback - ERROR - Error processing callback for node 69c92ff66a4c026e16349477: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56726 - "GET /node/698c587d755d7811a4141957%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:56732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38824 - "GET /node/698f59243013abef95cf7221%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:38840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33126 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46754 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:38:05,801 - lava_callback - INFO - Received callback for node 69c9394b6a4c026e163494f7 INFO: 172.19.0.1:35808 - "POST /node/69c9394b6a4c026e163494f7 HTTP/1.0" 202 Accepted 2026-03-31 14:38:05,811 - lava_callback - INFO - Starting async job submit for node 69c9394b6a4c026e163494f7 2026-03-31 14:38:05,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:05,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9394b6a4c026e163494f7 HTTP/1.1" 200 None 2026-03-31 14:38:05,830 - lava_callback - DEBUG - Processing node 69c9394b6a4c026e163494f7: baseline-arm-baylibre 2026-03-31 14:38:05,830 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9394b6a4c026e163494f7 2026-03-31 14:38:05,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:05,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9394b6a4c026e163494f7 HTTP/1.1" 200 None 2026-03-31 14:38:05,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:05,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:38:05,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:05,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9394b6a4c026e163494f7 HTTP/1.1" 200 None 2026-03-31 14:38:05,905 - lava_callback - ERROR - Error processing callback for node 69c9394b6a4c026e163494f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46840 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:38:45,744 - lava_callback - INFO - Received callback for node 69c939706a4c026e16349503 INFO: 172.19.0.1:41514 - "POST /node/69c939706a4c026e16349503 HTTP/1.0" 202 Accepted 2026-03-31 14:38:45,750 - lava_callback - INFO - Starting async job submit for node 69c939706a4c026e16349503 2026-03-31 14:38:45,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:45,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939706a4c026e16349503 HTTP/1.1" 200 None 2026-03-31 14:38:45,768 - lava_callback - DEBUG - Processing node 69c939706a4c026e16349503: baseline-arm-baylibre 2026-03-31 14:38:45,768 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c939706a4c026e16349503 2026-03-31 14:38:45,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:45,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939706a4c026e16349503 HTTP/1.1" 200 None 2026-03-31 14:38:45,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:45,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:38:45,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:45,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c939706a4c026e16349503 HTTP/1.1" 200 None 2026-03-31 14:38:45,834 - lava_callback - ERROR - Error processing callback for node 69c939706a4c026e16349503: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55172 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:39:25,858 - lava_callback - INFO - Received callback for node 69c939956a4c026e1634950d INFO: 172.19.0.1:37472 - "POST /node/69c939956a4c026e1634950d HTTP/1.0" 202 Accepted 2026-03-31 14:39:25,864 - lava_callback - INFO - Starting async job submit for node 69c939956a4c026e1634950d 2026-03-31 14:39:25,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:39:25,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939956a4c026e1634950d HTTP/1.1" 200 None 2026-03-31 14:39:25,880 - lava_callback - DEBUG - Processing node 69c939956a4c026e1634950d: baseline-arm-baylibre 2026-03-31 14:39:25,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c939956a4c026e1634950d 2026-03-31 14:39:25,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:39:25,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939956a4c026e1634950d HTTP/1.1" 200 None 2026-03-31 14:39:25,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:39:25,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:39:25,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:39:25,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c939956a4c026e1634950d HTTP/1.1" 200 None 2026-03-31 14:39:25,953 - lava_callback - ERROR - Error processing callback for node 69c939956a4c026e1634950d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59474 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:40:05,815 - lava_callback - INFO - Received callback for node 69c939b96a4c026e1634951b INFO: 172.19.0.1:53256 - "POST /node/69c939b96a4c026e1634951b HTTP/1.0" 202 Accepted 2026-03-31 14:40:05,820 - lava_callback - INFO - Starting async job submit for node 69c939b96a4c026e1634951b 2026-03-31 14:40:05,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:05,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939b96a4c026e1634951b HTTP/1.1" 200 None 2026-03-31 14:40:05,840 - lava_callback - DEBUG - Processing node 69c939b96a4c026e1634951b: baseline-arm-baylibre 2026-03-31 14:40:05,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c939b96a4c026e1634951b 2026-03-31 14:40:05,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:05,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939b96a4c026e1634951b HTTP/1.1" 200 None 2026-03-31 14:40:05,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:05,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:40:05,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:05,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c939b96a4c026e1634951b HTTP/1.1" 200 None 2026-03-31 14:40:05,895 - lava_callback - ERROR - Error processing callback for node 69c939b96a4c026e1634951b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54078 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:40:45,965 - lava_callback - INFO - Received callback for node 69c939df6a4c026e16349527 INFO: 172.19.0.1:49078 - "POST /node/69c939df6a4c026e16349527 HTTP/1.0" 202 Accepted 2026-03-31 14:40:45,970 - lava_callback - INFO - Starting async job submit for node 69c939df6a4c026e16349527 2026-03-31 14:40:45,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:45,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939df6a4c026e16349527 HTTP/1.1" 200 None 2026-03-31 14:40:45,991 - lava_callback - DEBUG - Processing node 69c939df6a4c026e16349527: baseline-arm-baylibre 2026-03-31 14:40:45,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c939df6a4c026e16349527 2026-03-31 14:40:45,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:46,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939df6a4c026e16349527 HTTP/1.1" 200 None 2026-03-31 14:40:46,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:46,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:40:46,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:46,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c939df6a4c026e16349527 HTTP/1.1" 200 None 2026-03-31 14:40:46,052 - lava_callback - ERROR - Error processing callback for node 69c939df6a4c026e16349527: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52314 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:42:41,710 - lava_callback - INFO - Received callback for node 69c93a4d6a4c026e1634954a INFO: 172.19.0.1:34304 - "POST /node/69c93a4d6a4c026e1634954a HTTP/1.0" 202 Accepted 2026-03-31 14:42:41,717 - lava_callback - INFO - Starting async job submit for node 69c93a4d6a4c026e1634954a 2026-03-31 14:42:41,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:42:41,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a4d6a4c026e1634954a HTTP/1.1" 200 None 2026-03-31 14:42:41,740 - lava_callback - DEBUG - Processing node 69c93a4d6a4c026e1634954a: baseline-arm-clabbe 2026-03-31 14:42:41,740 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93a4d6a4c026e1634954a 2026-03-31 14:42:41,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:42:41,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a4d6a4c026e1634954a HTTP/1.1" 200 None 2026-03-31 14:42:41,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:42:41,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:42:41,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:42:41,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93a4d6a4c026e1634954a HTTP/1.1" 200 None 2026-03-31 14:42:41,799 - lava_callback - ERROR - Error processing callback for node 69c93a4d6a4c026e1634954a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33582 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:43:01,690 - lava_callback - INFO - Received callback for node 69c93a726a4c026e16349555 INFO: 172.19.0.1:60988 - "POST /node/69c93a726a4c026e16349555 HTTP/1.0" 202 Accepted 2026-03-31 14:43:01,695 - lava_callback - INFO - Starting async job submit for node 69c93a726a4c026e16349555 2026-03-31 14:43:01,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:01,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a726a4c026e16349555 HTTP/1.1" 200 None 2026-03-31 14:43:01,726 - lava_callback - DEBUG - Processing node 69c93a726a4c026e16349555: baseline-arm-clabbe 2026-03-31 14:43:01,727 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93a726a4c026e16349555 2026-03-31 14:43:01,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:01,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a726a4c026e16349555 HTTP/1.1" 200 None 2026-03-31 14:43:01,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:01,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:43:01,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:01,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93a726a4c026e16349555 HTTP/1.1" 200 None 2026-03-31 14:43:01,792 - lava_callback - ERROR - Error processing callback for node 69c93a726a4c026e16349555: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:32772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46798 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:43:41,749 - lava_callback - INFO - Received callback for node 69c93a986a4c026e1634955f INFO: 172.19.0.1:58568 - "POST /node/69c93a986a4c026e1634955f HTTP/1.0" 202 Accepted 2026-03-31 14:43:41,754 - lava_callback - INFO - Starting async job submit for node 69c93a986a4c026e1634955f 2026-03-31 14:43:41,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:41,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a986a4c026e1634955f HTTP/1.1" 200 None 2026-03-31 14:43:41,776 - lava_callback - DEBUG - Processing node 69c93a986a4c026e1634955f: baseline-arm-clabbe 2026-03-31 14:43:41,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93a986a4c026e1634955f 2026-03-31 14:43:41,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:41,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a986a4c026e1634955f HTTP/1.1" 200 None 2026-03-31 14:43:41,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:41,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:43:41,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:41,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93a986a4c026e1634955f HTTP/1.1" 200 None 2026-03-31 14:43:41,833 - lava_callback - ERROR - Error processing callback for node 69c93a986a4c026e1634955f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58974 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:44:21,729 - lava_callback - INFO - Received callback for node 69c93abe6a4c026e1634956f INFO: 172.19.0.1:52716 - "POST /node/69c93abe6a4c026e1634956f HTTP/1.0" 202 Accepted 2026-03-31 14:44:21,734 - lava_callback - INFO - Starting async job submit for node 69c93abe6a4c026e1634956f 2026-03-31 14:44:21,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:44:21,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93abe6a4c026e1634956f HTTP/1.1" 200 None 2026-03-31 14:44:21,751 - lava_callback - DEBUG - Processing node 69c93abe6a4c026e1634956f: baseline-arm-clabbe 2026-03-31 14:44:21,751 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93abe6a4c026e1634956f 2026-03-31 14:44:21,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:44:21,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93abe6a4c026e1634956f HTTP/1.1" 200 None 2026-03-31 14:44:21,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:44:21,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:44:21,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:44:21,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93abe6a4c026e1634956f HTTP/1.1" 200 None 2026-03-31 14:44:21,839 - lava_callback - ERROR - Error processing callback for node 69c93abe6a4c026e1634956f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48902 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:45:01,782 - lava_callback - INFO - Received callback for node 69c93ae36a4c026e1634957c INFO: 172.19.0.1:41992 - "POST /node/69c93ae36a4c026e1634957c HTTP/1.0" 202 Accepted 2026-03-31 14:45:01,788 - lava_callback - INFO - Starting async job submit for node 69c93ae36a4c026e1634957c 2026-03-31 14:45:01,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:01,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ae36a4c026e1634957c HTTP/1.1" 200 None 2026-03-31 14:45:01,805 - lava_callback - DEBUG - Processing node 69c93ae36a4c026e1634957c: baseline-arm-clabbe 2026-03-31 14:45:01,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93ae36a4c026e1634957c 2026-03-31 14:45:01,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:01,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ae36a4c026e1634957c HTTP/1.1" 200 None 2026-03-31 14:45:01,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:01,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:45:01,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:01,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93ae36a4c026e1634957c HTTP/1.1" 200 None 2026-03-31 14:45:01,874 - lava_callback - ERROR - Error processing callback for node 69c93ae36a4c026e1634957c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46826 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:45:26,174 - lava_callback - INFO - Received callback for node 69c93b026a4c026e1634958b INFO: 172.19.0.1:48614 - "POST /node/69c93b026a4c026e1634958b HTTP/1.0" 202 Accepted 2026-03-31 14:45:26,180 - lava_callback - INFO - Starting async job submit for node 69c93b026a4c026e1634958b 2026-03-31 14:45:26,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:26,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b026a4c026e1634958b HTTP/1.1" 200 None 2026-03-31 14:45:26,197 - lava_callback - DEBUG - Processing node 69c93b026a4c026e1634958b: baseline-arm-baylibre 2026-03-31 14:45:26,197 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93b026a4c026e1634958b 2026-03-31 14:45:26,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:26,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b026a4c026e1634958b HTTP/1.1" 200 None 2026-03-31 14:45:26,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:26,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:45:26,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:26,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93b026a4c026e1634958b HTTP/1.1" 200 None 2026-03-31 14:45:26,303 - lava_callback - ERROR - Error processing callback for node 69c93b026a4c026e1634958b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54614 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:46:06,141 - lava_callback - INFO - Received callback for node 69c93b276a4c026e16349597 INFO: 172.19.0.1:55766 - "POST /node/69c93b276a4c026e16349597 HTTP/1.0" 202 Accepted 2026-03-31 14:46:06,147 - lava_callback - INFO - Starting async job submit for node 69c93b276a4c026e16349597 2026-03-31 14:46:06,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:06,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b276a4c026e16349597 HTTP/1.1" 200 None 2026-03-31 14:46:06,175 - lava_callback - DEBUG - Processing node 69c93b276a4c026e16349597: baseline-arm-baylibre 2026-03-31 14:46:06,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93b276a4c026e16349597 2026-03-31 14:46:06,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:06,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b276a4c026e16349597 HTTP/1.1" 200 None 2026-03-31 14:46:06,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:06,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:46:06,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:06,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93b276a4c026e16349597 HTTP/1.1" 200 None 2026-03-31 14:46:06,236 - lava_callback - ERROR - Error processing callback for node 69c93b276a4c026e16349597: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34440 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:46:46,212 - lava_callback - INFO - Received callback for node 69c93b4b6a4c026e163495a4 INFO: 172.19.0.1:38506 - "POST /node/69c93b4b6a4c026e163495a4 HTTP/1.0" 202 Accepted 2026-03-31 14:46:46,218 - lava_callback - INFO - Starting async job submit for node 69c93b4b6a4c026e163495a4 2026-03-31 14:46:46,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:46,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b4b6a4c026e163495a4 HTTP/1.1" 200 None 2026-03-31 14:46:46,235 - lava_callback - DEBUG - Processing node 69c93b4b6a4c026e163495a4: baseline-arm-baylibre 2026-03-31 14:46:46,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93b4b6a4c026e163495a4 2026-03-31 14:46:46,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:46,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b4b6a4c026e163495a4 HTTP/1.1" 200 None 2026-03-31 14:46:46,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:46,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:46:46,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:46,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93b4b6a4c026e163495a4 HTTP/1.1" 200 None 2026-03-31 14:46:46,308 - lava_callback - ERROR - Error processing callback for node 69c93b4b6a4c026e163495a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58786 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:47:26,216 - lava_callback - INFO - Received callback for node 69c93b706a4c026e163495b7 INFO: 172.19.0.1:39854 - "POST /node/69c93b706a4c026e163495b7 HTTP/1.0" 202 Accepted 2026-03-31 14:47:26,222 - lava_callback - INFO - Starting async job submit for node 69c93b706a4c026e163495b7 2026-03-31 14:47:26,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:47:26,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b706a4c026e163495b7 HTTP/1.1" 200 None 2026-03-31 14:47:26,238 - lava_callback - DEBUG - Processing node 69c93b706a4c026e163495b7: baseline-arm-baylibre 2026-03-31 14:47:26,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93b706a4c026e163495b7 2026-03-31 14:47:26,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:47:26,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b706a4c026e163495b7 HTTP/1.1" 200 None 2026-03-31 14:47:26,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:47:26,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:47:26,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:47:26,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93b706a4c026e163495b7 HTTP/1.1" 200 None 2026-03-31 14:47:26,291 - lava_callback - ERROR - Error processing callback for node 69c93b706a4c026e163495b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60590 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:48:06,376 - lava_callback - INFO - Received callback for node 69c93b966a4c026e163495c0 INFO: 172.19.0.1:35488 - "POST /node/69c93b966a4c026e163495c0 HTTP/1.0" 202 Accepted 2026-03-31 14:48:06,382 - lava_callback - INFO - Starting async job submit for node 69c93b966a4c026e163495c0 2026-03-31 14:48:06,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:06,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b966a4c026e163495c0 HTTP/1.1" 200 None 2026-03-31 14:48:06,398 - lava_callback - DEBUG - Processing node 69c93b966a4c026e163495c0: baseline-arm-baylibre 2026-03-31 14:48:06,398 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93b966a4c026e163495c0 2026-03-31 14:48:06,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:06,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b966a4c026e163495c0 HTTP/1.1" 200 None 2026-03-31 14:48:06,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:06,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:48:06,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:06,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93b966a4c026e163495c0 HTTP/1.1" 200 None 2026-03-31 14:48:06,453 - lava_callback - ERROR - Error processing callback for node 69c93b966a4c026e163495c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45006 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:48:46,307 - lava_callback - INFO - Received callback for node 69c93bba6a4c026e163495c9 INFO: 172.19.0.1:41574 - "POST /node/69c93bba6a4c026e163495c9 HTTP/1.0" 202 Accepted 2026-03-31 14:48:46,313 - lava_callback - INFO - Starting async job submit for node 69c93bba6a4c026e163495c9 2026-03-31 14:48:46,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:46,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93bba6a4c026e163495c9 HTTP/1.1" 200 None 2026-03-31 14:48:46,334 - lava_callback - DEBUG - Processing node 69c93bba6a4c026e163495c9: baseline-arm-baylibre 2026-03-31 14:48:46,335 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93bba6a4c026e163495c9 2026-03-31 14:48:46,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:46,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93bba6a4c026e163495c9 HTTP/1.1" 200 None 2026-03-31 14:48:46,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:46,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:48:46,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:46,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93bba6a4c026e163495c9 HTTP/1.1" 200 None 2026-03-31 14:48:46,408 - lava_callback - ERROR - Error processing callback for node 69c93bba6a4c026e163495c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49792 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:49:21,990 - lava_callback - INFO - Received callback for node 69c93be16a4c026e163495de INFO: 172.19.0.1:47044 - "POST /node/69c93be16a4c026e163495de HTTP/1.0" 202 Accepted 2026-03-31 14:49:21,996 - lava_callback - INFO - Starting async job submit for node 69c93be16a4c026e163495de 2026-03-31 14:49:21,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:22,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93be16a4c026e163495de HTTP/1.1" 200 None 2026-03-31 14:49:22,014 - lava_callback - DEBUG - Processing node 69c93be16a4c026e163495de: baseline-arm-clabbe 2026-03-31 14:49:22,014 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93be16a4c026e163495de 2026-03-31 14:49:22,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:22,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93be16a4c026e163495de HTTP/1.1" 200 None 2026-03-31 14:49:22,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:22,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:49:22,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:22,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93be16a4c026e163495de HTTP/1.1" 200 None 2026-03-31 14:49:22,089 - lava_callback - ERROR - Error processing callback for node 69c93be16a4c026e163495de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56322 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:49:46,360 - lava_callback - INFO - Received callback for node 69c93c016a4c026e163495ed INFO: 172.19.0.1:35300 - "POST /node/69c93c016a4c026e163495ed HTTP/1.0" 202 Accepted 2026-03-31 14:49:46,366 - lava_callback - INFO - Starting async job submit for node 69c93c016a4c026e163495ed 2026-03-31 14:49:46,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:46,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c016a4c026e163495ed HTTP/1.1" 200 None 2026-03-31 14:49:46,384 - lava_callback - DEBUG - Processing node 69c93c016a4c026e163495ed: baseline-arm-baylibre 2026-03-31 14:49:46,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93c016a4c026e163495ed 2026-03-31 14:49:46,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:46,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c016a4c026e163495ed HTTP/1.1" 200 None 2026-03-31 14:49:46,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:46,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:49:46,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:46,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93c016a4c026e163495ed HTTP/1.1" 200 None 2026-03-31 14:49:46,442 - lava_callback - ERROR - Error processing callback for node 69c93c016a4c026e163495ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39758 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:50:22,108 - lava_callback - INFO - Received callback for node 69c93c2b6a4c026e163495ff INFO: 172.19.0.1:47050 - "POST /node/69c93c2b6a4c026e163495ff HTTP/1.0" 202 Accepted 2026-03-31 14:50:22,113 - lava_callback - INFO - Starting async job submit for node 69c93c2b6a4c026e163495ff 2026-03-31 14:50:22,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:50:22,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c2b6a4c026e163495ff HTTP/1.1" 200 None 2026-03-31 14:50:22,137 - lava_callback - DEBUG - Processing node 69c93c2b6a4c026e163495ff: baseline-arm-clabbe 2026-03-31 14:50:22,137 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93c2b6a4c026e163495ff 2026-03-31 14:50:22,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:50:22,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c2b6a4c026e163495ff HTTP/1.1" 200 None 2026-03-31 14:50:22,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:50:22,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:50:22,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:50:22,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93c2b6a4c026e163495ff HTTP/1.1" 200 None 2026-03-31 14:50:22,196 - lava_callback - ERROR - Error processing callback for node 69c93c2b6a4c026e163495ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58916 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:51:06,683 - lava_callback - INFO - Received callback for node 69c93c4b6a4c026e1634960a INFO: 172.19.0.1:58930 - "POST /node/69c93c4b6a4c026e1634960a HTTP/1.0" 202 Accepted 2026-03-31 14:51:06,689 - lava_callback - INFO - Starting async job submit for node 69c93c4b6a4c026e1634960a 2026-03-31 14:51:06,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:06,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c4b6a4c026e1634960a HTTP/1.1" 200 None 2026-03-31 14:51:06,704 - lava_callback - DEBUG - Processing node 69c93c4b6a4c026e1634960a: baseline-arm-baylibre 2026-03-31 14:51:06,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93c4b6a4c026e1634960a 2026-03-31 14:51:06,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:06,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c4b6a4c026e1634960a HTTP/1.1" 200 None 2026-03-31 14:51:06,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:06,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:51:06,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:06,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93c4b6a4c026e1634960a HTTP/1.1" 200 None 2026-03-31 14:51:06,769 - lava_callback - ERROR - Error processing callback for node 69c93c4b6a4c026e1634960a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42008 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:51:46,445 - lava_callback - INFO - Received callback for node 69c93c706a4c026e16349616 INFO: 172.19.0.1:36320 - "POST /node/69c93c706a4c026e16349616 HTTP/1.0" 202 Accepted 2026-03-31 14:51:46,450 - lava_callback - INFO - Starting async job submit for node 69c93c706a4c026e16349616 2026-03-31 14:51:46,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:46,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c706a4c026e16349616 HTTP/1.1" 200 None 2026-03-31 14:51:46,466 - lava_callback - DEBUG - Processing node 69c93c706a4c026e16349616: baseline-arm-baylibre 2026-03-31 14:51:46,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93c706a4c026e16349616 2026-03-31 14:51:46,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:46,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c706a4c026e16349616 HTTP/1.1" 200 None 2026-03-31 14:51:46,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:46,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:51:46,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:36330 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:51:46,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93c706a4c026e16349616 HTTP/1.1" 200 None 2026-03-31 14:51:46,533 - lava_callback - ERROR - Error processing callback for node 69c93c706a4c026e16349616: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33474 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:52:22,164 - lava_callback - INFO - Received callback for node 69c93c986a4c026e1634962f INFO: 172.19.0.1:55930 - "POST /node/69c93c986a4c026e1634962f HTTP/1.0" 202 Accepted 2026-03-31 14:52:22,170 - lava_callback - INFO - Starting async job submit for node 69c93c986a4c026e1634962f 2026-03-31 14:52:22,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:52:22,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c986a4c026e1634962f HTTP/1.1" 200 None 2026-03-31 14:52:22,191 - lava_callback - DEBUG - Processing node 69c93c986a4c026e1634962f: baseline-arm-clabbe 2026-03-31 14:52:22,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93c986a4c026e1634962f 2026-03-31 14:52:22,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:52:22,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c986a4c026e1634962f HTTP/1.1" 200 None 2026-03-31 14:52:22,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:52:22,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:52:22,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:52:22,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93c986a4c026e1634962f HTTP/1.1" 200 None 2026-03-31 14:52:22,267 - lava_callback - ERROR - Error processing callback for node 69c93c986a4c026e1634962f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55770 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:53:02,132 - lava_callback - INFO - Received callback for node 69c93cbe6a4c026e16349654 INFO: 172.19.0.1:47212 - "POST /node/69c93cbe6a4c026e16349654 HTTP/1.0" 202 Accepted 2026-03-31 14:53:02,137 - lava_callback - INFO - Starting async job submit for node 69c93cbe6a4c026e16349654 2026-03-31 14:53:02,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:02,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93cbe6a4c026e16349654 HTTP/1.1" 200 None 2026-03-31 14:53:02,160 - lava_callback - DEBUG - Processing node 69c93cbe6a4c026e16349654: baseline-arm-clabbe 2026-03-31 14:53:02,160 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93cbe6a4c026e16349654 2026-03-31 14:53:02,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:02,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93cbe6a4c026e16349654 HTTP/1.1" 200 None 2026-03-31 14:53:02,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:02,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:53:02,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:02,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93cbe6a4c026e16349654 HTTP/1.1" 200 None 2026-03-31 14:53:02,260 - lava_callback - ERROR - Error processing callback for node 69c93cbe6a4c026e16349654: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52614 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:53:26,583 - lava_callback - INFO - Received callback for node 69c93cdf6a4c026e16349667 INFO: 172.19.0.1:52618 - "POST /node/69c93cdf6a4c026e16349667 HTTP/1.0" 202 Accepted 2026-03-31 14:53:26,589 - lava_callback - INFO - Starting async job submit for node 69c93cdf6a4c026e16349667 2026-03-31 14:53:26,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:26,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93cdf6a4c026e16349667 HTTP/1.1" 200 None 2026-03-31 14:53:26,605 - lava_callback - DEBUG - Processing node 69c93cdf6a4c026e16349667: baseline-arm-baylibre 2026-03-31 14:53:26,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93cdf6a4c026e16349667 2026-03-31 14:53:26,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:26,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93cdf6a4c026e16349667 HTTP/1.1" 200 None 2026-03-31 14:53:26,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:26,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:53:26,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:26,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93cdf6a4c026e16349667 HTTP/1.1" 200 None 2026-03-31 14:53:26,668 - lava_callback - ERROR - Error processing callback for node 69c93cdf6a4c026e16349667: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55650 - "GET /robots.txt HTTP/1.0" 404 Not Found INFO: 172.19.0.1:55654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51194 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:54:02,151 - lava_callback - INFO - Received callback for node 69c93d076a4c026e16349672 INFO: 172.19.0.1:33338 - "POST /node/69c93d076a4c026e16349672 HTTP/1.0" 202 Accepted 2026-03-31 14:54:02,157 - lava_callback - INFO - Starting async job submit for node 69c93d076a4c026e16349672 2026-03-31 14:54:02,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:02,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d076a4c026e16349672 HTTP/1.1" 200 None 2026-03-31 14:54:02,173 - lava_callback - DEBUG - Processing node 69c93d076a4c026e16349672: baseline-arm-clabbe 2026-03-31 14:54:02,173 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93d076a4c026e16349672 2026-03-31 14:54:02,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:02,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d076a4c026e16349672 HTTP/1.1" 200 None 2026-03-31 14:54:02,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:02,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:54:02,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:02,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93d076a4c026e16349672 HTTP/1.1" 200 None 2026-03-31 14:54:02,233 - lava_callback - ERROR - Error processing callback for node 69c93d076a4c026e16349672: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39410 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:54:46,827 - lava_callback - INFO - Received callback for node 69c93d266a4c026e1634967f INFO: 172.19.0.1:39412 - "POST /node/69c93d266a4c026e1634967f HTTP/1.0" 202 Accepted 2026-03-31 14:54:46,833 - lava_callback - INFO - Starting async job submit for node 69c93d266a4c026e1634967f 2026-03-31 14:54:46,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:46,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d266a4c026e1634967f HTTP/1.1" 200 None 2026-03-31 14:54:46,849 - lava_callback - DEBUG - Processing node 69c93d266a4c026e1634967f: baseline-arm-baylibre 2026-03-31 14:54:46,849 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93d266a4c026e1634967f 2026-03-31 14:54:46,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:46,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d266a4c026e1634967f HTTP/1.1" 200 None 2026-03-31 14:54:46,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:46,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 14:54:46,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:46,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93d266a4c026e1634967f HTTP/1.1" 200 None 2026-03-31 14:54:46,916 - lava_callback - ERROR - Error processing callback for node 69c93d266a4c026e1634967f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54558 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:55:26,684 - lava_callback - INFO - Received callback for node 69c93d4d6a4c026e16349689 INFO: 172.19.0.1:54566 - "POST /node/69c93d4d6a4c026e16349689 HTTP/1.0" 202 Accepted 2026-03-31 14:55:26,689 - lava_callback - INFO - Starting async job submit for node 69c93d4d6a4c026e16349689 2026-03-31 14:55:26,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:55:26,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d4d6a4c026e16349689 HTTP/1.1" 200 None 2026-03-31 14:55:26,706 - lava_callback - DEBUG - Processing node 69c93d4d6a4c026e16349689: baseline-arm-baylibre 2026-03-31 14:55:26,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93d4d6a4c026e16349689 2026-03-31 14:55:26,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:55:26,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d4d6a4c026e16349689 HTTP/1.1" 200 None 2026-03-31 14:55:26,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:55:26,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:55:26,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:55:26,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93d4d6a4c026e16349689 HTTP/1.1" 200 None 2026-03-31 14:55:26,764 - lava_callback - ERROR - Error processing callback for node 69c93d4d6a4c026e16349689: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38342 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:56:06,734 - lava_callback - INFO - Received callback for node 69c93d716a4c026e16349695 INFO: 172.19.0.1:38358 - "POST /node/69c93d716a4c026e16349695 HTTP/1.0" 202 Accepted 2026-03-31 14:56:06,740 - lava_callback - INFO - Starting async job submit for node 69c93d716a4c026e16349695 2026-03-31 14:56:06,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:06,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d716a4c026e16349695 HTTP/1.1" 200 None 2026-03-31 14:56:06,756 - lava_callback - DEBUG - Processing node 69c93d716a4c026e16349695: baseline-arm-baylibre 2026-03-31 14:56:06,756 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93d716a4c026e16349695 2026-03-31 14:56:06,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:06,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d716a4c026e16349695 HTTP/1.1" 200 None 2026-03-31 14:56:06,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:06,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:56:06,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:06,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93d716a4c026e16349695 HTTP/1.1" 200 None 2026-03-31 14:56:06,819 - lava_callback - ERROR - Error processing callback for node 69c93d716a4c026e16349695: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52916 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:56:26,849 - lava_callback - INFO - Received callback for node 69c93d936a4c026e163496a0 INFO: 172.19.0.1:52922 - "POST /node/69c93d936a4c026e163496a0 HTTP/1.0" 202 Accepted 2026-03-31 14:56:26,854 - lava_callback - INFO - Starting async job submit for node 69c93d936a4c026e163496a0 2026-03-31 14:56:26,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:26,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d936a4c026e163496a0 HTTP/1.1" 200 None 2026-03-31 14:56:26,871 - lava_callback - DEBUG - Processing node 69c93d936a4c026e163496a0: baseline-arm-baylibre 2026-03-31 14:56:26,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93d936a4c026e163496a0 2026-03-31 14:56:26,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:26,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d936a4c026e163496a0 HTTP/1.1" 200 None 2026-03-31 14:56:26,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:26,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 14:56:26,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:26,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93d936a4c026e163496a0 HTTP/1.1" 200 None 2026-03-31 14:56:26,945 - lava_callback - ERROR - Error processing callback for node 69c93d936a4c026e163496a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43860 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:57:06,728 - lava_callback - INFO - Received callback for node 69c93db86a4c026e163496ae INFO: 172.19.0.1:43872 - "POST /node/69c93db86a4c026e163496ae HTTP/1.0" 202 Accepted 2026-03-31 14:57:06,734 - lava_callback - INFO - Starting async job submit for node 69c93db86a4c026e163496ae 2026-03-31 14:57:06,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:06,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93db86a4c026e163496ae HTTP/1.1" 200 None 2026-03-31 14:57:06,750 - lava_callback - DEBUG - Processing node 69c93db86a4c026e163496ae: baseline-arm-baylibre 2026-03-31 14:57:06,750 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93db86a4c026e163496ae 2026-03-31 14:57:06,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:06,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93db86a4c026e163496ae HTTP/1.1" 200 None 2026-03-31 14:57:06,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:06,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 14:57:06,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:06,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93db86a4c026e163496ae HTTP/1.1" 200 None 2026-03-31 14:57:06,814 - lava_callback - ERROR - Error processing callback for node 69c93db86a4c026e163496ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45318 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:57:46,786 - lava_callback - INFO - Received callback for node 69c93ddc6a4c026e163496bd INFO: 172.19.0.1:45328 - "POST /node/69c93ddc6a4c026e163496bd HTTP/1.0" 202 Accepted 2026-03-31 14:57:46,792 - lava_callback - INFO - Starting async job submit for node 69c93ddc6a4c026e163496bd 2026-03-31 14:57:46,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:46,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ddc6a4c026e163496bd HTTP/1.1" 200 None 2026-03-31 14:57:46,808 - lava_callback - DEBUG - Processing node 69c93ddc6a4c026e163496bd: baseline-arm-baylibre 2026-03-31 14:57:46,808 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93ddc6a4c026e163496bd 2026-03-31 14:57:46,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:46,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ddc6a4c026e163496bd HTTP/1.1" 200 None 2026-03-31 14:57:46,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:46,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 14:57:46,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:46,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93ddc6a4c026e163496bd HTTP/1.1" 200 None 2026-03-31 14:57:46,866 - lava_callback - ERROR - Error processing callback for node 69c93ddc6a4c026e163496bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56012 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:58:22,394 - lava_callback - INFO - Received callback for node 69c93e046a4c026e163496d1 INFO: 172.19.0.1:40706 - "POST /node/69c93e046a4c026e163496d1 HTTP/1.0" 202 Accepted 2026-03-31 14:58:22,400 - lava_callback - INFO - Starting async job submit for node 69c93e046a4c026e163496d1 2026-03-31 14:58:22,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:58:22,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e046a4c026e163496d1 HTTP/1.1" 200 None 2026-03-31 14:58:22,415 - lava_callback - DEBUG - Processing node 69c93e046a4c026e163496d1: baseline-arm-clabbe 2026-03-31 14:58:22,416 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93e046a4c026e163496d1 2026-03-31 14:58:22,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:58:22,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e046a4c026e163496d1 HTTP/1.1" 200 None 2026-03-31 14:58:22,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:58:22,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:58:22,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:58:22,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93e046a4c026e163496d1 HTTP/1.1" 200 None 2026-03-31 14:58:22,475 - lava_callback - ERROR - Error processing callback for node 69c93e046a4c026e163496d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41426 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:59:02,320 - lava_callback - INFO - Received callback for node 69c93e296a4c026e163496dd INFO: 172.19.0.1:55568 - "POST /node/69c93e296a4c026e163496dd HTTP/1.0" 202 Accepted 2026-03-31 14:59:02,325 - lava_callback - INFO - Starting async job submit for node 69c93e296a4c026e163496dd 2026-03-31 14:59:02,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:59:02,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e296a4c026e163496dd HTTP/1.1" 200 None 2026-03-31 14:59:02,342 - lava_callback - DEBUG - Processing node 69c93e296a4c026e163496dd: baseline-arm-clabbe 2026-03-31 14:59:02,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93e296a4c026e163496dd 2026-03-31 14:59:02,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:59:02,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e296a4c026e163496dd HTTP/1.1" 200 None 2026-03-31 14:59:02,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:59:02,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:59:02,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:59:02,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93e296a4c026e163496dd HTTP/1.1" 200 None 2026-03-31 14:59:02,405 - lava_callback - ERROR - Error processing callback for node 69c93e296a4c026e163496dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43676 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:00:42,472 - lava_callback - INFO - Received callback for node 69c93e946a4c026e163496fe INFO: 172.19.0.1:50726 - "POST /node/69c93e946a4c026e163496fe HTTP/1.0" 202 Accepted 2026-03-31 15:00:42,478 - lava_callback - INFO - Starting async job submit for node 69c93e946a4c026e163496fe 2026-03-31 15:00:42,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:00:42,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e946a4c026e163496fe HTTP/1.1" 200 None 2026-03-31 15:00:42,494 - lava_callback - DEBUG - Processing node 69c93e946a4c026e163496fe: baseline-arm-clabbe 2026-03-31 15:00:42,494 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93e946a4c026e163496fe 2026-03-31 15:00:42,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:00:42,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e946a4c026e163496fe HTTP/1.1" 200 None 2026-03-31 15:00:42,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:00:42,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 15:00:42,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:00:42,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93e946a4c026e163496fe HTTP/1.1" 200 None 2026-03-31 15:00:42,549 - lava_callback - ERROR - Error processing callback for node 69c93e946a4c026e163496fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38364 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:01:22,430 - lava_callback - INFO - Received callback for node 69c93eb96a4c026e16349710 INFO: 172.19.0.1:39168 - "POST /node/69c93eb96a4c026e16349710 HTTP/1.0" 202 Accepted 2026-03-31 15:01:22,437 - lava_callback - INFO - Starting async job submit for node 69c93eb96a4c026e16349710 2026-03-31 15:01:22,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:22,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93eb96a4c026e16349710 HTTP/1.1" 200 None 2026-03-31 15:01:22,463 - lava_callback - DEBUG - Processing node 69c93eb96a4c026e16349710: baseline-arm-clabbe 2026-03-31 15:01:22,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93eb96a4c026e16349710 2026-03-31 15:01:22,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:22,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93eb96a4c026e16349710 HTTP/1.1" 200 None 2026-03-31 15:01:22,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:22,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 15:01:22,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:22,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93eb96a4c026e16349710 HTTP/1.1" 200 None 2026-03-31 15:01:22,527 - lava_callback - ERROR - Error processing callback for node 69c93eb96a4c026e16349710: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35986 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:01:47,053 - lava_callback - INFO - Received callback for node 69c93ed96a4c026e1634971b INFO: 172.19.0.1:35990 - "POST /node/69c93ed96a4c026e1634971b HTTP/1.0" 202 Accepted 2026-03-31 15:01:47,059 - lava_callback - INFO - Starting async job submit for node 69c93ed96a4c026e1634971b 2026-03-31 15:01:47,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:47,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ed96a4c026e1634971b HTTP/1.1" 200 None 2026-03-31 15:01:47,076 - lava_callback - DEBUG - Processing node 69c93ed96a4c026e1634971b: baseline-arm-baylibre 2026-03-31 15:01:47,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93ed96a4c026e1634971b 2026-03-31 15:01:47,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:47,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ed96a4c026e1634971b HTTP/1.1" 200 None 2026-03-31 15:01:47,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:47,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:01:47,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:47,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93ed96a4c026e1634971b HTTP/1.1" 200 None 2026-03-31 15:01:47,131 - lava_callback - ERROR - Error processing callback for node 69c93ed96a4c026e1634971b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36928 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:02:42,592 - lava_callback - INFO - Received callback for node 69c93f026a4c026e16349728 INFO: 172.19.0.1:52590 - "POST /node/69c93f026a4c026e16349728 HTTP/1.0" 202 Accepted 2026-03-31 15:02:42,598 - lava_callback - INFO - Starting async job submit for node 69c93f026a4c026e16349728 2026-03-31 15:02:42,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:02:42,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f026a4c026e16349728 HTTP/1.1" 200 None 2026-03-31 15:02:42,619 - lava_callback - DEBUG - Processing node 69c93f026a4c026e16349728: baseline-arm-clabbe 2026-03-31 15:02:42,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93f026a4c026e16349728 2026-03-31 15:02:42,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:02:42,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f026a4c026e16349728 HTTP/1.1" 200 None 2026-03-31 15:02:42,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:02:42,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 15:02:42,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:02:42,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93f026a4c026e16349728 HTTP/1.1" 200 None 2026-03-31 15:02:42,676 - lava_callback - ERROR - Error processing callback for node 69c93f026a4c026e16349728: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60042 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:03:22,529 - lava_callback - INFO - Received callback for node 69c93f276a4c026e16349735 INFO: 172.19.0.1:38912 - "POST /node/69c93f276a4c026e16349735 HTTP/1.0" 202 Accepted 2026-03-31 15:03:22,535 - lava_callback - INFO - Starting async job submit for node 69c93f276a4c026e16349735 2026-03-31 15:03:22,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:22,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f276a4c026e16349735 HTTP/1.1" 200 None 2026-03-31 15:03:22,562 - lava_callback - DEBUG - Processing node 69c93f276a4c026e16349735: baseline-arm-clabbe 2026-03-31 15:03:22,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93f276a4c026e16349735 2026-03-31 15:03:22,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:22,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f276a4c026e16349735 HTTP/1.1" 200 None 2026-03-31 15:03:22,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:22,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:03:22,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:22,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93f276a4c026e16349735 HTTP/1.1" 200 None 2026-03-31 15:03:22,620 - lava_callback - ERROR - Error processing callback for node 69c93f276a4c026e16349735: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37912 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:03:47,205 - lava_callback - INFO - Received callback for node 69c93f476a4c026e16349742 INFO: 172.19.0.1:37922 - "POST /node/69c93f476a4c026e16349742 HTTP/1.0" 202 Accepted 2026-03-31 15:03:47,211 - lava_callback - INFO - Starting async job submit for node 69c93f476a4c026e16349742 2026-03-31 15:03:47,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:47,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f476a4c026e16349742 HTTP/1.1" 200 None 2026-03-31 15:03:47,228 - lava_callback - DEBUG - Processing node 69c93f476a4c026e16349742: baseline-arm-baylibre 2026-03-31 15:03:47,228 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93f476a4c026e16349742 2026-03-31 15:03:47,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:47,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f476a4c026e16349742 HTTP/1.1" 200 None 2026-03-31 15:03:47,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:47,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:03:47,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:47,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93f476a4c026e16349742 HTTP/1.1" 200 None 2026-03-31 15:03:47,282 - lava_callback - ERROR - Error processing callback for node 69c93f476a4c026e16349742: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41974 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:09:27,475 - lava_callback - INFO - Received callback for node 69c940a06a4c026e16349772 INFO: 172.19.0.1:41976 - "POST /node/69c940a06a4c026e16349772 HTTP/1.0" 202 Accepted 2026-03-31 15:09:27,480 - lava_callback - INFO - Starting async job submit for node 69c940a06a4c026e16349772 2026-03-31 15:09:27,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:09:27,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940a06a4c026e16349772 HTTP/1.1" 200 None 2026-03-31 15:09:27,496 - lava_callback - DEBUG - Processing node 69c940a06a4c026e16349772: baseline-arm-baylibre 2026-03-31 15:09:27,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c940a06a4c026e16349772 2026-03-31 15:09:27,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:09:27,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940a06a4c026e16349772 HTTP/1.1" 200 None 2026-03-31 15:09:27,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:09:27,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:09:27,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:09:27,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c940a06a4c026e16349772 HTTP/1.1" 200 None 2026-03-31 15:09:27,581 - lava_callback - ERROR - Error processing callback for node 69c940a06a4c026e16349772: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52638 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:10:22,905 - lava_callback - INFO - Received callback for node 69c940c86a4c026e1634977f INFO: 172.19.0.1:34974 - "POST /node/69c940c86a4c026e1634977f HTTP/1.0" 202 Accepted 2026-03-31 15:10:22,911 - lava_callback - INFO - Starting async job submit for node 69c940c86a4c026e1634977f 2026-03-31 15:10:22,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:22,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940c86a4c026e1634977f HTTP/1.1" 200 None 2026-03-31 15:10:22,927 - lava_callback - DEBUG - Processing node 69c940c86a4c026e1634977f: baseline-arm-clabbe 2026-03-31 15:10:22,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c940c86a4c026e1634977f 2026-03-31 15:10:22,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:22,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940c86a4c026e1634977f HTTP/1.1" 200 None 2026-03-31 15:10:22,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:22,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:10:22,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:22,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c940c86a4c026e1634977f HTTP/1.1" 200 None 2026-03-31 15:10:22,981 - lava_callback - ERROR - Error processing callback for node 69c940c86a4c026e1634977f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34656 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:10:47,520 - lava_callback - INFO - Received callback for node 69c940f26a4c026e16349792 INFO: 172.19.0.1:34660 - "POST /node/69c940f26a4c026e16349792 HTTP/1.0" 202 Accepted 2026-03-31 15:10:47,525 - lava_callback - INFO - Starting async job submit for node 69c940f26a4c026e16349792 2026-03-31 15:10:47,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:47,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940f26a4c026e16349792 HTTP/1.1" 200 None 2026-03-31 15:10:47,544 - lava_callback - DEBUG - Processing node 69c940f26a4c026e16349792: baseline-arm64-baylibre 2026-03-31 15:10:47,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c940f26a4c026e16349792 2026-03-31 15:10:47,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:47,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940f26a4c026e16349792 HTTP/1.1" 200 None 2026-03-31 15:10:47,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:47,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:10:47,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:47,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c940f26a4c026e16349792 HTTP/1.1" 200 None 2026-03-31 15:10:47,604 - lava_callback - ERROR - Error processing callback for node 69c940f26a4c026e16349792: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34216 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:12:07,565 - lava_callback - INFO - Received callback for node 69c9413f6a4c026e16349797 INFO: 172.19.0.1:34230 - "POST /node/69c9413f6a4c026e16349797 HTTP/1.0" 202 Accepted 2026-03-31 15:12:07,571 - lava_callback - INFO - Starting async job submit for node 69c9413f6a4c026e16349797 2026-03-31 15:12:07,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:12:07,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9413f6a4c026e16349797 HTTP/1.1" 200 None 2026-03-31 15:12:07,588 - lava_callback - DEBUG - Processing node 69c9413f6a4c026e16349797: baseline-arm64-baylibre 2026-03-31 15:12:07,588 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9413f6a4c026e16349797 2026-03-31 15:12:07,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:12:07,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9413f6a4c026e16349797 HTTP/1.1" 200 None 2026-03-31 15:12:07,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:12:07,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:12:07,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:12:07,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9413f6a4c026e16349797 HTTP/1.1" 200 None 2026-03-31 15:12:07,678 - lava_callback - ERROR - Error processing callback for node 69c9413f6a4c026e16349797: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59254 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:13:27,692 - lava_callback - INFO - Received callback for node 69c9418a6a4c026e163497a3 INFO: 172.19.0.1:59260 - "POST /node/69c9418a6a4c026e163497a3 HTTP/1.0" 202 Accepted 2026-03-31 15:13:27,698 - lava_callback - INFO - Starting async job submit for node 69c9418a6a4c026e163497a3 2026-03-31 15:13:27,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:13:27,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9418a6a4c026e163497a3 HTTP/1.1" 200 None 2026-03-31 15:13:27,715 - lava_callback - DEBUG - Processing node 69c9418a6a4c026e163497a3: baseline-arm64-baylibre 2026-03-31 15:13:27,715 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9418a6a4c026e163497a3 2026-03-31 15:13:27,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:13:27,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9418a6a4c026e163497a3 HTTP/1.1" 200 None 2026-03-31 15:13:27,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:13:27,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:13:27,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:13:27,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9418a6a4c026e163497a3 HTTP/1.1" 200 None 2026-03-31 15:13:27,777 - lava_callback - ERROR - Error processing callback for node 69c9418a6a4c026e163497a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41698 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:14:47,814 - lava_callback - INFO - Received callback for node 69c941d66a4c026e163497a9 INFO: 172.19.0.1:41710 - "POST /node/69c941d66a4c026e163497a9 HTTP/1.0" 202 Accepted 2026-03-31 15:14:47,820 - lava_callback - INFO - Starting async job submit for node 69c941d66a4c026e163497a9 2026-03-31 15:14:47,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:14:47,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c941d66a4c026e163497a9 HTTP/1.1" 200 None 2026-03-31 15:14:47,839 - lava_callback - DEBUG - Processing node 69c941d66a4c026e163497a9: baseline-arm64-baylibre 2026-03-31 15:14:47,839 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c941d66a4c026e163497a9 2026-03-31 15:14:47,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:14:47,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c941d66a4c026e163497a9 HTTP/1.1" 200 None 2026-03-31 15:14:47,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:14:47,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:14:47,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:14:47,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c941d66a4c026e163497a9 HTTP/1.1" 200 None 2026-03-31 15:14:47,898 - lava_callback - ERROR - Error processing callback for node 69c941d66a4c026e163497a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36818 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:16:07,788 - lava_callback - INFO - Received callback for node 69c942266a4c026e163497b4 INFO: 172.19.0.1:36826 - "POST /node/69c942266a4c026e163497b4 HTTP/1.0" 202 Accepted 2026-03-31 15:16:07,793 - lava_callback - INFO - Starting async job submit for node 69c942266a4c026e163497b4 2026-03-31 15:16:07,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:16:07,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942266a4c026e163497b4 HTTP/1.1" 200 None 2026-03-31 15:16:07,809 - lava_callback - DEBUG - Processing node 69c942266a4c026e163497b4: baseline-arm64-baylibre 2026-03-31 15:16:07,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c942266a4c026e163497b4 2026-03-31 15:16:07,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:16:07,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942266a4c026e163497b4 HTTP/1.1" 200 None 2026-03-31 15:16:07,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:16:07,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:16:07,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:16:07,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c942266a4c026e163497b4 HTTP/1.1" 200 None 2026-03-31 15:16:07,868 - lava_callback - ERROR - Error processing callback for node 69c942266a4c026e163497b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59164 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:17:27,975 - lava_callback - INFO - Received callback for node 69c942736a4c026e163497bc INFO: 172.19.0.1:59174 - "POST /node/69c942736a4c026e163497bc HTTP/1.0" 202 Accepted 2026-03-31 15:17:27,981 - lava_callback - INFO - Starting async job submit for node 69c942736a4c026e163497bc 2026-03-31 15:17:27,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:17:27,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942736a4c026e163497bc HTTP/1.1" 200 None 2026-03-31 15:17:27,998 - lava_callback - DEBUG - Processing node 69c942736a4c026e163497bc: baseline-arm64-baylibre 2026-03-31 15:17:27,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c942736a4c026e163497bc 2026-03-31 15:17:27,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:17:28,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942736a4c026e163497bc HTTP/1.1" 200 None 2026-03-31 15:17:28,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:17:28,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:17:28,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:17:28,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c942736a4c026e163497bc HTTP/1.1" 200 None 2026-03-31 15:17:28,077 - lava_callback - ERROR - Error processing callback for node 69c942736a4c026e163497bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59100 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:18:43,244 - lava_callback - INFO - Received callback for node 69c942c46a4c026e163497c7 INFO: 172.19.0.1:45396 - "POST /node/69c942c46a4c026e163497c7 HTTP/1.0" 202 Accepted 2026-03-31 15:18:43,250 - lava_callback - INFO - Starting async job submit for node 69c942c46a4c026e163497c7 2026-03-31 15:18:43,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:18:43,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942c46a4c026e163497c7 HTTP/1.1" 200 None 2026-03-31 15:18:43,267 - lava_callback - DEBUG - Processing node 69c942c46a4c026e163497c7: baseline-arm64-clabbe 2026-03-31 15:18:43,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c942c46a4c026e163497c7 2026-03-31 15:18:43,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:18:43,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942c46a4c026e163497c7 HTTP/1.1" 200 None 2026-03-31 15:18:43,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:18:43,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:18:43,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:18:43,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c942c46a4c026e163497c7 HTTP/1.1" 200 None 2026-03-31 15:18:43,342 - lava_callback - ERROR - Error processing callback for node 69c942c46a4c026e163497c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38566 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:20:03,467 - lava_callback - INFO - Received callback for node 69c943126a4c026e163497d2 INFO: 172.19.0.1:54952 - "POST /node/69c943126a4c026e163497d2 HTTP/1.0" 202 Accepted 2026-03-31 15:20:03,472 - lava_callback - INFO - Starting async job submit for node 69c943126a4c026e163497d2 2026-03-31 15:20:03,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:20:03,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943126a4c026e163497d2 HTTP/1.1" 200 None 2026-03-31 15:20:03,490 - lava_callback - DEBUG - Processing node 69c943126a4c026e163497d2: baseline-arm64-clabbe 2026-03-31 15:20:03,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c943126a4c026e163497d2 2026-03-31 15:20:03,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:20:03,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943126a4c026e163497d2 HTTP/1.1" 200 None 2026-03-31 15:20:03,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:20:03,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:20:03,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:20:03,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c943126a4c026e163497d2 HTTP/1.1" 200 None 2026-03-31 15:20:03,556 - lava_callback - ERROR - Error processing callback for node 69c943126a4c026e163497d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55438 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:21:23,288 - lava_callback - INFO - Received callback for node 69c943606a4c026e163497d9 INFO: 172.19.0.1:44712 - "POST /node/69c943606a4c026e163497d9 HTTP/1.0" 202 Accepted 2026-03-31 15:21:23,293 - lava_callback - INFO - Starting async job submit for node 69c943606a4c026e163497d9 2026-03-31 15:21:23,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:21:23,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943606a4c026e163497d9 HTTP/1.1" 200 None 2026-03-31 15:21:23,310 - lava_callback - DEBUG - Processing node 69c943606a4c026e163497d9: baseline-arm64-clabbe 2026-03-31 15:21:23,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c943606a4c026e163497d9 2026-03-31 15:21:23,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:21:23,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943606a4c026e163497d9 HTTP/1.1" 200 None 2026-03-31 15:21:23,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:21:23,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:21:23,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:21:23,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c943606a4c026e163497d9 HTTP/1.1" 200 None 2026-03-31 15:21:23,396 - lava_callback - ERROR - Error processing callback for node 69c943606a4c026e163497d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46256 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:22:43,336 - lava_callback - INFO - Received callback for node 69c943ae6a4c026e163497e6 INFO: 172.19.0.1:60568 - "POST /node/69c943ae6a4c026e163497e6 HTTP/1.0" 202 Accepted 2026-03-31 15:22:43,342 - lava_callback - INFO - Starting async job submit for node 69c943ae6a4c026e163497e6 2026-03-31 15:22:43,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:22:43,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943ae6a4c026e163497e6 HTTP/1.1" 200 None 2026-03-31 15:22:43,366 - lava_callback - DEBUG - Processing node 69c943ae6a4c026e163497e6: baseline-arm64-clabbe 2026-03-31 15:22:43,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c943ae6a4c026e163497e6 2026-03-31 15:22:43,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:22:43,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943ae6a4c026e163497e6 HTTP/1.1" 200 None 2026-03-31 15:22:43,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:22:43,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:22:43,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:22:43,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c943ae6a4c026e163497e6 HTTP/1.1" 200 None 2026-03-31 15:22:43,443 - lava_callback - ERROR - Error processing callback for node 69c943ae6a4c026e163497e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49260 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:23:43,420 - lava_callback - INFO - Received callback for node 69c943fc6a4c026e163497ed INFO: 172.19.0.1:33834 - "POST /node/69c943fc6a4c026e163497ed HTTP/1.0" 202 Accepted 2026-03-31 15:23:43,426 - lava_callback - INFO - Starting async job submit for node 69c943fc6a4c026e163497ed 2026-03-31 15:23:43,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:23:43,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943fc6a4c026e163497ed HTTP/1.1" 200 None 2026-03-31 15:23:43,446 - lava_callback - DEBUG - Processing node 69c943fc6a4c026e163497ed: baseline-arm64-clabbe 2026-03-31 15:23:43,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c943fc6a4c026e163497ed 2026-03-31 15:23:43,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:23:43,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943fc6a4c026e163497ed HTTP/1.1" 200 None 2026-03-31 15:23:43,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:23:43,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:23:43,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:23:43,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c943fc6a4c026e163497ed HTTP/1.1" 200 None 2026-03-31 15:23:43,511 - lava_callback - ERROR - Error processing callback for node 69c943fc6a4c026e163497ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50370 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:25:03,503 - lava_callback - INFO - Received callback for node 69c9444a6a4c026e163497fb INFO: 172.19.0.1:32818 - "POST /node/69c9444a6a4c026e163497fb HTTP/1.0" 202 Accepted 2026-03-31 15:25:03,510 - lava_callback - INFO - Starting async job submit for node 69c9444a6a4c026e163497fb 2026-03-31 15:25:03,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:25:03,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9444a6a4c026e163497fb HTTP/1.1" 200 None 2026-03-31 15:25:03,527 - lava_callback - DEBUG - Processing node 69c9444a6a4c026e163497fb: baseline-arm64-clabbe 2026-03-31 15:25:03,527 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9444a6a4c026e163497fb 2026-03-31 15:25:03,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:25:03,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9444a6a4c026e163497fb HTTP/1.1" 200 None 2026-03-31 15:25:03,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:25:03,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:25:03,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:25:03,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9444a6a4c026e163497fb HTTP/1.1" 200 None 2026-03-31 15:25:03,613 - lava_callback - ERROR - Error processing callback for node 69c9444a6a4c026e163497fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:32820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44938 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:26:23,515 - lava_callback - INFO - Received callback for node 69c944896a4c026e16349805 INFO: 172.19.0.1:36760 - "POST /node/69c944896a4c026e16349805 HTTP/1.0" 202 Accepted 2026-03-31 15:26:23,521 - lava_callback - INFO - Starting async job submit for node 69c944896a4c026e16349805 2026-03-31 15:26:23,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:23,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944896a4c026e16349805 HTTP/1.1" 200 None 2026-03-31 15:26:23,537 - lava_callback - DEBUG - Processing node 69c944896a4c026e16349805: baseline-arm-clabbe 2026-03-31 15:26:23,537 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c944896a4c026e16349805 2026-03-31 15:26:23,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:23,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944896a4c026e16349805 HTTP/1.1" 200 None 2026-03-31 15:26:23,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:23,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:26:23,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:23,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c944896a4c026e16349805 HTTP/1.1" 200 None 2026-03-31 15:26:23,598 - lava_callback - ERROR - Error processing callback for node 69c944896a4c026e16349805: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36772 - "GET /node/69c72ca929015d0f959922b8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:46210 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:26:43,445 - lava_callback - INFO - Received callback for node 69c944ae6a4c026e16349814 INFO: 172.19.0.1:35304 - "POST /node/69c944ae6a4c026e16349814 HTTP/1.0" 202 Accepted 2026-03-31 15:26:43,450 - lava_callback - INFO - Starting async job submit for node 69c944ae6a4c026e16349814 2026-03-31 15:26:43,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:43,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944ae6a4c026e16349814 HTTP/1.1" 200 None 2026-03-31 15:26:43,468 - lava_callback - DEBUG - Processing node 69c944ae6a4c026e16349814: baseline-arm-clabbe 2026-03-31 15:26:43,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c944ae6a4c026e16349814 2026-03-31 15:26:43,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:43,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944ae6a4c026e16349814 HTTP/1.1" 200 None 2026-03-31 15:26:43,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:43,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:26:43,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:43,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c944ae6a4c026e16349814 HTTP/1.1" 200 None 2026-03-31 15:26:43,527 - lava_callback - ERROR - Error processing callback for node 69c944ae6a4c026e16349814: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46236 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:27:23,600 - lava_callback - INFO - Received callback for node 69c944d36a4c026e16349824 INFO: 172.19.0.1:60510 - "POST /node/69c944d36a4c026e16349824 HTTP/1.0" 202 Accepted 2026-03-31 15:27:23,606 - lava_callback - INFO - Starting async job submit for node 69c944d36a4c026e16349824 2026-03-31 15:27:23,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:27:23,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944d36a4c026e16349824 HTTP/1.1" 200 None 2026-03-31 15:27:23,622 - lava_callback - DEBUG - Processing node 69c944d36a4c026e16349824: baseline-arm-clabbe 2026-03-31 15:27:23,623 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c944d36a4c026e16349824 2026-03-31 15:27:23,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:27:23,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944d36a4c026e16349824 HTTP/1.1" 200 None 2026-03-31 15:27:23,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:27:23,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:27:23,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:27:23,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c944d36a4c026e16349824 HTTP/1.1" 200 None 2026-03-31 15:27:23,686 - lava_callback - ERROR - Error processing callback for node 69c944d36a4c026e16349824: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46062 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:28:03,550 - lava_callback - INFO - Received callback for node 69c944f96a4c026e16349830 INFO: 172.19.0.1:58144 - "POST /node/69c944f96a4c026e16349830 HTTP/1.0" 202 Accepted 2026-03-31 15:28:03,555 - lava_callback - INFO - Starting async job submit for node 69c944f96a4c026e16349830 2026-03-31 15:28:03,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:03,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944f96a4c026e16349830 HTTP/1.1" 200 None 2026-03-31 15:28:03,571 - lava_callback - DEBUG - Processing node 69c944f96a4c026e16349830: baseline-arm-clabbe 2026-03-31 15:28:03,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c944f96a4c026e16349830 2026-03-31 15:28:03,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:03,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944f96a4c026e16349830 HTTP/1.1" 200 None 2026-03-31 15:28:03,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:03,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:28:03,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:03,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c944f96a4c026e16349830 HTTP/1.1" 200 None 2026-03-31 15:28:03,631 - lava_callback - ERROR - Error processing callback for node 69c944f96a4c026e16349830: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33860 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:28:43,628 - lava_callback - INFO - Received callback for node 69c9451d6a4c026e1634983d INFO: 172.19.0.1:35912 - "POST /node/69c9451d6a4c026e1634983d HTTP/1.0" 202 Accepted 2026-03-31 15:28:43,633 - lava_callback - INFO - Starting async job submit for node 69c9451d6a4c026e1634983d 2026-03-31 15:28:43,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:43,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9451d6a4c026e1634983d HTTP/1.1" 200 None 2026-03-31 15:28:43,670 - lava_callback - DEBUG - Processing node 69c9451d6a4c026e1634983d: baseline-arm-clabbe 2026-03-31 15:28:43,670 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9451d6a4c026e1634983d 2026-03-31 15:28:43,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:43,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9451d6a4c026e1634983d HTTP/1.1" 200 None 2026-03-31 15:28:43,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:43,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:28:43,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:43,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9451d6a4c026e1634983d HTTP/1.1" 200 None 2026-03-31 15:28:43,732 - lava_callback - ERROR - Error processing callback for node 69c9451d6a4c026e1634983d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52996 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:29:23,609 - lava_callback - INFO - Received callback for node 69c945436a4c026e1634984f INFO: 172.19.0.1:58418 - "POST /node/69c945436a4c026e1634984f HTTP/1.0" 202 Accepted 2026-03-31 15:29:23,614 - lava_callback - INFO - Starting async job submit for node 69c945436a4c026e1634984f 2026-03-31 15:29:23,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:29:23,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945436a4c026e1634984f HTTP/1.1" 200 None 2026-03-31 15:29:23,630 - lava_callback - DEBUG - Processing node 69c945436a4c026e1634984f: baseline-arm-clabbe 2026-03-31 15:29:23,631 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c945436a4c026e1634984f 2026-03-31 15:29:23,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:29:23,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945436a4c026e1634984f HTTP/1.1" 200 None 2026-03-31 15:29:23,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:29:23,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:29:23,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:29:23,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c945436a4c026e1634984f HTTP/1.1" 200 None 2026-03-31 15:29:23,703 - lava_callback - ERROR - Error processing callback for node 69c945436a4c026e1634984f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55368 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:30:23,684 - lava_callback - INFO - Received callback for node 69c9458a6a4c026e1634986b INFO: 172.19.0.1:43708 - "POST /node/69c9458a6a4c026e1634986b HTTP/1.0" 202 Accepted 2026-03-31 15:30:23,689 - lava_callback - INFO - Starting async job submit for node 69c9458a6a4c026e1634986b 2026-03-31 15:30:23,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:30:23,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9458a6a4c026e1634986b HTTP/1.1" 200 None 2026-03-31 15:30:23,706 - lava_callback - DEBUG - Processing node 69c9458a6a4c026e1634986b: baseline-arm-clabbe 2026-03-31 15:30:23,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9458a6a4c026e1634986b 2026-03-31 15:30:23,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:30:23,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9458a6a4c026e1634986b HTTP/1.1" 200 None 2026-03-31 15:30:23,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:30:23,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:30:23,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:30:23,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9458a6a4c026e1634986b HTTP/1.1" 200 None 2026-03-31 15:30:23,762 - lava_callback - ERROR - Error processing callback for node 69c9458a6a4c026e1634986b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34118 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:31:03,676 - lava_callback - INFO - Received callback for node 69c945af6a4c026e1634987a INFO: 172.19.0.1:51004 - "POST /node/69c945af6a4c026e1634987a HTTP/1.0" 202 Accepted 2026-03-31 15:31:03,681 - lava_callback - INFO - Starting async job submit for node 69c945af6a4c026e1634987a 2026-03-31 15:31:03,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:03,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945af6a4c026e1634987a HTTP/1.1" 200 None 2026-03-31 15:31:03,698 - lava_callback - DEBUG - Processing node 69c945af6a4c026e1634987a: baseline-arm-clabbe 2026-03-31 15:31:03,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c945af6a4c026e1634987a 2026-03-31 15:31:03,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:03,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945af6a4c026e1634987a HTTP/1.1" 200 None 2026-03-31 15:31:03,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:03,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:31:03,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:03,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c945af6a4c026e1634987a HTTP/1.1" 200 None 2026-03-31 15:31:03,762 - lava_callback - ERROR - Error processing callback for node 69c945af6a4c026e1634987a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48798 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:31:48,755 - lava_callback - INFO - Received callback for node 69c945cf6a4c026e1634988a INFO: 172.19.0.1:48802 - "POST /node/69c945cf6a4c026e1634988a HTTP/1.0" 202 Accepted 2026-03-31 15:31:48,761 - lava_callback - INFO - Starting async job submit for node 69c945cf6a4c026e1634988a 2026-03-31 15:31:48,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:48,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945cf6a4c026e1634988a HTTP/1.1" 200 None 2026-03-31 15:31:48,777 - lava_callback - DEBUG - Processing node 69c945cf6a4c026e1634988a: baseline-arm-baylibre 2026-03-31 15:31:48,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c945cf6a4c026e1634988a 2026-03-31 15:31:48,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:48,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945cf6a4c026e1634988a HTTP/1.1" 200 None 2026-03-31 15:31:48,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:48,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:31:48,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:48,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c945cf6a4c026e1634988a HTTP/1.1" 200 None 2026-03-31 15:31:48,842 - lava_callback - ERROR - Error processing callback for node 69c945cf6a4c026e1634988a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45336 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:32:08,599 - lava_callback - INFO - Received callback for node 69c945f36a4c026e1634989a INFO: 172.19.0.1:45344 - "POST /node/69c945f36a4c026e1634989a HTTP/1.0" 202 Accepted 2026-03-31 15:32:08,604 - lava_callback - INFO - Starting async job submit for node 69c945f36a4c026e1634989a 2026-03-31 15:32:08,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:32:08,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945f36a4c026e1634989a HTTP/1.1" 200 None 2026-03-31 15:32:08,620 - lava_callback - DEBUG - Processing node 69c945f36a4c026e1634989a: baseline-arm-baylibre 2026-03-31 15:32:08,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c945f36a4c026e1634989a 2026-03-31 15:32:08,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:32:08,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945f36a4c026e1634989a HTTP/1.1" 200 None 2026-03-31 15:32:08,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:32:08,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:32:08,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:32:08,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c945f36a4c026e1634989a HTTP/1.1" 200 None 2026-03-31 15:32:08,677 - lava_callback - ERROR - Error processing callback for node 69c945f36a4c026e1634989a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53626 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:33:03,807 - lava_callback - INFO - Received callback for node 69c9461c6a4c026e163498ae INFO: 172.19.0.1:42678 - "POST /node/69c9461c6a4c026e163498ae HTTP/1.0" 202 Accepted 2026-03-31 15:33:03,812 - lava_callback - INFO - Starting async job submit for node 69c9461c6a4c026e163498ae 2026-03-31 15:33:03,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:03,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9461c6a4c026e163498ae HTTP/1.1" 200 None 2026-03-31 15:33:03,828 - lava_callback - DEBUG - Processing node 69c9461c6a4c026e163498ae: baseline-arm-clabbe 2026-03-31 15:33:03,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9461c6a4c026e163498ae 2026-03-31 15:33:03,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:03,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9461c6a4c026e163498ae HTTP/1.1" 200 None 2026-03-31 15:33:03,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:03,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:33:03,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:03,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9461c6a4c026e163498ae HTTP/1.1" 200 None 2026-03-31 15:33:03,879 - lava_callback - ERROR - Error processing callback for node 69c9461c6a4c026e163498ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39038 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:33:43,790 - lava_callback - INFO - Received callback for node 69c946426a4c026e163498b7 INFO: 172.19.0.1:43568 - "POST /node/69c946426a4c026e163498b7 HTTP/1.0" 202 Accepted 2026-03-31 15:33:43,795 - lava_callback - INFO - Starting async job submit for node 69c946426a4c026e163498b7 2026-03-31 15:33:43,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:43,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946426a4c026e163498b7 HTTP/1.1" 200 None 2026-03-31 15:33:43,811 - lava_callback - DEBUG - Processing node 69c946426a4c026e163498b7: baseline-arm-clabbe 2026-03-31 15:33:43,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c946426a4c026e163498b7 2026-03-31 15:33:43,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:43,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946426a4c026e163498b7 HTTP/1.1" 200 None 2026-03-31 15:33:43,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:43,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:33:43,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:43,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c946426a4c026e163498b7 HTTP/1.1" 200 None 2026-03-31 15:33:43,868 - lava_callback - ERROR - Error processing callback for node 69c946426a4c026e163498b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47544 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:34:03,863 - lava_callback - INFO - Received callback for node 69c946676a4c026e163498c6 INFO: 172.19.0.1:55654 - "POST /node/69c946676a4c026e163498c6 HTTP/1.0" 202 Accepted 2026-03-31 15:34:03,869 - lava_callback - INFO - Starting async job submit for node 69c946676a4c026e163498c6 2026-03-31 15:34:03,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:03,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946676a4c026e163498c6 HTTP/1.1" 200 None 2026-03-31 15:34:03,886 - lava_callback - DEBUG - Processing node 69c946676a4c026e163498c6: baseline-arm-clabbe 2026-03-31 15:34:03,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c946676a4c026e163498c6 2026-03-31 15:34:03,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:03,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946676a4c026e163498c6 HTTP/1.1" 200 None 2026-03-31 15:34:03,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:03,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:34:03,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:03,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c946676a4c026e163498c6 HTTP/1.1" 200 None 2026-03-31 15:34:03,943 - lava_callback - ERROR - Error processing callback for node 69c946676a4c026e163498c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42782 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:34:43,862 - lava_callback - INFO - Received callback for node 69c9468c6a4c026e163498d6 INFO: 172.19.0.1:43182 - "POST /node/69c9468c6a4c026e163498d6 HTTP/1.0" 202 Accepted 2026-03-31 15:34:43,867 - lava_callback - INFO - Starting async job submit for node 69c9468c6a4c026e163498d6 2026-03-31 15:34:43,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:43,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9468c6a4c026e163498d6 HTTP/1.1" 200 None 2026-03-31 15:34:43,904 - lava_callback - DEBUG - Processing node 69c9468c6a4c026e163498d6: baseline-arm-clabbe 2026-03-31 15:34:43,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9468c6a4c026e163498d6 2026-03-31 15:34:43,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:43,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9468c6a4c026e163498d6 HTTP/1.1" 200 None 2026-03-31 15:34:43,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:43,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:34:43,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:43,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9468c6a4c026e163498d6 HTTP/1.1" 200 None 2026-03-31 15:34:43,966 - lava_callback - ERROR - Error processing callback for node 69c9468c6a4c026e163498d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47724 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:35:23,918 - lava_callback - INFO - Received callback for node 69c946b16a4c026e163498ec INFO: 172.19.0.1:38596 - "POST /node/69c946b16a4c026e163498ec HTTP/1.0" 202 Accepted 2026-03-31 15:35:23,924 - lava_callback - INFO - Starting async job submit for node 69c946b16a4c026e163498ec 2026-03-31 15:35:23,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:35:23,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946b16a4c026e163498ec HTTP/1.1" 200 None 2026-03-31 15:35:23,941 - lava_callback - DEBUG - Processing node 69c946b16a4c026e163498ec: baseline-arm-clabbe 2026-03-31 15:35:23,941 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c946b16a4c026e163498ec 2026-03-31 15:35:23,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:35:23,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946b16a4c026e163498ec HTTP/1.1" 200 None 2026-03-31 15:35:23,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:35:23,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:35:23,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:35:24,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c946b16a4c026e163498ec HTTP/1.1" 200 None 2026-03-31 15:35:24,014 - lava_callback - ERROR - Error processing callback for node 69c946b16a4c026e163498ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35296 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:36:08,800 - lava_callback - INFO - Received callback for node 69c946dd6a4c026e163498f5 INFO: 172.19.0.1:35302 - "POST /node/69c946dd6a4c026e163498f5 HTTP/1.0" 202 Accepted 2026-03-31 15:36:08,809 - lava_callback - INFO - Starting async job submit for node 69c946dd6a4c026e163498f5 2026-03-31 15:36:08,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:36:08,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946dd6a4c026e163498f5 HTTP/1.1" 200 None 2026-03-31 15:36:08,827 - lava_callback - DEBUG - Processing node 69c946dd6a4c026e163498f5: baseline-arm64-baylibre 2026-03-31 15:36:08,827 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c946dd6a4c026e163498f5 2026-03-31 15:36:08,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:36:08,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946dd6a4c026e163498f5 HTTP/1.1" 200 None 2026-03-31 15:36:08,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:36:08,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 15:36:08,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:36:08,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c946dd6a4c026e163498f5 HTTP/1.1" 200 None 2026-03-31 15:36:08,901 - lava_callback - ERROR - Error processing callback for node 69c946dd6a4c026e163498f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41392 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:37:28,889 - lava_callback - INFO - Received callback for node 69c9472a6a4c026e16349906 INFO: 172.19.0.1:41402 - "POST /node/69c9472a6a4c026e16349906 HTTP/1.0" 202 Accepted 2026-03-31 15:37:28,894 - lava_callback - INFO - Starting async job submit for node 69c9472a6a4c026e16349906 2026-03-31 15:37:28,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:37:28,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9472a6a4c026e16349906 HTTP/1.1" 200 None 2026-03-31 15:37:28,915 - lava_callback - DEBUG - Processing node 69c9472a6a4c026e16349906: baseline-arm64-baylibre 2026-03-31 15:37:28,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9472a6a4c026e16349906 2026-03-31 15:37:28,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:37:28,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9472a6a4c026e16349906 HTTP/1.1" 200 None 2026-03-31 15:37:28,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:37:28,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 15:37:28,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:37:28,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9472a6a4c026e16349906 HTTP/1.1" 200 None 2026-03-31 15:37:28,975 - lava_callback - ERROR - Error processing callback for node 69c9472a6a4c026e16349906: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37260 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:38:48,974 - lava_callback - INFO - Received callback for node 69c947776a4c026e1634990b INFO: 172.19.0.1:37276 - "POST /node/69c947776a4c026e1634990b HTTP/1.0" 202 Accepted 2026-03-31 15:38:48,980 - lava_callback - INFO - Starting async job submit for node 69c947776a4c026e1634990b 2026-03-31 15:38:48,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:38:49,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c947776a4c026e1634990b HTTP/1.1" 200 None 2026-03-31 15:38:49,001 - lava_callback - DEBUG - Processing node 69c947776a4c026e1634990b: baseline-arm64-baylibre 2026-03-31 15:38:49,001 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c947776a4c026e1634990b 2026-03-31 15:38:49,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:38:49,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c947776a4c026e1634990b HTTP/1.1" 200 None 2026-03-31 15:38:49,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:38:49,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:38:49,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:38:49,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c947776a4c026e1634990b HTTP/1.1" 200 None 2026-03-31 15:38:49,061 - lava_callback - ERROR - Error processing callback for node 69c947776a4c026e1634990b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42424 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:39:48,993 - lava_callback - INFO - Received callback for node 69c947c26a4c026e1634991d INFO: 172.19.0.1:42438 - "POST /node/69c947c26a4c026e1634991d HTTP/1.0" 202 Accepted 2026-03-31 15:39:48,999 - lava_callback - INFO - Starting async job submit for node 69c947c26a4c026e1634991d 2026-03-31 15:39:49,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:39:49,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c947c26a4c026e1634991d HTTP/1.1" 200 None 2026-03-31 15:39:49,021 - lava_callback - DEBUG - Processing node 69c947c26a4c026e1634991d: baseline-arm64-baylibre 2026-03-31 15:39:49,022 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c947c26a4c026e1634991d 2026-03-31 15:39:49,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:39:49,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c947c26a4c026e1634991d HTTP/1.1" 200 None 2026-03-31 15:39:49,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:39:49,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:39:49,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:39:49,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c947c26a4c026e1634991d HTTP/1.1" 200 None 2026-03-31 15:39:49,098 - lava_callback - ERROR - Error processing callback for node 69c947c26a4c026e1634991d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47380 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:41:09,170 - lava_callback - INFO - Received callback for node 69c948116a4c026e1634992b INFO: 172.19.0.1:47386 - "POST /node/69c948116a4c026e1634992b HTTP/1.0" 202 Accepted 2026-03-31 15:41:09,176 - lava_callback - INFO - Starting async job submit for node 69c948116a4c026e1634992b 2026-03-31 15:41:09,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:41:09,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c948116a4c026e1634992b HTTP/1.1" 200 None 2026-03-31 15:41:09,192 - lava_callback - DEBUG - Processing node 69c948116a4c026e1634992b: baseline-arm64-baylibre 2026-03-31 15:41:09,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c948116a4c026e1634992b 2026-03-31 15:41:09,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:41:09,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c948116a4c026e1634992b HTTP/1.1" 200 None 2026-03-31 15:41:09,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:41:09,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:41:09,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:41:09,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c948116a4c026e1634992b HTTP/1.1" 200 None 2026-03-31 15:41:09,268 - lava_callback - ERROR - Error processing callback for node 69c948116a4c026e1634992b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56318 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:42:29,195 - lava_callback - INFO - Received callback for node 69c9485f6a4c026e16349936 INFO: 172.19.0.1:56334 - "POST /node/69c9485f6a4c026e16349936 HTTP/1.0" 202 Accepted 2026-03-31 15:42:29,203 - lava_callback - INFO - Starting async job submit for node 69c9485f6a4c026e16349936 2026-03-31 15:42:29,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:42:29,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9485f6a4c026e16349936 HTTP/1.1" 200 None 2026-03-31 15:42:29,227 - lava_callback - DEBUG - Processing node 69c9485f6a4c026e16349936: baseline-arm64-baylibre 2026-03-31 15:42:29,227 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9485f6a4c026e16349936 2026-03-31 15:42:29,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:42:29,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9485f6a4c026e16349936 HTTP/1.1" 200 None 2026-03-31 15:42:29,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:42:29,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:42:29,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:42:29,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9485f6a4c026e16349936 HTTP/1.1" 200 None 2026-03-31 15:42:29,298 - lava_callback - ERROR - Error processing callback for node 69c9485f6a4c026e16349936: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40556 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:43:49,589 - lava_callback - INFO - Received callback for node 69c948ad6a4c026e1634993f INFO: 172.19.0.1:40560 - "POST /node/69c948ad6a4c026e1634993f HTTP/1.0" 202 Accepted 2026-03-31 15:43:49,594 - lava_callback - INFO - Starting async job submit for node 69c948ad6a4c026e1634993f 2026-03-31 15:43:49,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:43:49,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c948ad6a4c026e1634993f HTTP/1.1" 200 None 2026-03-31 15:43:49,614 - lava_callback - DEBUG - Processing node 69c948ad6a4c026e1634993f: baseline-arm64-baylibre 2026-03-31 15:43:49,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c948ad6a4c026e1634993f 2026-03-31 15:43:49,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:43:49,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c948ad6a4c026e1634993f HTTP/1.1" 200 None 2026-03-31 15:43:49,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:43:49,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:43:49,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:43:49,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c948ad6a4c026e1634993f HTTP/1.1" 200 None 2026-03-31 15:43:49,686 - lava_callback - ERROR - Error processing callback for node 69c948ad6a4c026e1634993f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45824 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:46:29,465 - lava_callback - INFO - Received callback for node 69c949426a4c026e16349955 INFO: 172.19.0.1:45828 - "POST /node/69c949426a4c026e16349955 HTTP/1.0" 202 Accepted 2026-03-31 15:46:29,471 - lava_callback - INFO - Starting async job submit for node 69c949426a4c026e16349955 2026-03-31 15:46:29,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:46:29,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949426a4c026e16349955 HTTP/1.1" 200 None 2026-03-31 15:46:29,487 - lava_callback - DEBUG - Processing node 69c949426a4c026e16349955: baseline-arm64-baylibre 2026-03-31 15:46:29,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c949426a4c026e16349955 2026-03-31 15:46:29,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:46:29,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949426a4c026e16349955 HTTP/1.1" 200 None 2026-03-31 15:46:29,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:46:29,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 15:46:29,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:46:29,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c949426a4c026e16349955 HTTP/1.1" 200 None 2026-03-31 15:46:29,577 - lava_callback - ERROR - Error processing callback for node 69c949426a4c026e16349955: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43406 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:47:49,428 - lava_callback - INFO - Received callback for node 69c949916a4c026e16349961 INFO: 172.19.0.1:43408 - "POST /node/69c949916a4c026e16349961 HTTP/1.0" 202 Accepted 2026-03-31 15:47:49,433 - lava_callback - INFO - Starting async job submit for node 69c949916a4c026e16349961 2026-03-31 15:47:49,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:47:49,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949916a4c026e16349961 HTTP/1.1" 200 None 2026-03-31 15:47:49,462 - lava_callback - DEBUG - Processing node 69c949916a4c026e16349961: baseline-arm64-baylibre 2026-03-31 15:47:49,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c949916a4c026e16349961 2026-03-31 15:47:49,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:47:49,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949916a4c026e16349961 HTTP/1.1" 200 None 2026-03-31 15:47:49,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:47:49,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 15:47:49,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:47:49,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c949916a4c026e16349961 HTTP/1.1" 200 None 2026-03-31 15:47:49,525 - lava_callback - ERROR - Error processing callback for node 69c949916a4c026e16349961: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39466 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:49:04,574 - lava_callback - INFO - Received callback for node 69c949e36a4c026e16349975 INFO: 172.19.0.1:33492 - "POST /node/69c949e36a4c026e16349975 HTTP/1.0" 202 Accepted 2026-03-31 15:49:04,580 - lava_callback - INFO - Starting async job submit for node 69c949e36a4c026e16349975 2026-03-31 15:49:04,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:49:04,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949e36a4c026e16349975 HTTP/1.1" 200 None 2026-03-31 15:49:04,619 - lava_callback - DEBUG - Processing node 69c949e36a4c026e16349975: baseline-arm64-clabbe 2026-03-31 15:49:04,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c949e36a4c026e16349975 2026-03-31 15:49:04,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:49:04,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949e36a4c026e16349975 HTTP/1.1" 200 None 2026-03-31 15:49:04,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:49:04,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:49:04,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:49:04,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c949e36a4c026e16349975 HTTP/1.1" 200 None 2026-03-31 15:49:04,695 - lava_callback - ERROR - Error processing callback for node 69c949e36a4c026e16349975: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58800 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.19.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50932 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:50:24,583 - lava_callback - INFO - Received callback for node 69c94a306a4c026e16349981 INFO: 172.19.0.1:40362 - "POST /node/69c94a306a4c026e16349981 HTTP/1.0" 202 Accepted 2026-03-31 15:50:24,589 - lava_callback - INFO - Starting async job submit for node 69c94a306a4c026e16349981 2026-03-31 15:50:24,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:50:24,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94a306a4c026e16349981 HTTP/1.1" 200 None 2026-03-31 15:50:24,611 - lava_callback - DEBUG - Processing node 69c94a306a4c026e16349981: baseline-arm64-clabbe 2026-03-31 15:50:24,611 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94a306a4c026e16349981 2026-03-31 15:50:24,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:50:24,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94a306a4c026e16349981 HTTP/1.1" 200 None 2026-03-31 15:50:24,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:50:24,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:50:24,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:50:24,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94a306a4c026e16349981 HTTP/1.1" 200 None 2026-03-31 15:50:24,675 - lava_callback - ERROR - Error processing callback for node 69c94a306a4c026e16349981: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46224 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:51:29,748 - lava_callback - INFO - Received callback for node 69c94a756a4c026e1634998d INFO: 172.19.0.1:60484 - "POST /node/69c94a756a4c026e1634998d HTTP/1.0" 202 Accepted 2026-03-31 15:51:29,754 - lava_callback - INFO - Starting async job submit for node 69c94a756a4c026e1634998d 2026-03-31 15:51:29,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:51:29,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94a756a4c026e1634998d HTTP/1.1" 200 None 2026-03-31 15:51:29,770 - lava_callback - DEBUG - Processing node 69c94a756a4c026e1634998d: baseline-arm64-baylibre 2026-03-31 15:51:29,770 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94a756a4c026e1634998d 2026-03-31 15:51:29,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:51:29,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94a756a4c026e1634998d HTTP/1.1" 200 None 2026-03-31 15:51:29,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:51:29,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 15:51:29,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:51:29,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94a756a4c026e1634998d HTTP/1.1" 200 None 2026-03-31 15:51:29,995 - lava_callback - ERROR - Error processing callback for node 69c94a756a4c026e1634998d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53912 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:54:04,679 - lava_callback - INFO - Received callback for node 69c94b146a4c026e163499a8 INFO: 172.19.0.1:46794 - "POST /node/69c94b146a4c026e163499a8 HTTP/1.0" 202 Accepted 2026-03-31 15:54:04,685 - lava_callback - INFO - Starting async job submit for node 69c94b146a4c026e163499a8 2026-03-31 15:54:04,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:54:04,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94b146a4c026e163499a8 HTTP/1.1" 200 None 2026-03-31 15:54:04,707 - lava_callback - DEBUG - Processing node 69c94b146a4c026e163499a8: baseline-arm64-clabbe 2026-03-31 15:54:04,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94b146a4c026e163499a8 2026-03-31 15:54:04,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:54:04,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94b146a4c026e163499a8 HTTP/1.1" 200 None 2026-03-31 15:54:04,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:54:04,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:54:04,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:54:04,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94b146a4c026e163499a8 HTTP/1.1" 200 None 2026-03-31 15:54:04,773 - lava_callback - ERROR - Error processing callback for node 69c94b146a4c026e163499a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52046 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:56:44,831 - lava_callback - INFO - Received callback for node 69c94bac6a4c026e163499c2 INFO: 172.19.0.1:34978 - "POST /node/69c94bac6a4c026e163499c2 HTTP/1.0" 202 Accepted 2026-03-31 15:56:44,837 - lava_callback - INFO - Starting async job submit for node 69c94bac6a4c026e163499c2 2026-03-31 15:56:44,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:56:44,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94bac6a4c026e163499c2 HTTP/1.1" 200 None 2026-03-31 15:56:44,853 - lava_callback - DEBUG - Processing node 69c94bac6a4c026e163499c2: baseline-arm64-clabbe 2026-03-31 15:56:44,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94bac6a4c026e163499c2 2026-03-31 15:56:44,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:56:44,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94bac6a4c026e163499c2 HTTP/1.1" 200 None 2026-03-31 15:56:44,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:56:44,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:56:44,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:56:44,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94bac6a4c026e163499c2 HTTP/1.1" 200 None 2026-03-31 15:56:44,918 - lava_callback - ERROR - Error processing callback for node 69c94bac6a4c026e163499c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46456 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:59:04,929 - lava_callback - INFO - Received callback for node 69c94c446a4c026e163499de INFO: 172.19.0.1:51940 - "POST /node/69c94c446a4c026e163499de HTTP/1.0" 202 Accepted 2026-03-31 15:59:04,934 - lava_callback - INFO - Starting async job submit for node 69c94c446a4c026e163499de 2026-03-31 15:59:04,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:59:04,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94c446a4c026e163499de HTTP/1.1" 200 None 2026-03-31 15:59:04,958 - lava_callback - DEBUG - Processing node 69c94c446a4c026e163499de: baseline-arm64-clabbe 2026-03-31 15:59:04,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94c446a4c026e163499de 2026-03-31 15:59:04,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:59:04,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94c446a4c026e163499de HTTP/1.1" 200 None 2026-03-31 15:59:04,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:59:04,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:59:04,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:59:05,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94c446a4c026e163499de HTTP/1.1" 200 None 2026-03-31 15:59:05,015 - lava_callback - ERROR - Error processing callback for node 69c94c446a4c026e163499de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46106 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:00:30,224 - lava_callback - INFO - Received callback for node 69c94c8d6a4c026e163499f0 INFO: 172.19.0.1:36172 - "POST /node/69c94c8d6a4c026e163499f0 HTTP/1.0" 202 Accepted 2026-03-31 16:00:30,229 - lava_callback - INFO - Starting async job submit for node 69c94c8d6a4c026e163499f0 2026-03-31 16:00:30,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:00:30,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94c8d6a4c026e163499f0 HTTP/1.1" 200 None 2026-03-31 16:00:30,261 - lava_callback - DEBUG - Processing node 69c94c8d6a4c026e163499f0: baseline-arm64-baylibre 2026-03-31 16:00:30,261 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94c8d6a4c026e163499f0 2026-03-31 16:00:30,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:00:30,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94c8d6a4c026e163499f0 HTTP/1.1" 200 None 2026-03-31 16:00:30,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:00:30,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:00:30,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:00:30,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94c8d6a4c026e163499f0 HTTP/1.1" 200 None 2026-03-31 16:00:30,313 - lava_callback - ERROR - Error processing callback for node 69c94c8d6a4c026e163499f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58978 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:01:45,076 - lava_callback - INFO - Received callback for node 69c94cdf6a4c026e163499f7 INFO: 172.19.0.1:60706 - "POST /node/69c94cdf6a4c026e163499f7 HTTP/1.0" 202 Accepted 2026-03-31 16:01:45,081 - lava_callback - INFO - Starting async job submit for node 69c94cdf6a4c026e163499f7 2026-03-31 16:01:45,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:01:45,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94cdf6a4c026e163499f7 HTTP/1.1" 200 None 2026-03-31 16:01:45,098 - lava_callback - DEBUG - Processing node 69c94cdf6a4c026e163499f7: baseline-arm64-clabbe 2026-03-31 16:01:45,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94cdf6a4c026e163499f7 2026-03-31 16:01:45,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:01:45,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94cdf6a4c026e163499f7 HTTP/1.1" 200 None 2026-03-31 16:01:45,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:01:45,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:01:45,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:01:45,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94cdf6a4c026e163499f7 HTTP/1.1" 200 None 2026-03-31 16:01:45,189 - lava_callback - ERROR - Error processing callback for node 69c94cdf6a4c026e163499f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48584 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:02:50,360 - lava_callback - INFO - Received callback for node 69c94d276a4c026e16349a0a INFO: 172.19.0.1:52232 - "POST /node/69c94d276a4c026e16349a0a HTTP/1.0" 202 Accepted 2026-03-31 16:02:50,366 - lava_callback - INFO - Starting async job submit for node 69c94d276a4c026e16349a0a 2026-03-31 16:02:50,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:02:50,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94d276a4c026e16349a0a HTTP/1.1" 200 None 2026-03-31 16:02:50,382 - lava_callback - DEBUG - Processing node 69c94d276a4c026e16349a0a: baseline-arm64-baylibre 2026-03-31 16:02:50,383 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94d276a4c026e16349a0a 2026-03-31 16:02:50,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:02:50,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94d276a4c026e16349a0a HTTP/1.1" 200 None 2026-03-31 16:02:50,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:02:50,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:02:50,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:02:50,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94d276a4c026e16349a0a HTTP/1.1" 200 None 2026-03-31 16:02:50,435 - lava_callback - ERROR - Error processing callback for node 69c94d276a4c026e16349a0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35600 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:04:10,428 - lava_callback - INFO - Received callback for node 69c94d746a4c026e16349a14 INFO: 172.19.0.1:39686 - "POST /node/69c94d746a4c026e16349a14 HTTP/1.0" 202 Accepted 2026-03-31 16:04:10,439 - lava_callback - INFO - Starting async job submit for node 69c94d746a4c026e16349a14 2026-03-31 16:04:10,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:04:10,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94d746a4c026e16349a14 HTTP/1.1" 200 None 2026-03-31 16:04:10,464 - lava_callback - DEBUG - Processing node 69c94d746a4c026e16349a14: baseline-arm64-baylibre 2026-03-31 16:04:10,465 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94d746a4c026e16349a14 2026-03-31 16:04:10,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:04:10,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94d746a4c026e16349a14 HTTP/1.1" 200 None 2026-03-31 16:04:10,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:04:10,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:04:10,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:04:10,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94d746a4c026e16349a14 HTTP/1.1" 200 None 2026-03-31 16:04:10,526 - lava_callback - ERROR - Error processing callback for node 69c94d746a4c026e16349a14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51846 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:05:30,451 - lava_callback - INFO - Received callback for node 69c94dc16a4c026e16349a23 INFO: 172.19.0.1:58734 - "POST /node/69c94dc16a4c026e16349a23 HTTP/1.0" 202 Accepted 2026-03-31 16:05:30,456 - lava_callback - INFO - Starting async job submit for node 69c94dc16a4c026e16349a23 2026-03-31 16:05:30,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:05:30,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94dc16a4c026e16349a23 HTTP/1.1" 200 None 2026-03-31 16:05:30,478 - lava_callback - DEBUG - Processing node 69c94dc16a4c026e16349a23: baseline-arm64-baylibre 2026-03-31 16:05:30,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94dc16a4c026e16349a23 2026-03-31 16:05:30,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:05:30,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94dc16a4c026e16349a23 HTTP/1.1" 200 None 2026-03-31 16:05:30,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:05:30,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 16:05:30,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:05:30,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94dc16a4c026e16349a23 HTTP/1.1" 200 None 2026-03-31 16:05:30,540 - lava_callback - ERROR - Error processing callback for node 69c94dc16a4c026e16349a23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50682 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:06:50,610 - lava_callback - INFO - Received callback for node 69c94e0c6a4c026e16349a2e INFO: 172.19.0.1:52570 - "POST /node/69c94e0c6a4c026e16349a2e HTTP/1.0" 202 Accepted 2026-03-31 16:06:50,616 - lava_callback - INFO - Starting async job submit for node 69c94e0c6a4c026e16349a2e 2026-03-31 16:06:50,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:06:50,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94e0c6a4c026e16349a2e HTTP/1.1" 200 None 2026-03-31 16:06:50,633 - lava_callback - DEBUG - Processing node 69c94e0c6a4c026e16349a2e: baseline-arm64-baylibre 2026-03-31 16:06:50,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94e0c6a4c026e16349a2e 2026-03-31 16:06:50,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:06:50,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94e0c6a4c026e16349a2e HTTP/1.1" 200 None 2026-03-31 16:06:50,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:06:50,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 16:06:50,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:06:50,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94e0c6a4c026e16349a2e HTTP/1.1" 200 None 2026-03-31 16:06:50,691 - lava_callback - ERROR - Error processing callback for node 69c94e0c6a4c026e16349a2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33100 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:08:05,351 - lava_callback - INFO - Received callback for node 69c94e5f6a4c026e16349a3e INFO: 172.19.0.1:52878 - "POST /node/69c94e5f6a4c026e16349a3e HTTP/1.0" 202 Accepted 2026-03-31 16:08:05,356 - lava_callback - INFO - Starting async job submit for node 69c94e5f6a4c026e16349a3e 2026-03-31 16:08:05,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:08:05,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94e5f6a4c026e16349a3e HTTP/1.1" 200 None 2026-03-31 16:08:05,373 - lava_callback - DEBUG - Processing node 69c94e5f6a4c026e16349a3e: baseline-arm64-clabbe 2026-03-31 16:08:05,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94e5f6a4c026e16349a3e 2026-03-31 16:08:05,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:08:05,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94e5f6a4c026e16349a3e HTTP/1.1" 200 None 2026-03-31 16:08:05,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:08:05,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:08:05,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:08:05,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94e5f6a4c026e16349a3e HTTP/1.1" 200 None 2026-03-31 16:08:05,443 - lava_callback - ERROR - Error processing callback for node 69c94e5f6a4c026e16349a3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51098 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:09:30,684 - lava_callback - INFO - Received callback for node 69c94ea86a4c026e16349a4c INFO: 172.19.0.1:48686 - "POST /node/69c94ea86a4c026e16349a4c HTTP/1.0" 202 Accepted 2026-03-31 16:09:30,689 - lava_callback - INFO - Starting async job submit for node 69c94ea86a4c026e16349a4c 2026-03-31 16:09:30,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:09:30,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94ea86a4c026e16349a4c HTTP/1.1" 200 None 2026-03-31 16:09:30,716 - lava_callback - DEBUG - Processing node 69c94ea86a4c026e16349a4c: baseline-arm64-baylibre 2026-03-31 16:09:30,716 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94ea86a4c026e16349a4c 2026-03-31 16:09:30,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:09:30,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94ea86a4c026e16349a4c HTTP/1.1" 200 None 2026-03-31 16:09:30,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:09:30,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 16:09:30,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:09:30,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94ea86a4c026e16349a4c HTTP/1.1" 200 None 2026-03-31 16:09:30,767 - lava_callback - ERROR - Error processing callback for node 69c94ea86a4c026e16349a4c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42996 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:10:45,495 - lava_callback - INFO - Received callback for node 69c94efa6a4c026e16349a5c INFO: 172.19.0.1:34984 - "POST /node/69c94efa6a4c026e16349a5c HTTP/1.0" 202 Accepted 2026-03-31 16:10:45,501 - lava_callback - INFO - Starting async job submit for node 69c94efa6a4c026e16349a5c 2026-03-31 16:10:45,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:10:45,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94efa6a4c026e16349a5c HTTP/1.1" 200 None 2026-03-31 16:10:45,542 - lava_callback - DEBUG - Processing node 69c94efa6a4c026e16349a5c: baseline-arm64-clabbe 2026-03-31 16:10:45,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94efa6a4c026e16349a5c 2026-03-31 16:10:45,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:10:45,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94efa6a4c026e16349a5c HTTP/1.1" 200 None 2026-03-31 16:10:45,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:10:45,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:10:45,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:10:45,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94efa6a4c026e16349a5c HTTP/1.1" 200 None 2026-03-31 16:10:45,595 - lava_callback - ERROR - Error processing callback for node 69c94efa6a4c026e16349a5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40684 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:12:05,567 - lava_callback - INFO - Received callback for node 69c94f486a4c026e16349a70 INFO: 172.19.0.1:55280 - "POST /node/69c94f486a4c026e16349a70 HTTP/1.0" 202 Accepted 2026-03-31 16:12:05,572 - lava_callback - INFO - Starting async job submit for node 69c94f486a4c026e16349a70 2026-03-31 16:12:05,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:12:05,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94f486a4c026e16349a70 HTTP/1.1" 200 None 2026-03-31 16:12:05,596 - lava_callback - DEBUG - Processing node 69c94f486a4c026e16349a70: baseline-arm64-clabbe 2026-03-31 16:12:05,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94f486a4c026e16349a70 2026-03-31 16:12:05,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:12:05,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94f486a4c026e16349a70 HTTP/1.1" 200 None 2026-03-31 16:12:05,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:12:05,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:12:05,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:12:05,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94f486a4c026e16349a70 HTTP/1.1" 200 None 2026-03-31 16:12:05,651 - lava_callback - ERROR - Error processing callback for node 69c94f486a4c026e16349a70: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41334 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:13:25,618 - lava_callback - INFO - Received callback for node 69c94f966a4c026e16349a7e INFO: 172.19.0.1:38542 - "POST /node/69c94f966a4c026e16349a7e HTTP/1.0" 202 Accepted 2026-03-31 16:13:25,626 - lava_callback - INFO - Starting async job submit for node 69c94f966a4c026e16349a7e 2026-03-31 16:13:25,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:13:25,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94f966a4c026e16349a7e HTTP/1.1" 200 None 2026-03-31 16:13:25,648 - lava_callback - DEBUG - Processing node 69c94f966a4c026e16349a7e: baseline-arm64-clabbe 2026-03-31 16:13:25,648 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94f966a4c026e16349a7e 2026-03-31 16:13:25,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:13:25,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94f966a4c026e16349a7e HTTP/1.1" 200 None 2026-03-31 16:13:25,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:13:25,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 16:13:25,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:13:25,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94f966a4c026e16349a7e HTTP/1.1" 200 None 2026-03-31 16:13:25,724 - lava_callback - ERROR - Error processing callback for node 69c94f966a4c026e16349a7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1] INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.19.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50622 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.19.0.1:50626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36410 - "GET /wiki HTTP/1.0" 404 Not Found INFO: 172.19.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58938 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:33:06,297 - lava_callback - INFO - Received callback for node 69cb86a32a64fa9aff936f00 2026-03-31 08:33:06,305 - lava_callback - INFO - Starting async job submit for node 69cb86a32a64fa9aff936f00 INFO: 172.19.0.1:49410 - "POST /node/69cb86a32a64fa9aff936f00 HTTP/1.0" 202 Accepted 2026-03-31 08:33:06,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:06,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a32a64fa9aff936f00 HTTP/1.1" 200 None 2026-03-31 08:33:06,347 - lava_callback - DEBUG - Processing node 69cb86a32a64fa9aff936f00: baseline-x86 INFO: 172.19.0.1:49416 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:33:06,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:06,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:06,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f00/log.txt.gz 2026-03-31 08:33:06,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:06,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:06,977 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f00/lava-logs.yaml 2026-03-31 08:33:06,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:07,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:07,156 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f00/lava_callback.json.gz 2026-03-31 08:33:07,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:07,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a32a64fa9aff936f00 HTTP/1.1" 200 None 2026-03-31 08:33:07,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:07,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83602a64fa9aff936925 HTTP/1.1" 200 None 2026-03-31 08:33:07,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:07,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb86a32a64fa9aff936f00 HTTP/1.1" 200 None 2026-03-31 08:33:07,253 - lava_callback - INFO - Completed processing callback for node 69cb86a32a64fa9aff936f00 2026-03-31 08:33:07,368 - lava_callback - INFO - Received callback for node 69cb86a52a64fa9aff936f09 INFO: 172.19.0.1:49418 - "POST /node/69cb86a52a64fa9aff936f09 HTTP/1.0" 202 Accepted 2026-03-31 08:33:07,375 - lava_callback - INFO - Starting async job submit for node 69cb86a52a64fa9aff936f09 2026-03-31 08:33:07,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:07,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a52a64fa9aff936f09 HTTP/1.1" 200 None 2026-03-31 08:33:07,409 - lava_callback - DEBUG - Processing node 69cb86a52a64fa9aff936f09: baseline-x86 2026-03-31 08:33:07,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:07,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:07,923 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a52a64fa9aff936f09/log.txt.gz 2026-03-31 08:33:08,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:08,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:08,208 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a52a64fa9aff936f09/lava-logs.yaml 2026-03-31 08:33:08,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:33:08,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:33:08,366 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a52a64fa9aff936f09/lava_callback.json.gz 2026-03-31 08:33:08,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:08,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a52a64fa9aff936f09 HTTP/1.1" 200 None 2026-03-31 08:33:08,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:08,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83602a64fa9aff936925 HTTP/1.1" 200 None 2026-03-31 08:33:08,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:08,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb86a52a64fa9aff936f09 HTTP/1.1" 200 None 2026-03-31 08:33:08,449 - lava_callback - INFO - Completed processing callback for node 69cb86a52a64fa9aff936f09 INFO: 172.19.0.1:37394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50572 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:33:37,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:33:37,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:55450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43904 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:34:01,107 - lava_callback - INFO - Received callback for node 69cb86a32a64fa9aff936f03 INFO: 172.19.0.1:60388 - "POST /node/69cb86a32a64fa9aff936f03 HTTP/1.0" 202 Accepted 2026-03-31 08:34:01,113 - lava_callback - INFO - Starting async job submit for node 69cb86a32a64fa9aff936f03 2026-03-31 08:34:01,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:01,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a32a64fa9aff936f03 HTTP/1.1" 200 None 2026-03-31 08:34:01,148 - lava_callback - DEBUG - Processing node 69cb86a32a64fa9aff936f03: baseline-x86 2026-03-31 08:34:01,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:01,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:01,555 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f03/log.txt.gz 2026-03-31 08:34:01,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:01,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:01,930 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f03/lava-logs.yaml 2026-03-31 08:34:01,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:02,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:02,077 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a32a64fa9aff936f03/lava_callback.json.gz 2026-03-31 08:34:02,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:02,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a32a64fa9aff936f03 HTTP/1.1" 200 None 2026-03-31 08:34:02,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:02,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83602a64fa9aff936925 HTTP/1.1" 200 None 2026-03-31 08:34:02,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:02,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb86a32a64fa9aff936f03 HTTP/1.1" 200 None 2026-03-31 08:34:02,179 - lava_callback - INFO - Completed processing callback for node 69cb86a32a64fa9aff936f03 2026-03-31 08:34:04,446 - lava_callback - INFO - Received callback for node 69cb86a42a64fa9aff936f06 INFO: 172.19.0.1:60402 - "POST /node/69cb86a42a64fa9aff936f06 HTTP/1.0" 202 Accepted 2026-03-31 08:34:04,452 - lava_callback - INFO - Starting async job submit for node 69cb86a42a64fa9aff936f06 2026-03-31 08:34:04,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:04,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a42a64fa9aff936f06 HTTP/1.1" 200 None 2026-03-31 08:34:04,502 - lava_callback - DEBUG - Processing node 69cb86a42a64fa9aff936f06: baseline-x86 2026-03-31 08:34:04,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:04,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:04,868 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a42a64fa9aff936f06/log.txt.gz 2026-03-31 08:34:05,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:05,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:05,270 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a42a64fa9aff936f06/lava-logs.yaml 2026-03-31 08:34:05,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:34:05,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:34:05,450 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a42a64fa9aff936f06/lava_callback.json.gz 2026-03-31 08:34:05,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:05,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a42a64fa9aff936f06 HTTP/1.1" 200 None 2026-03-31 08:34:05,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:05,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83602a64fa9aff936925 HTTP/1.1" 200 None 2026-03-31 08:34:05,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:05,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb86a42a64fa9aff936f06 HTTP/1.1" 200 None 2026-03-31 08:34:05,531 - lava_callback - INFO - Completed processing callback for node 69cb86a42a64fa9aff936f06 INFO: 172.19.0.1:60412 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:34:07,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:34:07,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:32920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:32924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57536 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:36:46,096 - lava_callback - INFO - Received callback for node 69c8e49c6a4c026e163483b6 INFO: 172.19.0.1:33220 - "POST /node/69c8e49c6a4c026e163483b6 HTTP/1.0" 202 Accepted 2026-03-31 08:36:46,104 - lava_callback - INFO - Starting async job submit for node 69c8e49c6a4c026e163483b6 2026-03-31 08:36:46,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49c6a4c026e163483b6 HTTP/1.1" 200 None 2026-03-31 08:36:46,150 - lava_callback - DEBUG - Processing node 69c8e49c6a4c026e163483b6: baseline-arm-baylibre 2026-03-31 08:36:46,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e49c6a4c026e163483b6 2026-03-31 08:36:46,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49c6a4c026e163483b6 HTTP/1.1" 200 None 2026-03-31 08:36:46,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:36:46,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e49c6a4c026e163483b6 HTTP/1.1" 200 None 2026-03-31 08:36:46,274 - lava_callback - ERROR - Error processing callback for node 69c8e49c6a4c026e163483b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33240 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:36:46,764 - lava_callback - INFO - Received callback for node 69c8e49b6a4c026e163483b3 INFO: 172.19.0.1:33252 - "POST /node/69c8e49b6a4c026e163483b3 HTTP/1.0" 202 Accepted 2026-03-31 08:36:46,772 - lava_callback - INFO - Starting async job submit for node 69c8e49b6a4c026e163483b3 2026-03-31 08:36:46,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49b6a4c026e163483b3 HTTP/1.1" 200 None 2026-03-31 08:36:46,803 - lava_callback - DEBUG - Processing node 69c8e49b6a4c026e163483b3: baseline-arm-baylibre 2026-03-31 08:36:46,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e49b6a4c026e163483b3 2026-03-31 08:36:46,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49b6a4c026e163483b3 HTTP/1.1" 200 None 2026-03-31 08:36:46,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:36:46,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:36:46,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e49b6a4c026e163483b3 HTTP/1.1" 200 None 2026-03-31 08:36:46,889 - lava_callback - ERROR - Error processing callback for node 69c8e49b6a4c026e163483b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55646 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:37:06,055 - lava_callback - INFO - Received callback for node 69c8e49d6a4c026e163483b9 INFO: 172.19.0.1:60096 - "POST /node/69c8e49d6a4c026e163483b9 HTTP/1.0" 202 Accepted 2026-03-31 08:37:06,061 - lava_callback - INFO - Starting async job submit for node 69c8e49d6a4c026e163483b9 2026-03-31 08:37:06,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49d6a4c026e163483b9 HTTP/1.1" 200 None 2026-03-31 08:37:06,077 - lava_callback - DEBUG - Processing node 69c8e49d6a4c026e163483b9: baseline-arm-baylibre 2026-03-31 08:37:06,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e49d6a4c026e163483b9 2026-03-31 08:37:06,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49d6a4c026e163483b9 HTTP/1.1" 200 None 2026-03-31 08:37:06,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:06,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e49d6a4c026e163483b9 HTTP/1.1" 200 None 2026-03-31 08:37:06,153 - lava_callback - ERROR - Error processing callback for node 69c8e49d6a4c026e163483b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:06,246 - lava_callback - INFO - Received callback for node 69c8e4aa6a4c026e163483de INFO: 172.19.0.1:60106 - "POST /node/69c8e4aa6a4c026e163483de HTTP/1.0" 202 Accepted 2026-03-31 08:37:06,256 - lava_callback - INFO - Starting async job submit for node 69c8e4aa6a4c026e163483de 2026-03-31 08:37:06,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4aa6a4c026e163483de HTTP/1.1" 200 None 2026-03-31 08:37:06,314 - lava_callback - DEBUG - Processing node 69c8e4aa6a4c026e163483de: baseline-arm-clabbe 2026-03-31 08:37:06,315 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4aa6a4c026e163483de 2026-03-31 08:37:06,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4aa6a4c026e163483de HTTP/1.1" 200 None 2026-03-31 08:37:06,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:06,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4aa6a4c026e163483de HTTP/1.1" 200 None 2026-03-31 08:37:06,416 - lava_callback - ERROR - Error processing callback for node 69c8e4aa6a4c026e163483de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60120 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:37:06,731 - lava_callback - INFO - Received callback for node 69c8e49f6a4c026e163483bc INFO: 172.19.0.1:60134 - "POST /node/69c8e49f6a4c026e163483bc HTTP/1.0" 202 Accepted 2026-03-31 08:37:06,745 - lava_callback - INFO - Starting async job submit for node 69c8e49f6a4c026e163483bc 2026-03-31 08:37:06,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49f6a4c026e163483bc HTTP/1.1" 200 None 2026-03-31 08:37:06,779 - lava_callback - DEBUG - Processing node 69c8e49f6a4c026e163483bc: baseline-arm-baylibre 2026-03-31 08:37:06,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e49f6a4c026e163483bc 2026-03-31 08:37:06,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e49f6a4c026e163483bc HTTP/1.1" 200 None 2026-03-31 08:37:06,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:06,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e49f6a4c026e163483bc HTTP/1.1" 200 None 2026-03-31 08:37:06,869 - lava_callback - ERROR - Error processing callback for node 69c8e49f6a4c026e163483bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:06,929 - lava_callback - INFO - Received callback for node 69c8e4ae6a4c026e16348405 INFO: 172.19.0.1:60138 - "POST /node/69c8e4ae6a4c026e16348405 HTTP/1.0" 202 Accepted 2026-03-31 08:37:06,942 - lava_callback - INFO - Starting async job submit for node 69c8e4ae6a4c026e16348405 2026-03-31 08:37:06,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:06,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4ae6a4c026e16348405 HTTP/1.1" 200 None 2026-03-31 08:37:06,993 - lava_callback - DEBUG - Processing node 69c8e4ae6a4c026e16348405: baseline-arm-clabbe 2026-03-31 08:37:06,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4ae6a4c026e16348405 2026-03-31 08:37:06,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4ae6a4c026e16348405 HTTP/1.1" 200 None 2026-03-31 08:37:07,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:07,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4ae6a4c026e16348405 HTTP/1.1" 200 None 2026-03-31 08:37:07,155 - lava_callback - ERROR - Error processing callback for node 69c8e4ae6a4c026e16348405: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:07,410 - lava_callback - INFO - Received callback for node 69c8e4a26a4c026e163483c7 INFO: 172.19.0.1:60150 - "POST /node/69c8e4a26a4c026e163483c7 HTTP/1.0" 202 Accepted 2026-03-31 08:37:07,419 - lava_callback - INFO - Starting async job submit for node 69c8e4a26a4c026e163483c7 2026-03-31 08:37:07,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4a26a4c026e163483c7 HTTP/1.1" 200 None 2026-03-31 08:37:07,446 - lava_callback - DEBUG - Processing node 69c8e4a26a4c026e163483c7: baseline-arm-baylibre 2026-03-31 08:37:07,447 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4a26a4c026e163483c7 2026-03-31 08:37:07,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4a26a4c026e163483c7 HTTP/1.1" 200 None 2026-03-31 08:37:07,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:07,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4a26a4c026e163483c7 HTTP/1.1" 200 None 2026-03-31 08:37:07,533 - lava_callback - ERROR - Error processing callback for node 69c8e4a26a4c026e163483c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:07,624 - lava_callback - INFO - Received callback for node 69c8e4af6a4c026e1634840c INFO: 172.19.0.1:60162 - "POST /node/69c8e4af6a4c026e1634840c HTTP/1.0" 202 Accepted 2026-03-31 08:37:07,635 - lava_callback - INFO - Starting async job submit for node 69c8e4af6a4c026e1634840c 2026-03-31 08:37:07,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4af6a4c026e1634840c HTTP/1.1" 200 None 2026-03-31 08:37:07,684 - lava_callback - DEBUG - Processing node 69c8e4af6a4c026e1634840c: baseline-arm-clabbe 2026-03-31 08:37:07,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4af6a4c026e1634840c 2026-03-31 08:37:07,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4af6a4c026e1634840c HTTP/1.1" 200 None 2026-03-31 08:37:07,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:07,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:07,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4af6a4c026e1634840c HTTP/1.1" 200 None 2026-03-31 08:37:07,845 - lava_callback - ERROR - Error processing callback for node 69c8e4af6a4c026e1634840c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:08,318 - lava_callback - INFO - Received callback for node 69c8e4b06a4c026e16348410 INFO: 172.19.0.1:60174 - "POST /node/69c8e4b06a4c026e16348410 HTTP/1.0" 202 Accepted 2026-03-31 08:37:08,333 - lava_callback - INFO - Starting async job submit for node 69c8e4b06a4c026e16348410 2026-03-31 08:37:08,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:08,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4b06a4c026e16348410 HTTP/1.1" 200 None 2026-03-31 08:37:08,365 - lava_callback - DEBUG - Processing node 69c8e4b06a4c026e16348410: baseline-arm-clabbe 2026-03-31 08:37:08,365 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4b06a4c026e16348410 2026-03-31 08:37:08,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:08,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4b06a4c026e16348410 HTTP/1.1" 200 None 2026-03-31 08:37:08,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:08,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:08,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:08,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4b06a4c026e16348410 HTTP/1.1" 200 None 2026-03-31 08:37:08,480 - lava_callback - ERROR - Error processing callback for node 69c8e4b06a4c026e16348410: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:37:11,669 - lava_callback - INFO - Received callback for node 69cb87a42a64fa9aff936f6d INFO: 172.19.0.1:34136 - "POST /node/69cb87a42a64fa9aff936f6d HTTP/1.0" 202 Accepted 2026-03-31 08:37:11,682 - lava_callback - INFO - Starting async job submit for node 69cb87a42a64fa9aff936f6d 2026-03-31 08:37:11,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:11,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a42a64fa9aff936f6d HTTP/1.1" 200 None 2026-03-31 08:37:11,718 - lava_callback - DEBUG - Processing node 69cb87a42a64fa9aff936f6d: baseline-arm 2026-03-31 08:37:11,726 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:37:11,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:11,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:11,989 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a42a64fa9aff936f6d/lava-logs.yaml 2026-03-31 08:37:11,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:12,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:12,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a42a64fa9aff936f6d/lava_callback.json.gz 2026-03-31 08:37:12,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:12,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a42a64fa9aff936f6d HTTP/1.1" 200 None 2026-03-31 08:37:12,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:12,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:37:12,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:12,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a42a64fa9aff936f6d HTTP/1.1" 200 None 2026-03-31 08:37:12,277 - lava_callback - INFO - Completed processing callback for node 69cb87a42a64fa9aff936f6d INFO: 172.19.0.1:34152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34160 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:37:26,304 - lava_callback - INFO - Received callback for node 69c8e4b36a4c026e1634841f INFO: 172.19.0.1:43268 - "POST /node/69c8e4b36a4c026e1634841f HTTP/1.0" 202 Accepted 2026-03-31 08:37:26,315 - lava_callback - INFO - Starting async job submit for node 69c8e4b36a4c026e1634841f 2026-03-31 08:37:26,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:26,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4b36a4c026e1634841f HTTP/1.1" 200 None 2026-03-31 08:37:26,341 - lava_callback - DEBUG - Processing node 69c8e4b36a4c026e1634841f: baseline-arm-clabbe 2026-03-31 08:37:26,341 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e4b36a4c026e1634841f 2026-03-31 08:37:26,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:26,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e4b36a4c026e1634841f HTTP/1.1" 200 None 2026-03-31 08:37:26,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:26,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 08:37:26,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:26,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e4b36a4c026e1634841f HTTP/1.1" 200 None 2026-03-31 08:37:26,412 - lava_callback - ERROR - Error processing callback for node 69c8e4b36a4c026e1634841f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43282 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:37:27,507 - lava_callback - INFO - Received callback for node 69cb87a82a64fa9aff936f79 INFO: 172.19.0.1:43286 - "POST /node/69cb87a82a64fa9aff936f79 HTTP/1.0" 202 Accepted 2026-03-31 08:37:27,558 - lava_callback - INFO - Starting async job submit for node 69cb87a82a64fa9aff936f79 2026-03-31 08:37:27,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:27,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a82a64fa9aff936f79 HTTP/1.1" 200 None 2026-03-31 08:37:27,591 - lava_callback - DEBUG - Processing node 69cb87a82a64fa9aff936f79: baseline-arm 2026-03-31 08:37:27,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:28,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:28,203 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a82a64fa9aff936f79/log.txt.gz 2026-03-31 08:37:28,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:28,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:28,587 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a82a64fa9aff936f79/lava-logs.yaml 2026-03-31 08:37:28,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:28,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:28,909 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a82a64fa9aff936f79/lava_callback.json.gz 2026-03-31 08:37:28,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:28,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a82a64fa9aff936f79 HTTP/1.1" 200 None 2026-03-31 08:37:28,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:29,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:37:29,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:29,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a82a64fa9aff936f79 HTTP/1.1" 200 None 2026-03-31 08:37:29,117 - lava_callback - INFO - Completed processing callback for node 69cb87a82a64fa9aff936f79 INFO: 172.19.0.1:52892 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:37:37,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:37,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:37:42,840 - lava_callback - INFO - Received callback for node 69cb87a52a64fa9aff936f70 INFO: 172.19.0.1:52990 - "POST /node/69cb87a52a64fa9aff936f70 HTTP/1.0" 202 Accepted 2026-03-31 08:37:42,890 - lava_callback - INFO - Starting async job submit for node 69cb87a52a64fa9aff936f70 2026-03-31 08:37:42,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:42,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a52a64fa9aff936f70 HTTP/1.1" 200 None 2026-03-31 08:37:42,923 - lava_callback - DEBUG - Processing node 69cb87a52a64fa9aff936f70: baseline-arm 2026-03-31 08:37:43,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:43,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:43,162 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a52a64fa9aff936f70/log.txt.gz 2026-03-31 08:37:43,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:43,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:43,395 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a52a64fa9aff936f70/lava-logs.yaml 2026-03-31 08:37:43,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:37:43,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:37:43,515 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a52a64fa9aff936f70/lava_callback.json.gz 2026-03-31 08:37:43,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:43,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a52a64fa9aff936f70 HTTP/1.1" 200 None 2026-03-31 08:37:43,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:43,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:37:43,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:37:43,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a52a64fa9aff936f70 HTTP/1.1" 200 None 2026-03-31 08:37:43,625 - lava_callback - INFO - Completed processing callback for node 69cb87a52a64fa9aff936f70 INFO: 172.19.0.1:52996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53262 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:38:07,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:07,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:42786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42788 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:38:20,460 - lava_callback - INFO - Received callback for node 69cb87a42a64fa9aff936f6a INFO: 172.19.0.1:42646 - "POST /node/69cb87a42a64fa9aff936f6a HTTP/1.0" 202 Accepted 2026-03-31 08:38:20,467 - lava_callback - INFO - Starting async job submit for node 69cb87a42a64fa9aff936f6a 2026-03-31 08:38:20,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:20,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a42a64fa9aff936f6a HTTP/1.1" 200 None 2026-03-31 08:38:20,513 - lava_callback - DEBUG - Processing node 69cb87a42a64fa9aff936f6a: baseline-arm 2026-03-31 08:38:20,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:21,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:21,038 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a42a64fa9aff936f6a/log.txt.gz 2026-03-31 08:38:21,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:21,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:21,445 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a42a64fa9aff936f6a/lava-logs.yaml 2026-03-31 08:38:21,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:21,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:21,694 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a42a64fa9aff936f6a/lava_callback.json.gz 2026-03-31 08:38:21,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:21,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a42a64fa9aff936f6a HTTP/1.1" 200 None 2026-03-31 08:38:21,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:21,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:21,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:21,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a42a64fa9aff936f6a HTTP/1.1" 200 None 2026-03-31 08:38:21,831 - lava_callback - INFO - Completed processing callback for node 69cb87a42a64fa9aff936f6a 2026-03-31 08:38:23,951 - lava_callback - INFO - Received callback for node 69cb87a72a64fa9aff936f76 INFO: 172.19.0.1:42654 - "POST /node/69cb87a72a64fa9aff936f76 HTTP/1.0" 202 Accepted 2026-03-31 08:38:23,960 - lava_callback - INFO - Starting async job submit for node 69cb87a72a64fa9aff936f76 2026-03-31 08:38:23,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:24,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a72a64fa9aff936f76 HTTP/1.1" 200 None 2026-03-31 08:38:24,001 - lava_callback - DEBUG - Processing node 69cb87a72a64fa9aff936f76: baseline-arm 2026-03-31 08:38:24,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:24,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:24,425 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a72a64fa9aff936f76/log.txt.gz 2026-03-31 08:38:24,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:24,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:24,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a72a64fa9aff936f76/lava-logs.yaml 2026-03-31 08:38:24,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:24,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:24,959 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a72a64fa9aff936f76/lava_callback.json.gz 2026-03-31 08:38:24,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:25,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a72a64fa9aff936f76 HTTP/1.1" 200 None 2026-03-31 08:38:25,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:25,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:25,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:25,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a72a64fa9aff936f76 HTTP/1.1" 200 None 2026-03-31 08:38:25,059 - lava_callback - INFO - Completed processing callback for node 69cb87a72a64fa9aff936f76 INFO: 172.19.0.1:42656 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:38:36,031 - lava_callback - INFO - Received callback for node 69cb87cd2a64fa9aff937011 INFO: 172.19.0.1:52296 - "POST /node/69cb87cd2a64fa9aff937011 HTTP/1.0" 202 Accepted 2026-03-31 08:38:36,077 - lava_callback - INFO - Starting async job submit for node 69cb87cd2a64fa9aff937011 2026-03-31 08:38:36,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:36,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87cd2a64fa9aff937011 HTTP/1.1" 200 None 2026-03-31 08:38:36,118 - lava_callback - DEBUG - Processing node 69cb87cd2a64fa9aff937011: baseline-arm-pengutronix 2026-03-31 08:38:36,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:36,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:36,368 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cd2a64fa9aff937011/log.txt.gz 2026-03-31 08:38:36,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:52312 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:38:36,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:36,754 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cd2a64fa9aff937011/lava-logs.yaml 2026-03-31 08:38:36,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:36,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:36,885 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cd2a64fa9aff937011/lava_callback.json.gz 2026-03-31 08:38:36,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:36,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87cd2a64fa9aff937011 HTTP/1.1" 200 None 2026-03-31 08:38:36,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:36,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:36,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:36,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87cd2a64fa9aff937011 HTTP/1.1" 200 None 2026-03-31 08:38:36,967 - lava_callback - INFO - Completed processing callback for node 69cb87cd2a64fa9aff937011 2026-03-31 08:38:37,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:37,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:38:38,466 - lava_callback - INFO - Received callback for node 69cb87c92a64fa9aff937005 INFO: 172.19.0.1:52324 - "POST /node/69cb87c92a64fa9aff937005 HTTP/1.0" 202 Accepted 2026-03-31 08:38:38,472 - lava_callback - INFO - Starting async job submit for node 69cb87c92a64fa9aff937005 2026-03-31 08:38:38,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:38,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c92a64fa9aff937005 HTTP/1.1" 200 None 2026-03-31 08:38:38,507 - lava_callback - DEBUG - Processing node 69cb87c92a64fa9aff937005: baseline-arm-pengutronix 2026-03-31 08:38:38,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:38,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:38,797 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87c92a64fa9aff937005/log.txt.gz 2026-03-31 08:38:38,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:39,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:39,073 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87c92a64fa9aff937005/lava-logs.yaml 2026-03-31 08:38:39,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:39,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:39,238 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87c92a64fa9aff937005/lava_callback.json.gz 2026-03-31 08:38:39,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:39,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c92a64fa9aff937005 HTTP/1.1" 200 None 2026-03-31 08:38:39,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:39,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:39,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:39,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87c92a64fa9aff937005 HTTP/1.1" 200 None 2026-03-31 08:38:39,333 - lava_callback - INFO - Completed processing callback for node 69cb87c92a64fa9aff937005 2026-03-31 08:38:39,933 - lava_callback - INFO - Received callback for node 69cb87ca2a64fa9aff937009 INFO: 172.19.0.1:43030 - "POST /node/69cb87ca2a64fa9aff937009 HTTP/1.0" 202 Accepted 2026-03-31 08:38:39,939 - lava_callback - INFO - Starting async job submit for node 69cb87ca2a64fa9aff937009 2026-03-31 08:38:39,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:39,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ca2a64fa9aff937009 HTTP/1.1" 200 None 2026-03-31 08:38:39,972 - lava_callback - DEBUG - Processing node 69cb87ca2a64fa9aff937009: baseline-arm-pengutronix 2026-03-31 08:38:40,131 - lava_callback - INFO - Received callback for node 69cb87be2a64fa9aff936fdd 2026-03-31 08:38:40,137 - lava_callback - INFO - Starting async job submit for node 69cb87be2a64fa9aff936fdd INFO: 172.19.0.1:43038 - "POST /node/69cb87be2a64fa9aff936fdd HTTP/1.0" 202 Accepted 2026-03-31 08:38:40,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:40,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:40,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87be2a64fa9aff936fdd HTTP/1.1" 200 None 2026-03-31 08:38:40,185 - lava_callback - DEBUG - Processing node 69cb87be2a64fa9aff936fdd: baseline-arm-clabbe 2026-03-31 08:38:40,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:40,375 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87ca2a64fa9aff937009/log.txt.gz 2026-03-31 08:38:40,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:40,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:40,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:40,702 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87be2a64fa9aff936fdd/log.txt.gz 2026-03-31 08:38:40,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:40,778 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87ca2a64fa9aff937009/lava-logs.yaml 2026-03-31 08:38:40,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:40,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:41,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:41,020 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87ca2a64fa9aff937009/lava_callback.json.gz 2026-03-31 08:38:41,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:41,059 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87be2a64fa9aff936fdd/lava-logs.yaml 2026-03-31 08:38:41,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ca2a64fa9aff937009 HTTP/1.1" 200 None 2026-03-31 08:38:41,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:41,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:41,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87ca2a64fa9aff937009 HTTP/1.1" 200 None 2026-03-31 08:38:41,122 - lava_callback - INFO - Completed processing callback for node 69cb87ca2a64fa9aff937009 2026-03-31 08:38:41,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:41,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87be2a64fa9aff936fdd/lava_callback.json.gz 2026-03-31 08:38:41,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87be2a64fa9aff936fdd HTTP/1.1" 200 None 2026-03-31 08:38:41,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:41,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:41,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87be2a64fa9aff936fdd HTTP/1.1" 200 None 2026-03-31 08:38:41,371 - lava_callback - INFO - Completed processing callback for node 69cb87be2a64fa9aff936fdd INFO: 172.19.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43052 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:38:50,692 - lava_callback - INFO - Received callback for node 69cb87b82a64fa9aff936faa INFO: 172.19.0.1:52502 - "POST /node/69cb87b82a64fa9aff936faa HTTP/1.0" 202 Accepted 2026-03-31 08:38:50,699 - lava_callback - INFO - Starting async job submit for node 69cb87b82a64fa9aff936faa 2026-03-31 08:38:50,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:50,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b82a64fa9aff936faa HTTP/1.1" 200 None 2026-03-31 08:38:50,751 - lava_callback - DEBUG - Processing node 69cb87b82a64fa9aff936faa: baseline-arm-broonie 2026-03-31 08:38:51,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:51,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:51,211 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b82a64fa9aff936faa/log.txt.gz 2026-03-31 08:38:51,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:51,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:51,637 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b82a64fa9aff936faa/lava-logs.yaml 2026-03-31 08:38:51,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:38:51,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:38:51,811 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b82a64fa9aff936faa/lava_callback.json.gz 2026-03-31 08:38:51,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:51,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b82a64fa9aff936faa HTTP/1.1" 200 None 2026-03-31 08:38:51,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:51,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:38:51,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:38:51,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87b82a64fa9aff936faa HTTP/1.1" 200 None 2026-03-31 08:38:51,933 - lava_callback - INFO - Completed processing callback for node 69cb87b82a64fa9aff936faa INFO: 172.19.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46508 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:39:07,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:07,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:39:09,283 - lava_callback - INFO - Received callback for node 69cb87b92a64fa9aff936faf INFO: 172.19.0.1:46520 - "POST /node/69cb87b92a64fa9aff936faf HTTP/1.0" 202 Accepted 2026-03-31 08:39:09,289 - lava_callback - INFO - Starting async job submit for node 69cb87b92a64fa9aff936faf 2026-03-31 08:39:09,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:09,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b92a64fa9aff936faf HTTP/1.1" 200 None 2026-03-31 08:39:09,328 - lava_callback - DEBUG - Processing node 69cb87b92a64fa9aff936faf: baseline-arm-broonie 2026-03-31 08:39:09,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:09,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:09,673 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b92a64fa9aff936faf/log.txt.gz 2026-03-31 08:39:09,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:09,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:09,994 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b92a64fa9aff936faf/lava-logs.yaml 2026-03-31 08:39:10,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:10,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:10,142 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b92a64fa9aff936faf/lava_callback.json.gz 2026-03-31 08:39:10,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:10,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b92a64fa9aff936faf HTTP/1.1" 200 None 2026-03-31 08:39:10,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:10,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:39:10,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:10,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87b92a64fa9aff936faf HTTP/1.1" 200 None 2026-03-31 08:39:10,241 - lava_callback - INFO - Completed processing callback for node 69cb87b92a64fa9aff936faf 2026-03-31 08:39:10,660 - lava_callback - INFO - Received callback for node 69cb87b62a64fa9aff936fa7 INFO: 172.19.0.1:48296 - "POST /node/69cb87b62a64fa9aff936fa7 HTTP/1.0" 202 Accepted 2026-03-31 08:39:10,666 - lava_callback - INFO - Starting async job submit for node 69cb87b62a64fa9aff936fa7 2026-03-31 08:39:10,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:10,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b62a64fa9aff936fa7 HTTP/1.1" 200 None 2026-03-31 08:39:10,707 - lava_callback - DEBUG - Processing node 69cb87b62a64fa9aff936fa7: baseline-arm-broonie 2026-03-31 08:39:10,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:11,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:11,047 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b62a64fa9aff936fa7/log.txt.gz 2026-03-31 08:39:11,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:11,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:11,346 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b62a64fa9aff936fa7/lava-logs.yaml 2026-03-31 08:39:11,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:11,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:11,493 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b62a64fa9aff936fa7/lava_callback.json.gz 2026-03-31 08:39:11,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:11,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b62a64fa9aff936fa7 HTTP/1.1" 200 None 2026-03-31 08:39:11,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:11,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:39:11,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:11,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87b62a64fa9aff936fa7 HTTP/1.1" 200 None 2026-03-31 08:39:11,576 - lava_callback - INFO - Completed processing callback for node 69cb87b62a64fa9aff936fa7 INFO: 172.19.0.1:48308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60312 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:39:29,526 - lava_callback - INFO - Received callback for node 69cb87b52a64fa9aff936fa4 INFO: 172.19.0.1:60322 - "POST /node/69cb87b52a64fa9aff936fa4 HTTP/1.0" 202 Accepted 2026-03-31 08:39:29,532 - lava_callback - INFO - Starting async job submit for node 69cb87b52a64fa9aff936fa4 2026-03-31 08:39:29,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:29,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b52a64fa9aff936fa4 HTTP/1.1" 200 None 2026-03-31 08:39:29,574 - lava_callback - DEBUG - Processing node 69cb87b52a64fa9aff936fa4: baseline-arm-broonie 2026-03-31 08:39:29,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:29,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:29,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b52a64fa9aff936fa4/log.txt.gz 2026-03-31 08:39:30,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:30,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:30,408 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b52a64fa9aff936fa4/lava-logs.yaml 2026-03-31 08:39:30,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:39:30,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:39:30,562 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cb87b52a64fa9aff936fa4/lava_callback.json.gz 2026-03-31 08:39:30,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:30,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87b52a64fa9aff936fa4 HTTP/1.1" 200 None 2026-03-31 08:39:30,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:30,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:39:30,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:30,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87b52a64fa9aff936fa4 HTTP/1.1" 200 None 2026-03-31 08:39:30,651 - lava_callback - INFO - Completed processing callback for node 69cb87b52a64fa9aff936fa4 INFO: 172.19.0.1:44268 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:39:37,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:39:37,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:34680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45128 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:40:00,407 - lava_callback - INFO - Received callback for node 69cb87e52a64fa9aff937046 INFO: 172.19.0.1:51216 - "POST /node/69cb87e52a64fa9aff937046 HTTP/1.0" 202 Accepted 2026-03-31 08:40:00,415 - lava_callback - INFO - Starting async job submit for node 69cb87e52a64fa9aff937046 2026-03-31 08:40:00,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:00,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e52a64fa9aff937046 HTTP/1.1" 200 None 2026-03-31 08:40:00,454 - lava_callback - DEBUG - Processing node 69cb87e52a64fa9aff937046: ltp-dio 2026-03-31 08:40:00,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:00,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:00,978 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69cb87e52a64fa9aff937046/log.txt.gz 2026-03-31 08:40:01,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:01,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:01,454 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69cb87e52a64fa9aff937046/lava-logs.yaml 2026-03-31 08:40:01,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:01,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:01,625 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69cb87e52a64fa9aff937046/lava_callback.json.gz 2026-03-31 08:40:01,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:01,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e52a64fa9aff937046 HTTP/1.1" 200 None 2026-03-31 08:40:01,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:01,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:40:01,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:01,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e52a64fa9aff937046 HTTP/1.1" 200 None 2026-03-31 08:40:01,758 - lava_callback - INFO - Completed processing callback for node 69cb87e52a64fa9aff937046 INFO: 172.19.0.1:51228 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:40:07,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:07,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:40:07,561 - lava_callback - INFO - Received callback for node 69cb87bf2a64fa9aff936fe0 INFO: 172.19.0.1:51238 - "POST /node/69cb87bf2a64fa9aff936fe0 HTTP/1.0" 202 Accepted 2026-03-31 08:40:07,567 - lava_callback - INFO - Starting async job submit for node 69cb87bf2a64fa9aff936fe0 2026-03-31 08:40:07,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:07,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87bf2a64fa9aff936fe0 HTTP/1.1" 200 None 2026-03-31 08:40:07,604 - lava_callback - DEBUG - Processing node 69cb87bf2a64fa9aff936fe0: baseline-arm-clabbe 2026-03-31 08:40:07,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:07,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:07,929 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87bf2a64fa9aff936fe0/log.txt.gz 2026-03-31 08:40:08,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:08,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:08,353 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87bf2a64fa9aff936fe0/lava-logs.yaml 2026-03-31 08:40:08,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:08,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:08,542 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87bf2a64fa9aff936fe0/lava_callback.json.gz 2026-03-31 08:40:08,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:08,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87bf2a64fa9aff936fe0 HTTP/1.1" 200 None 2026-03-31 08:40:08,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:08,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:40:08,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:08,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87bf2a64fa9aff936fe0 HTTP/1.1" 200 None 2026-03-31 08:40:08,627 - lava_callback - INFO - Completed processing callback for node 69cb87bf2a64fa9aff936fe0 INFO: 172.19.0.1:58894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40660 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:40:30,130 - lava_callback - INFO - Received callback for node 69cb87fa2a64fa9aff937085 INFO: 172.19.0.1:39768 - "POST /node/69cb87fa2a64fa9aff937085 HTTP/1.0" 202 Accepted 2026-03-31 08:40:30,136 - lava_callback - INFO - Starting async job submit for node 69cb87fa2a64fa9aff937085 2026-03-31 08:40:30,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:30,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87fa2a64fa9aff937085 HTTP/1.1" 200 None 2026-03-31 08:40:30,174 - lava_callback - DEBUG - Processing node 69cb87fa2a64fa9aff937085: baseline-arm 2026-03-31 08:40:30,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:30,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:30,498 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87fa2a64fa9aff937085/log.txt.gz 2026-03-31 08:40:30,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:30,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:30,792 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87fa2a64fa9aff937085/lava-logs.yaml 2026-03-31 08:40:30,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:40:30,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:40:30,977 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87fa2a64fa9aff937085/lava_callback.json.gz 2026-03-31 08:40:31,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:31,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87fa2a64fa9aff937085 HTTP/1.1" 200 None 2026-03-31 08:40:31,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:31,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83462a64fa9aff936905 HTTP/1.1" 200 None 2026-03-31 08:40:31,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:31,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87fa2a64fa9aff937085 HTTP/1.1" 200 None 2026-03-31 08:40:31,109 - lava_callback - INFO - Completed processing callback for node 69cb87fa2a64fa9aff937085 INFO: 172.19.0.1:39778 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:40:37,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:40:37,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:47726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44078 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:41:42,211 - lava_callback - INFO - Received callback for node 69cb87c32a64fa9aff936fec INFO: 172.19.0.1:35790 - "POST /node/69cb87c32a64fa9aff936fec HTTP/1.0" 202 Accepted 2026-03-31 08:41:42,220 - lava_callback - INFO - Starting async job submit for node 69cb87c32a64fa9aff936fec 2026-03-31 08:41:42,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:41:42,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c32a64fa9aff936fec HTTP/1.1" 200 None 2026-03-31 08:41:42,292 - lava_callback - DEBUG - Processing node 69cb87c32a64fa9aff936fec: baseline-arm-clabbe 2026-03-31 08:41:42,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:41:42,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:41:42,751 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c32a64fa9aff936fec/log.txt.gz 2026-03-31 08:41:42,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:41:43,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:41:43,255 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c32a64fa9aff936fec/lava-logs.yaml 2026-03-31 08:41:43,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:41:43,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:41:43,681 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c32a64fa9aff936fec/lava_callback.json.gz 2026-03-31 08:41:43,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:41:43,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c32a64fa9aff936fec HTTP/1.1" 200 None 2026-03-31 08:41:43,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:41:43,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:41:43,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:41:43,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87c32a64fa9aff936fec HTTP/1.1" 200 None 2026-03-31 08:41:43,807 - lava_callback - INFO - Completed processing callback for node 69cb87c32a64fa9aff936fec INFO: 172.19.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34512 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:42:07,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:07,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:42:09,634 - lava_callback - INFO - Received callback for node 69cb87de2a64fa9aff93703a INFO: 172.19.0.1:34514 - "POST /node/69cb87de2a64fa9aff93703a HTTP/1.0" 202 Accepted 2026-03-31 08:42:09,648 - lava_callback - INFO - Starting async job submit for node 69cb87de2a64fa9aff93703a 2026-03-31 08:42:09,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:09,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87de2a64fa9aff93703a HTTP/1.1" 200 None 2026-03-31 08:42:09,726 - lava_callback - DEBUG - Processing node 69cb87de2a64fa9aff93703a: ltp-cap_bounds 2026-03-31 08:42:10,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:10,681 - lava_callback - INFO - Received callback for node 69cb87e02a64fa9aff93703d INFO: 172.19.0.1:54582 - "POST /node/69cb87e02a64fa9aff93703d HTTP/1.0" 202 Accepted 2026-03-31 08:42:10,691 - lava_callback - INFO - Starting async job submit for node 69cb87e02a64fa9aff93703d 2026-03-31 08:42:10,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:10,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e02a64fa9aff93703d HTTP/1.1" 200 None 2026-03-31 08:42:10,771 - lava_callback - DEBUG - Processing node 69cb87e02a64fa9aff93703d: ltp-containers 2026-03-31 08:42:10,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:10,894 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cb87de2a64fa9aff93703a/log.txt.gz 2026-03-31 08:42:11,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:11,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:12,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:12,520 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69cb87e02a64fa9aff93703d/log.txt.gz 2026-03-31 08:42:12,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:12,888 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cb87de2a64fa9aff93703a/lava-logs.yaml 2026-03-31 08:42:12,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:12,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:13,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:13,766 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cb87de2a64fa9aff93703a/lava_callback.json.gz 2026-03-31 08:42:13,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:13,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87de2a64fa9aff93703a HTTP/1.1" 200 None 2026-03-31 08:42:13,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:13,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:42:13,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:13,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87de2a64fa9aff93703a HTTP/1.1" 200 None 2026-03-31 08:42:13,929 - lava_callback - INFO - Completed processing callback for node 69cb87de2a64fa9aff93703a 2026-03-31 08:42:14,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:14,156 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69cb87e02a64fa9aff93703d/lava-logs.yaml 2026-03-31 08:42:14,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:15,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:15,092 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69cb87e02a64fa9aff93703d/lava_callback.json.gz 2026-03-31 08:42:15,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:54590 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 08:42:15,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e02a64fa9aff93703d HTTP/1.1" 200 None 2026-03-31 08:42:15,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:15,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:42:15,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:15,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e02a64fa9aff93703d HTTP/1.1" 200 None 2026-03-31 08:42:15,255 - lava_callback - INFO - Completed processing callback for node 69cb87e02a64fa9aff93703d INFO: 172.19.0.1:54604 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49668 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:42:29,649 - lava_callback - INFO - Received callback for node 69cb87e12a64fa9aff937040 INFO: 172.19.0.1:49676 - "POST /node/69cb87e12a64fa9aff937040 HTTP/1.0" 202 Accepted 2026-03-31 08:42:29,660 - lava_callback - INFO - Starting async job submit for node 69cb87e12a64fa9aff937040 2026-03-31 08:42:29,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:29,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e12a64fa9aff937040 HTTP/1.1" 200 None 2026-03-31 08:42:29,753 - lava_callback - DEBUG - Processing node 69cb87e12a64fa9aff937040: ltp-controllers 2026-03-31 08:42:30,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:31,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:31,024 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69cb87e12a64fa9aff937040/log.txt.gz 2026-03-31 08:42:31,324 - lava_callback - INFO - Received callback for node 69cb87f22a64fa9aff937065 INFO: 172.19.0.1:45458 - "POST /node/69cb87f22a64fa9aff937065 HTTP/1.0" 202 Accepted 2026-03-31 08:42:31,336 - lava_callback - INFO - Starting async job submit for node 69cb87f22a64fa9aff937065 2026-03-31 08:42:31,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:31,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f22a64fa9aff937065 HTTP/1.1" 200 None 2026-03-31 08:42:31,500 - lava_callback - DEBUG - Processing node 69cb87f22a64fa9aff937065: ltp-syscalls 2026-03-31 08:42:31,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:32,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:32,599 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69cb87e12a64fa9aff937040/lava-logs.yaml 2026-03-31 08:42:32,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:32,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:33,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:33,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cb87f22a64fa9aff937065/log.txt.gz 2026-03-31 08:42:33,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:33,583 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69cb87e12a64fa9aff937040/lava_callback.json.gz 2026-03-31 08:42:33,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:33,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e12a64fa9aff937040 HTTP/1.1" 200 None 2026-03-31 08:42:33,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:33,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:42:33,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:33,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e12a64fa9aff937040 HTTP/1.1" 200 None 2026-03-31 08:42:33,934 - lava_callback - INFO - Completed processing callback for node 69cb87e12a64fa9aff937040 2026-03-31 08:42:34,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:35,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:35,551 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cb87f22a64fa9aff937065/lava-logs.yaml 2026-03-31 08:42:35,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:42:36,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:42:36,395 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cb87f22a64fa9aff937065/lava_callback.json.gz 2026-03-31 08:42:36,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:36,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f22a64fa9aff937065 HTTP/1.1" 200 None 2026-03-31 08:42:36,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:36,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:42:36,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:45464 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:42:36,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87f22a64fa9aff937065 HTTP/1.1" 200 None 2026-03-31 08:42:36,538 - lava_callback - INFO - Completed processing callback for node 69cb87f22a64fa9aff937065 2026-03-31 08:42:37,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:37,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:42:46,436 - lava_callback - INFO - Received callback for node 69c8e6026a4c026e163485a0 INFO: 172.19.0.1:58186 - "POST /node/69c8e6026a4c026e163485a0 HTTP/1.0" 202 Accepted 2026-03-31 08:42:46,445 - lava_callback - INFO - Starting async job submit for node 69c8e6026a4c026e163485a0 2026-03-31 08:42:46,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:46,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6026a4c026e163485a0 HTTP/1.1" 200 None 2026-03-31 08:42:46,497 - lava_callback - DEBUG - Processing node 69c8e6026a4c026e163485a0: baseline-arm-baylibre 2026-03-31 08:42:46,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e6026a4c026e163485a0 2026-03-31 08:42:46,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:58202 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:42:46,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6026a4c026e163485a0 HTTP/1.1" 200 None 2026-03-31 08:42:46,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:46,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:42:46,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:42:46,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e6026a4c026e163485a0 HTTP/1.1" 200 None 2026-03-31 08:42:46,600 - lava_callback - ERROR - Error processing callback for node 69c8e6026a4c026e163485a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41092 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:43:09,308 - lava_callback - INFO - Received callback for node 69cb87c42a64fa9aff936fef INFO: 172.19.0.1:41094 - "POST /node/69cb87c42a64fa9aff936fef HTTP/1.0" 202 Accepted 2026-03-31 08:43:09,319 - lava_callback - INFO - Starting async job submit for node 69cb87c42a64fa9aff936fef 2026-03-31 08:43:09,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:09,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c42a64fa9aff936fef HTTP/1.1" 200 None 2026-03-31 08:43:09,396 - lava_callback - DEBUG - Processing node 69cb87c42a64fa9aff936fef: baseline-arm-clabbe 2026-03-31 08:43:09,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:10,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:10,112 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c42a64fa9aff936fef/log.txt.gz 2026-03-31 08:43:10,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:10,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:10,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c42a64fa9aff936fef/lava-logs.yaml 2026-03-31 08:43:10,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:11,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:11,512 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cb87c42a64fa9aff936fef/lava_callback.json.gz 2026-03-31 08:43:11,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:11,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87c42a64fa9aff936fef HTTP/1.1" 200 None 2026-03-31 08:43:11,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:11,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:43:11,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:11,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87c42a64fa9aff936fef HTTP/1.1" 200 None 2026-03-31 08:43:11,698 - lava_callback - INFO - Completed processing callback for node 69cb87c42a64fa9aff936fef INFO: 172.19.0.1:43020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38956 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:43:37,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:37,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:51894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51898 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:43:49,753 - lava_callback - INFO - Received callback for node 69cb87e32a64fa9aff937043 INFO: 172.19.0.1:40428 - "POST /node/69cb87e32a64fa9aff937043 HTTP/1.0" 202 Accepted 2026-03-31 08:43:49,760 - lava_callback - INFO - Starting async job submit for node 69cb87e32a64fa9aff937043 2026-03-31 08:43:49,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:49,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e32a64fa9aff937043 HTTP/1.1" 200 None 2026-03-31 08:43:49,802 - lava_callback - DEBUG - Processing node 69cb87e32a64fa9aff937043: ltp-cve 2026-03-31 08:43:50,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:50,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:50,424 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69cb87e32a64fa9aff937043/log.txt.gz 2026-03-31 08:43:50,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:50,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:50,941 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69cb87e32a64fa9aff937043/lava-logs.yaml 2026-03-31 08:43:50,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:43:51,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:43:51,142 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69cb87e32a64fa9aff937043/lava_callback.json.gz 2026-03-31 08:43:51,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:51,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e32a64fa9aff937043 HTTP/1.1" 200 None 2026-03-31 08:43:51,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:51,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:43:51,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:43:51,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e32a64fa9aff937043 HTTP/1.1" 200 None 2026-03-31 08:43:51,261 - lava_callback - INFO - Completed processing callback for node 69cb87e32a64fa9aff937043 INFO: 172.19.0.1:40442 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:44:06,419 - lava_callback - INFO - Received callback for node 69c8e64d6a4c026e163485c8 INFO: 172.19.0.1:32852 - "POST /node/69c8e64d6a4c026e163485c8 HTTP/1.0" 202 Accepted 2026-03-31 08:44:06,427 - lava_callback - INFO - Starting async job submit for node 69c8e64d6a4c026e163485c8 2026-03-31 08:44:06,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:06,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e64d6a4c026e163485c8 HTTP/1.1" 200 None 2026-03-31 08:44:06,450 - lava_callback - DEBUG - Processing node 69c8e64d6a4c026e163485c8: baseline-arm-baylibre 2026-03-31 08:44:06,450 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e64d6a4c026e163485c8 2026-03-31 08:44:06,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:06,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e64d6a4c026e163485c8 HTTP/1.1" 200 None 2026-03-31 08:44:06,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:06,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:44:06,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:32854 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:44:06,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e64d6a4c026e163485c8 HTTP/1.1" 200 None 2026-03-31 08:44:06,526 - lava_callback - ERROR - Error processing callback for node 69c8e64d6a4c026e163485c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:44:07,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:07,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:44:10,041 - lava_callback - INFO - Received callback for node 69cb89512a64fa9aff93716a INFO: 172.19.0.1:42186 - "POST /node/69cb89512a64fa9aff93716a HTTP/1.0" 202 Accepted 2026-03-31 08:44:10,047 - lava_callback - INFO - Starting async job submit for node 69cb89512a64fa9aff93716a 2026-03-31 08:44:10,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:10,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb89512a64fa9aff93716a HTTP/1.1" 200 None 2026-03-31 08:44:10,067 - lava_callback - DEBUG - Processing node 69cb89512a64fa9aff93716a: baseline-arm 2026-03-31 08:44:10,075 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:44:10,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:44:10,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:44:10,247 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb89512a64fa9aff93716a/lava-logs.yaml 2026-03-31 08:44:10,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:44:10,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:44:10,395 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb89512a64fa9aff93716a/lava_callback.json.gz 2026-03-31 08:44:10,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:10,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb89512a64fa9aff93716a HTTP/1.1" 200 None 2026-03-31 08:44:10,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:10,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:44:10,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:10,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb89512a64fa9aff93716a HTTP/1.1" 200 None 2026-03-31 08:44:10,464 - lava_callback - INFO - Completed processing callback for node 69cb89512a64fa9aff93716a INFO: 172.19.0.1:42198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48050 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:44:37,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:37,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:44:46,437 - lava_callback - INFO - Received callback for node 69c8e6736a4c026e163485d6 INFO: 172.19.0.1:36958 - "POST /node/69c8e6736a4c026e163485d6 HTTP/1.0" 202 Accepted 2026-03-31 08:44:46,442 - lava_callback - INFO - Starting async job submit for node 69c8e6736a4c026e163485d6 2026-03-31 08:44:46,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:46,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6736a4c026e163485d6 HTTP/1.1" 200 None 2026-03-31 08:44:46,458 - lava_callback - DEBUG - Processing node 69c8e6736a4c026e163485d6: baseline-arm-baylibre 2026-03-31 08:44:46,459 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e6736a4c026e163485d6 2026-03-31 08:44:46,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:46,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6736a4c026e163485d6 HTTP/1.1" 200 None 2026-03-31 08:44:46,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:44:46,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:44:46,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:36972 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:44:46,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e6736a4c026e163485d6 HTTP/1.1" 200 None 2026-03-31 08:44:46,533 - lava_callback - ERROR - Error processing callback for node 69c8e6736a4c026e163485d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53770 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:45:26,638 - lava_callback - INFO - Received callback for node 69c8e6996a4c026e163485e7 INFO: 172.19.0.1:53780 - "POST /node/69c8e6996a4c026e163485e7 HTTP/1.0" 202 Accepted 2026-03-31 08:45:26,644 - lava_callback - INFO - Starting async job submit for node 69c8e6996a4c026e163485e7 2026-03-31 08:45:26,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:45:26,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6996a4c026e163485e7 HTTP/1.1" 200 None 2026-03-31 08:45:26,661 - lava_callback - DEBUG - Processing node 69c8e6996a4c026e163485e7: baseline-arm-baylibre 2026-03-31 08:45:26,661 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e6996a4c026e163485e7 2026-03-31 08:45:26,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:45:26,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6996a4c026e163485e7 HTTP/1.1" 200 None 2026-03-31 08:45:26,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:45:26,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:45:26,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:45:26,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e6996a4c026e163485e7 HTTP/1.1" 200 None 2026-03-31 08:45:26,719 - lava_callback - ERROR - Error processing callback for node 69c8e6996a4c026e163485e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58512 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:46:06,491 - lava_callback - INFO - Received callback for node 69c8e6bf6a4c026e163485fe INFO: 172.19.0.1:54770 - "POST /node/69c8e6bf6a4c026e163485fe HTTP/1.0" 202 Accepted 2026-03-31 08:46:06,497 - lava_callback - INFO - Starting async job submit for node 69c8e6bf6a4c026e163485fe 2026-03-31 08:46:06,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:06,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6bf6a4c026e163485fe HTTP/1.1" 200 None 2026-03-31 08:46:06,514 - lava_callback - DEBUG - Processing node 69c8e6bf6a4c026e163485fe: baseline-arm-baylibre 2026-03-31 08:46:06,514 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e6bf6a4c026e163485fe 2026-03-31 08:46:06,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:54782 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:46:06,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6bf6a4c026e163485fe HTTP/1.1" 200 None 2026-03-31 08:46:06,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:06,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:46:06,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:06,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e6bf6a4c026e163485fe HTTP/1.1" 200 None 2026-03-31 08:46:06,571 - lava_callback - ERROR - Error processing callback for node 69c8e6bf6a4c026e163485fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33428 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:46:30,655 - lava_callback - INFO - Received callback for node 69cb87e62a64fa9aff937049 INFO: 172.19.0.1:49728 - "POST /node/69cb87e62a64fa9aff937049 HTTP/1.0" 202 Accepted 2026-03-31 08:46:30,671 - lava_callback - INFO - Starting async job submit for node 69cb87e62a64fa9aff937049 2026-03-31 08:46:30,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:30,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e62a64fa9aff937049 HTTP/1.1" 200 None 2026-03-31 08:46:30,752 - lava_callback - DEBUG - Processing node 69cb87e62a64fa9aff937049: ltp-fs_bind 2026-03-31 08:46:31,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:46:31,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:46:31,943 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cb87e62a64fa9aff937049/log.txt.gz 2026-03-31 08:46:32,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:46:33,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:46:33,125 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cb87e62a64fa9aff937049/lava-logs.yaml 2026-03-31 08:46:33,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:46:33,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:46:33,699 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cb87e62a64fa9aff937049/lava_callback.json.gz 2026-03-31 08:46:33,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:33,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e62a64fa9aff937049 HTTP/1.1" 200 None 2026-03-31 08:46:33,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:33,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:46:33,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:33,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e62a64fa9aff937049 HTTP/1.1" 200 None 2026-03-31 08:46:33,860 - lava_callback - INFO - Completed processing callback for node 69cb87e62a64fa9aff937049 INFO: 172.19.0.1:49742 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:46:37,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:37,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:48496 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:46:46,731 - lava_callback - INFO - Received callback for node 69c8e6e36a4c026e1634860e INFO: 172.19.0.1:48510 - "POST /node/69c8e6e36a4c026e1634860e HTTP/1.0" 202 Accepted 2026-03-31 08:46:46,741 - lava_callback - INFO - Starting async job submit for node 69c8e6e36a4c026e1634860e 2026-03-31 08:46:46,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:46,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6e36a4c026e1634860e HTTP/1.1" 200 None 2026-03-31 08:46:46,776 - lava_callback - DEBUG - Processing node 69c8e6e36a4c026e1634860e: baseline-arm-baylibre 2026-03-31 08:46:46,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e6e36a4c026e1634860e 2026-03-31 08:46:46,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:46,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e6e36a4c026e1634860e HTTP/1.1" 200 None 2026-03-31 08:46:46,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:46,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:46:46,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:46:46,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e6e36a4c026e1634860e HTTP/1.1" 200 None 2026-03-31 08:46:46,879 - lava_callback - ERROR - Error processing callback for node 69c8e6e36a4c026e1634860e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57876 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:47:09,798 - lava_callback - INFO - Received callback for node 69cb87e82a64fa9aff93704c INFO: 172.19.0.1:58954 - "POST /node/69cb87e82a64fa9aff93704c HTTP/1.0" 202 Accepted 2026-03-31 08:47:09,808 - lava_callback - INFO - Starting async job submit for node 69cb87e82a64fa9aff93704c 2026-03-31 08:47:09,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:09,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e82a64fa9aff93704c HTTP/1.1" 200 None 2026-03-31 08:47:09,902 - lava_callback - DEBUG - Processing node 69cb87e82a64fa9aff93704c: ltp-fsx 2026-03-31 08:47:10,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:47:10,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:47:10,945 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69cb87e82a64fa9aff93704c/log.txt.gz 2026-03-31 08:47:11,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:47:12,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:47:12,362 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69cb87e82a64fa9aff93704c/lava-logs.yaml 2026-03-31 08:47:12,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:47:13,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:47:13,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69cb87e82a64fa9aff93704c/lava_callback.json.gz 2026-03-31 08:47:13,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:13,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87e82a64fa9aff93704c HTTP/1.1" 200 None 2026-03-31 08:47:13,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:13,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:47:13,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:13,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87e82a64fa9aff93704c HTTP/1.1" 200 None 2026-03-31 08:47:13,486 - lava_callback - INFO - Completed processing callback for node 69cb87e82a64fa9aff93704c INFO: 172.19.0.1:58970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60628 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:47:37,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:37,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:59994 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:47:46,754 - lava_callback - INFO - Received callback for node 69c8e72d6a4c026e16348639 INFO: 172.19.0.1:59998 - "POST /node/69c8e72d6a4c026e16348639 HTTP/1.0" 202 Accepted 2026-03-31 08:47:46,762 - lava_callback - INFO - Starting async job submit for node 69c8e72d6a4c026e16348639 2026-03-31 08:47:46,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:46,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e72d6a4c026e16348639 HTTP/1.1" 200 None 2026-03-31 08:47:46,802 - lava_callback - DEBUG - Processing node 69c8e72d6a4c026e16348639: baseline-arm-clabbe 2026-03-31 08:47:46,803 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e72d6a4c026e16348639 2026-03-31 08:47:46,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:46,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e72d6a4c026e16348639 HTTP/1.1" 200 None 2026-03-31 08:47:46,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:46,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:47:46,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:47:46,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e72d6a4c026e16348639 HTTP/1.1" 200 None 2026-03-31 08:47:46,899 - lava_callback - ERROR - Error processing callback for node 69c8e72d6a4c026e16348639: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40164 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:48:09,908 - lava_callback - INFO - Received callback for node 69cb87ea2a64fa9aff93704f INFO: 172.19.0.1:40006 - "POST /node/69cb87ea2a64fa9aff93704f HTTP/1.0" 202 Accepted 2026-03-31 08:48:09,918 - lava_callback - INFO - Starting async job submit for node 69cb87ea2a64fa9aff93704f 2026-03-31 08:48:09,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:09,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ea2a64fa9aff93704f HTTP/1.1" 200 None 2026-03-31 08:48:10,000 - lava_callback - DEBUG - Processing node 69cb87ea2a64fa9aff93704f: ltp-input 2026-03-31 08:48:10,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:10,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:10,798 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69cb87ea2a64fa9aff93704f/log.txt.gz 2026-03-31 08:48:11,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:11,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:11,561 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69cb87ea2a64fa9aff93704f/lava-logs.yaml 2026-03-31 08:48:11,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:11,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:11,871 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69cb87ea2a64fa9aff93704f/lava_callback.json.gz 2026-03-31 08:48:11,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:11,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ea2a64fa9aff93704f HTTP/1.1" 200 None 2026-03-31 08:48:11,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:11,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:48:11,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:12,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87ea2a64fa9aff93704f HTTP/1.1" 200 None 2026-03-31 08:48:12,023 - lava_callback - INFO - Completed processing callback for node 69cb87ea2a64fa9aff93704f INFO: 172.19.0.1:40010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40830 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:48:26,753 - lava_callback - INFO - Received callback for node 69c8e7536a4c026e16348646 INFO: 172.19.0.1:40838 - "POST /node/69c8e7536a4c026e16348646 HTTP/1.0" 202 Accepted 2026-03-31 08:48:26,759 - lava_callback - INFO - Starting async job submit for node 69c8e7536a4c026e16348646 2026-03-31 08:48:26,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:26,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7536a4c026e16348646 HTTP/1.1" 200 None 2026-03-31 08:48:26,782 - lava_callback - DEBUG - Processing node 69c8e7536a4c026e16348646: baseline-arm-clabbe 2026-03-31 08:48:26,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e7536a4c026e16348646 2026-03-31 08:48:26,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:26,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7536a4c026e16348646 HTTP/1.1" 200 None 2026-03-31 08:48:26,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:26,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:48:26,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:26,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e7536a4c026e16348646 HTTP/1.1" 200 None 2026-03-31 08:48:26,872 - lava_callback - ERROR - Error processing callback for node 69c8e7536a4c026e16348646: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44162 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:48:37,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:37,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54264 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:48:49,902 - lava_callback - INFO - Received callback for node 69cb87eb2a64fa9aff937052 INFO: 172.19.0.1:53312 - "POST /node/69cb87eb2a64fa9aff937052 HTTP/1.0" 202 Accepted 2026-03-31 08:48:49,908 - lava_callback - INFO - Starting async job submit for node 69cb87eb2a64fa9aff937052 2026-03-31 08:48:49,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:49,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87eb2a64fa9aff937052 HTTP/1.1" 200 None 2026-03-31 08:48:49,962 - lava_callback - DEBUG - Processing node 69cb87eb2a64fa9aff937052: ltp-ipc 2026-03-31 08:48:50,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:51,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:51,037 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69cb87eb2a64fa9aff937052/log.txt.gz 2026-03-31 08:48:51,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:51,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:51,888 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69cb87eb2a64fa9aff937052/lava-logs.yaml 2026-03-31 08:48:51,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:48:52,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:48:52,318 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69cb87eb2a64fa9aff937052/lava_callback.json.gz 2026-03-31 08:48:52,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:52,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87eb2a64fa9aff937052 HTTP/1.1" 200 None 2026-03-31 08:48:52,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:52,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:48:52,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:48:52,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87eb2a64fa9aff937052 HTTP/1.1" 200 None 2026-03-31 08:48:52,492 - lava_callback - INFO - Completed processing callback for node 69cb87eb2a64fa9aff937052 INFO: 172.19.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42704 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:49:06,823 - lava_callback - INFO - Received callback for node 69c8e7796a4c026e16348653 INFO: 172.19.0.1:42712 - "POST /node/69c8e7796a4c026e16348653 HTTP/1.0" 202 Accepted 2026-03-31 08:49:06,831 - lava_callback - INFO - Starting async job submit for node 69c8e7796a4c026e16348653 2026-03-31 08:49:06,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:06,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7796a4c026e16348653 HTTP/1.1" 200 None 2026-03-31 08:49:06,854 - lava_callback - DEBUG - Processing node 69c8e7796a4c026e16348653: baseline-arm-clabbe 2026-03-31 08:49:06,854 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e7796a4c026e16348653 2026-03-31 08:49:06,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:06,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7796a4c026e16348653 HTTP/1.1" 200 None 2026-03-31 08:49:06,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:06,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:49:06,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:06,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e7796a4c026e16348653 HTTP/1.1" 200 None 2026-03-31 08:49:06,926 - lava_callback - ERROR - Error processing callback for node 69c8e7796a4c026e16348653: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:49:07,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:07,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:58334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36434 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:49:46,778 - lava_callback - INFO - Received callback for node 69c8e7a06a4c026e16348666 INFO: 172.19.0.1:36438 - "POST /node/69c8e7a06a4c026e16348666 HTTP/1.0" 202 Accepted 2026-03-31 08:49:46,784 - lava_callback - INFO - Starting async job submit for node 69c8e7a06a4c026e16348666 2026-03-31 08:49:46,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:46,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7a06a4c026e16348666 HTTP/1.1" 200 None 2026-03-31 08:49:46,801 - lava_callback - DEBUG - Processing node 69c8e7a06a4c026e16348666: baseline-arm-clabbe 2026-03-31 08:49:46,802 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e7a06a4c026e16348666 2026-03-31 08:49:46,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:46,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7a06a4c026e16348666 HTTP/1.1" 200 None 2026-03-31 08:49:46,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:46,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:49:46,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:49:46,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e7a06a4c026e16348666 HTTP/1.1" 200 None 2026-03-31 08:49:46,904 - lava_callback - ERROR - Error processing callback for node 69c8e7a06a4c026e16348666: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54302 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:50:26,872 - lava_callback - INFO - Received callback for node 69c8e7c66a4c026e16348688 INFO: 172.19.0.1:54308 - "POST /node/69c8e7c66a4c026e16348688 HTTP/1.0" 202 Accepted 2026-03-31 08:50:26,879 - lava_callback - INFO - Starting async job submit for node 69c8e7c66a4c026e16348688 2026-03-31 08:50:26,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:26,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7c66a4c026e16348688 HTTP/1.1" 200 None 2026-03-31 08:50:26,905 - lava_callback - DEBUG - Processing node 69c8e7c66a4c026e16348688: baseline-arm-clabbe 2026-03-31 08:50:26,905 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e7c66a4c026e16348688 2026-03-31 08:50:26,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:26,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e7c66a4c026e16348688 HTTP/1.1" 200 None 2026-03-31 08:50:26,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:26,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 08:50:26,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:26,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e7c66a4c026e16348688 HTTP/1.1" 200 None 2026-03-31 08:50:26,981 - lava_callback - ERROR - Error processing callback for node 69c8e7c66a4c026e16348688: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40842 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:50:50,315 - lava_callback - INFO - Received callback for node 69cb8ad82a64fa9aff93720c INFO: 172.19.0.1:52038 - "POST /node/69cb8ad82a64fa9aff93720c HTTP/1.0" 202 Accepted 2026-03-31 08:50:50,324 - lava_callback - INFO - Starting async job submit for node 69cb8ad82a64fa9aff93720c 2026-03-31 08:50:50,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:50,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8ad82a64fa9aff93720c HTTP/1.1" 200 None 2026-03-31 08:50:50,359 - lava_callback - DEBUG - Processing node 69cb8ad82a64fa9aff93720c: baseline-arm 2026-03-31 08:50:50,367 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:50:50,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:50:50,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:50:50,691 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb8ad82a64fa9aff93720c/lava-logs.yaml 2026-03-31 08:50:50,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:50:50,694 - lava_callback - INFO - Received callback for node 69cb87ed2a64fa9aff937055 INFO: 172.19.0.1:52050 - "POST /node/69cb87ed2a64fa9aff937055 HTTP/1.0" 202 Accepted 2026-03-31 08:50:50,702 - lava_callback - INFO - Starting async job submit for node 69cb87ed2a64fa9aff937055 2026-03-31 08:50:50,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:50,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ed2a64fa9aff937055 HTTP/1.1" 200 None 2026-03-31 08:50:50,746 - lava_callback - DEBUG - Processing node 69cb87ed2a64fa9aff937055: ltp-pty 2026-03-31 08:50:50,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:50:50,934 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb8ad82a64fa9aff93720c/lava_callback.json.gz 2026-03-31 08:50:50,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:51,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8ad82a64fa9aff93720c HTTP/1.1" 200 None 2026-03-31 08:50:51,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:51,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:50:51,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:51,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8ad82a64fa9aff93720c HTTP/1.1" 200 None 2026-03-31 08:50:51,246 - lava_callback - INFO - Completed processing callback for node 69cb8ad82a64fa9aff93720c 2026-03-31 08:50:51,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:50:51,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:50:51,640 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69cb87ed2a64fa9aff937055/log.txt.gz 2026-03-31 08:50:51,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:50:52,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:50:52,301 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69cb87ed2a64fa9aff937055/lava-logs.yaml 2026-03-31 08:50:52,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:50:52,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:50:52,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69cb87ed2a64fa9aff937055/lava_callback.json.gz 2026-03-31 08:50:52,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:52,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ed2a64fa9aff937055 HTTP/1.1" 200 None 2026-03-31 08:50:52,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:52,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:50:52,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:50:52,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87ed2a64fa9aff937055 HTTP/1.1" 200 None 2026-03-31 08:50:52,573 - lava_callback - INFO - Completed processing callback for node 69cb87ed2a64fa9aff937055 INFO: 172.19.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42632 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:51:07,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:07,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:49626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34902 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:51:30,047 - lava_callback - INFO - Received callback for node 69cb87ef2a64fa9aff93705b INFO: 172.19.0.1:56186 - "POST /node/69cb87ef2a64fa9aff93705b HTTP/1.0" 202 Accepted 2026-03-31 08:51:30,054 - lava_callback - INFO - Starting async job submit for node 69cb87ef2a64fa9aff93705b 2026-03-31 08:51:30,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:30,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ef2a64fa9aff93705b HTTP/1.1" 200 None 2026-03-31 08:51:30,095 - lava_callback - DEBUG - Processing node 69cb87ef2a64fa9aff93705b: ltp-sched 2026-03-31 08:51:30,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:51:30,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:51:30,622 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69cb87ef2a64fa9aff93705b/log.txt.gz 2026-03-31 08:51:30,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:51:31,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:51:31,037 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69cb87ef2a64fa9aff93705b/lava-logs.yaml 2026-03-31 08:51:31,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:51:31,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:51:31,213 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69cb87ef2a64fa9aff93705b/lava_callback.json.gz 2026-03-31 08:51:31,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:31,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87ef2a64fa9aff93705b HTTP/1.1" 200 None 2026-03-31 08:51:31,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:31,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:51:31,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:31,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87ef2a64fa9aff93705b HTTP/1.1" 200 None 2026-03-31 08:51:31,305 - lava_callback - INFO - Completed processing callback for node 69cb87ef2a64fa9aff93705b INFO: 172.19.0.1:56194 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:51:37,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:51:37,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:43892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46450 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:52:06,849 - lava_callback - INFO - Received callback for node 69c8e82d6a4c026e1634872a INFO: 172.19.0.1:46462 - "POST /node/69c8e82d6a4c026e1634872a HTTP/1.0" 202 Accepted 2026-03-31 08:52:06,855 - lava_callback - INFO - Starting async job submit for node 69c8e82d6a4c026e1634872a 2026-03-31 08:52:06,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:06,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e82d6a4c026e1634872a HTTP/1.1" 200 None 2026-03-31 08:52:06,872 - lava_callback - DEBUG - Processing node 69c8e82d6a4c026e1634872a: baseline-arm64-baylibre 2026-03-31 08:52:06,872 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e82d6a4c026e1634872a 2026-03-31 08:52:06,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:06,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e82d6a4c026e1634872a HTTP/1.1" 200 None 2026-03-31 08:52:06,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:06,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:06,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:06,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e82d6a4c026e1634872a HTTP/1.1" 200 None 2026-03-31 08:52:06,934 - lava_callback - ERROR - Error processing callback for node 69c8e82d6a4c026e1634872a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:07,492 - lava_callback - INFO - Received callback for node 69c8e82f6a4c026e1634872d INFO: 172.19.0.1:46472 - "POST /node/69c8e82f6a4c026e1634872d HTTP/1.0" 202 Accepted 2026-03-31 08:52:07,497 - lava_callback - INFO - Starting async job submit for node 69c8e82f6a4c026e1634872d 2026-03-31 08:52:07,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:07,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e82f6a4c026e1634872d HTTP/1.1" 200 None 2026-03-31 08:52:07,513 - lava_callback - DEBUG - Processing node 69c8e82f6a4c026e1634872d: baseline-arm64-baylibre 2026-03-31 08:52:07,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e82f6a4c026e1634872d 2026-03-31 08:52:07,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:07,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e82f6a4c026e1634872d HTTP/1.1" 200 None 2026-03-31 08:52:07,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:07,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:07,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:07,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e82f6a4c026e1634872d HTTP/1.1" 200 None 2026-03-31 08:52:07,569 - lava_callback - ERROR - Error processing callback for node 69c8e82f6a4c026e1634872d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:08,135 - lava_callback - INFO - Received callback for node 69c8e8306a4c026e16348730 INFO: 172.19.0.1:46482 - "POST /node/69c8e8306a4c026e16348730 HTTP/1.0" 202 Accepted 2026-03-31 08:52:08,143 - lava_callback - INFO - Starting async job submit for node 69c8e8306a4c026e16348730 2026-03-31 08:52:08,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8306a4c026e16348730 HTTP/1.1" 200 None 2026-03-31 08:52:08,163 - lava_callback - DEBUG - Processing node 69c8e8306a4c026e16348730: baseline-arm64-baylibre 2026-03-31 08:52:08,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8306a4c026e16348730 2026-03-31 08:52:08,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8306a4c026e16348730 HTTP/1.1" 200 None 2026-03-31 08:52:08,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:08,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8306a4c026e16348730 HTTP/1.1" 200 None 2026-03-31 08:52:08,225 - lava_callback - ERROR - Error processing callback for node 69c8e8306a4c026e16348730: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:08,822 - lava_callback - INFO - Received callback for node 69c8e8326a4c026e16348754 INFO: 172.19.0.1:46488 - "POST /node/69c8e8326a4c026e16348754 HTTP/1.0" 202 Accepted 2026-03-31 08:52:08,827 - lava_callback - INFO - Starting async job submit for node 69c8e8326a4c026e16348754 2026-03-31 08:52:08,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8326a4c026e16348754 HTTP/1.1" 200 None 2026-03-31 08:52:08,844 - lava_callback - DEBUG - Processing node 69c8e8326a4c026e16348754: baseline-arm64-baylibre 2026-03-31 08:52:08,844 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8326a4c026e16348754 2026-03-31 08:52:08,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8326a4c026e16348754 HTTP/1.1" 200 None 2026-03-31 08:52:08,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:08,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:08,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8326a4c026e16348754 HTTP/1.1" 200 None 2026-03-31 08:52:08,904 - lava_callback - ERROR - Error processing callback for node 69c8e8326a4c026e16348754: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:09,490 - lava_callback - INFO - Received callback for node 69c8e8346a4c026e16348757 INFO: 172.19.0.1:46496 - "POST /node/69c8e8346a4c026e16348757 HTTP/1.0" 202 Accepted 2026-03-31 08:52:09,495 - lava_callback - INFO - Starting async job submit for node 69c8e8346a4c026e16348757 2026-03-31 08:52:09,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:09,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8346a4c026e16348757 HTTP/1.1" 200 None 2026-03-31 08:52:09,512 - lava_callback - DEBUG - Processing node 69c8e8346a4c026e16348757: baseline-arm64-baylibre 2026-03-31 08:52:09,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8346a4c026e16348757 2026-03-31 08:52:09,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:09,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8346a4c026e16348757 HTTP/1.1" 200 None 2026-03-31 08:52:09,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:09,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:09,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:09,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8346a4c026e16348757 HTTP/1.1" 200 None 2026-03-31 08:52:09,569 - lava_callback - ERROR - Error processing callback for node 69c8e8346a4c026e16348757: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:09,933 - lava_callback - INFO - Received callback for node 69cb87f02a64fa9aff937062 INFO: 172.19.0.1:52678 - "POST /node/69cb87f02a64fa9aff937062 HTTP/1.0" 202 Accepted 2026-03-31 08:52:09,940 - lava_callback - INFO - Starting async job submit for node 69cb87f02a64fa9aff937062 2026-03-31 08:52:09,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:09,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f02a64fa9aff937062 HTTP/1.1" 200 None 2026-03-31 08:52:09,990 - lava_callback - DEBUG - Processing node 69cb87f02a64fa9aff937062: ltp-smoke 2026-03-31 08:52:10,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:10,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:10,521 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87f02a64fa9aff937062/log.txt.gz 2026-03-31 08:52:10,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:10,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:10,951 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87f02a64fa9aff937062/lava-logs.yaml 2026-03-31 08:52:10,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:11,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:11,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87f02a64fa9aff937062/lava_callback.json.gz 2026-03-31 08:52:11,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:11,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f02a64fa9aff937062 HTTP/1.1" 200 None 2026-03-31 08:52:11,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:11,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:52:11,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:11,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87f02a64fa9aff937062 HTTP/1.1" 200 None 2026-03-31 08:52:11,279 - lava_callback - INFO - Completed processing callback for node 69cb87f02a64fa9aff937062 INFO: 172.19.0.1:52682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38840 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:52:26,886 - lava_callback - INFO - Received callback for node 69c8e8356a4c026e1634875a INFO: 172.19.0.1:38854 - "POST /node/69c8e8356a4c026e1634875a HTTP/1.0" 202 Accepted 2026-03-31 08:52:26,892 - lava_callback - INFO - Starting async job submit for node 69c8e8356a4c026e1634875a 2026-03-31 08:52:26,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8356a4c026e1634875a HTTP/1.1" 200 None 2026-03-31 08:52:26,911 - lava_callback - DEBUG - Processing node 69c8e8356a4c026e1634875a: baseline-arm64-baylibre 2026-03-31 08:52:26,911 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8356a4c026e1634875a 2026-03-31 08:52:26,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8356a4c026e1634875a HTTP/1.1" 200 None 2026-03-31 08:52:26,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,940 - lava_callback - INFO - Received callback for node 69c8e8446a4c026e16348780 INFO: 172.19.0.1:38866 - "POST /node/69c8e8446a4c026e16348780 HTTP/1.0" 202 Accepted 2026-03-31 08:52:26,946 - lava_callback - INFO - Starting async job submit for node 69c8e8446a4c026e16348780 2026-03-31 08:52:26,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:26,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8446a4c026e16348780 HTTP/1.1" 200 None 2026-03-31 08:52:26,964 - lava_callback - DEBUG - Processing node 69c8e8446a4c026e16348780: baseline-arm64-clabbe 2026-03-31 08:52:26,964 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8446a4c026e16348780 2026-03-31 08:52:26,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:26,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8356a4c026e1634875a HTTP/1.1" 200 None 2026-03-31 08:52:26,979 - lava_callback - ERROR - Error processing callback for node 69c8e8356a4c026e1634875a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:27,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8446a4c026e16348780 HTTP/1.1" 200 None 2026-03-31 08:52:27,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:27,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8446a4c026e16348780 HTTP/1.1" 200 None 2026-03-31 08:52:27,122 - lava_callback - ERROR - Error processing callback for node 69c8e8446a4c026e16348780: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:27,623 - lava_callback - INFO - Received callback for node 69c8e8466a4c026e16348783 INFO: 172.19.0.1:38882 - "POST /node/69c8e8466a4c026e16348783 HTTP/1.0" 202 Accepted 2026-03-31 08:52:27,629 - lava_callback - INFO - Starting async job submit for node 69c8e8466a4c026e16348783 2026-03-31 08:52:27,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8466a4c026e16348783 HTTP/1.1" 200 None 2026-03-31 08:52:27,648 - lava_callback - DEBUG - Processing node 69c8e8466a4c026e16348783: baseline-arm64-clabbe 2026-03-31 08:52:27,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8466a4c026e16348783 2026-03-31 08:52:27,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8466a4c026e16348783 HTTP/1.1" 200 None 2026-03-31 08:52:27,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:27,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:27,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8466a4c026e16348783 HTTP/1.1" 200 None 2026-03-31 08:52:27,722 - lava_callback - ERROR - Error processing callback for node 69c8e8466a4c026e16348783: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:28,304 - lava_callback - INFO - Received callback for node 69c8e8476a4c026e16348786 INFO: 172.19.0.1:38884 - "POST /node/69c8e8476a4c026e16348786 HTTP/1.0" 202 Accepted 2026-03-31 08:52:28,312 - lava_callback - INFO - Starting async job submit for node 69c8e8476a4c026e16348786 2026-03-31 08:52:28,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:28,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8476a4c026e16348786 HTTP/1.1" 200 None 2026-03-31 08:52:28,330 - lava_callback - DEBUG - Processing node 69c8e8476a4c026e16348786: baseline-arm64-clabbe 2026-03-31 08:52:28,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8476a4c026e16348786 2026-03-31 08:52:28,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:28,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8476a4c026e16348786 HTTP/1.1" 200 None 2026-03-31 08:52:28,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:28,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:28,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:28,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8476a4c026e16348786 HTTP/1.1" 200 None 2026-03-31 08:52:28,396 - lava_callback - ERROR - Error processing callback for node 69c8e8476a4c026e16348786: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:28,986 - lava_callback - INFO - Received callback for node 69c8e8486a4c026e16348789 INFO: 172.19.0.1:38890 - "POST /node/69c8e8486a4c026e16348789 HTTP/1.0" 202 Accepted 2026-03-31 08:52:28,992 - lava_callback - INFO - Starting async job submit for node 69c8e8486a4c026e16348789 2026-03-31 08:52:28,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:29,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8486a4c026e16348789 HTTP/1.1" 200 None 2026-03-31 08:52:29,009 - lava_callback - DEBUG - Processing node 69c8e8486a4c026e16348789: baseline-arm64-clabbe 2026-03-31 08:52:29,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8486a4c026e16348789 2026-03-31 08:52:29,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:29,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8486a4c026e16348789 HTTP/1.1" 200 None 2026-03-31 08:52:29,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:29,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:29,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:29,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8486a4c026e16348789 HTTP/1.1" 200 None 2026-03-31 08:52:29,079 - lava_callback - ERROR - Error processing callback for node 69c8e8486a4c026e16348789: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60690 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:52:37,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:37,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:40078 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:52:46,901 - lava_callback - INFO - Received callback for node 69c8e84a6a4c026e163487a4 INFO: 172.19.0.1:40094 - "POST /node/69c8e84a6a4c026e163487a4 HTTP/1.0" 202 Accepted 2026-03-31 08:52:46,907 - lava_callback - INFO - Starting async job submit for node 69c8e84a6a4c026e163487a4 2026-03-31 08:52:46,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:46,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e84a6a4c026e163487a4 HTTP/1.1" 200 None 2026-03-31 08:52:46,924 - lava_callback - DEBUG - Processing node 69c8e84a6a4c026e163487a4: baseline-arm64-clabbe 2026-03-31 08:52:46,924 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e84a6a4c026e163487a4 2026-03-31 08:52:46,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:46,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e84a6a4c026e163487a4 HTTP/1.1" 200 None 2026-03-31 08:52:46,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:46,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:46,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:46,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e84a6a4c026e163487a4 HTTP/1.1" 200 None 2026-03-31 08:52:46,977 - lava_callback - ERROR - Error processing callback for node 69c8e84a6a4c026e163487a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40096 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:52:47,580 - lava_callback - INFO - Received callback for node 69c8e8496a4c026e1634879e INFO: 172.19.0.1:40104 - "POST /node/69c8e8496a4c026e1634879e HTTP/1.0" 202 Accepted 2026-03-31 08:52:47,586 - lava_callback - INFO - Starting async job submit for node 69c8e8496a4c026e1634879e 2026-03-31 08:52:47,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:47,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8496a4c026e1634879e HTTP/1.1" 200 None 2026-03-31 08:52:47,607 - lava_callback - DEBUG - Processing node 69c8e8496a4c026e1634879e: baseline-arm64-clabbe 2026-03-31 08:52:47,607 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e8496a4c026e1634879e 2026-03-31 08:52:47,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:47,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e8496a4c026e1634879e HTTP/1.1" 200 None 2026-03-31 08:52:47,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:47,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 08:52:47,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:47,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e8496a4c026e1634879e HTTP/1.1" 200 None 2026-03-31 08:52:47,665 - lava_callback - ERROR - Error processing callback for node 69c8e8496a4c026e1634879e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:52:50,029 - lava_callback - INFO - Received callback for node 69cb87f32a64fa9aff937068 INFO: 172.19.0.1:36986 - "POST /node/69cb87f32a64fa9aff937068 HTTP/1.0" 202 Accepted 2026-03-31 08:52:50,036 - lava_callback - INFO - Starting async job submit for node 69cb87f32a64fa9aff937068 2026-03-31 08:52:50,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:50,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f32a64fa9aff937068 HTTP/1.1" 200 None 2026-03-31 08:52:50,075 - lava_callback - DEBUG - Processing node 69cb87f32a64fa9aff937068: ltp-syscalls-ipc 2026-03-31 08:52:50,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:50,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:50,550 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cb87f32a64fa9aff937068/log.txt.gz 2026-03-31 08:52:50,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:51,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:51,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cb87f32a64fa9aff937068/lava-logs.yaml 2026-03-31 08:52:51,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:52:51,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:52:51,184 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cb87f32a64fa9aff937068/lava_callback.json.gz 2026-03-31 08:52:51,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:51,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f32a64fa9aff937068 HTTP/1.1" 200 None 2026-03-31 08:52:51,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:51,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:52:51,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:52:51,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87f32a64fa9aff937068 HTTP/1.1" 200 None 2026-03-31 08:52:51,282 - lava_callback - INFO - Completed processing callback for node 69cb87f32a64fa9aff937068 INFO: 172.19.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42414 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:53:07,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:07,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:44606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42000 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:53:27,371 - lava_callback - INFO - Received callback for node 69cb87a62a64fa9aff936f73 INFO: 172.19.0.1:42002 - "POST /node/69cb87a62a64fa9aff936f73 HTTP/1.0" 202 Accepted 2026-03-31 08:53:27,383 - lava_callback - INFO - Starting async job submit for node 69cb87a62a64fa9aff936f73 2026-03-31 08:53:27,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:27,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a62a64fa9aff936f73 HTTP/1.1" 200 None 2026-03-31 08:53:27,502 - lava_callback - DEBUG - Processing node 69cb87a62a64fa9aff936f73: baseline-arm 2026-03-31 08:53:28,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:53:32,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:53:32,656 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a62a64fa9aff936f73/log.txt.gz 2026-03-31 08:53:33,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:53:33,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:53:33,750 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a62a64fa9aff936f73/lava-logs.yaml 2026-03-31 08:53:33,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:53:33,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:53:33,964 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb87a62a64fa9aff936f73/lava_callback.json.gz DEBUG: Setup failed for 69cb87a62a64fa9aff936f73. Transitting job node result: incomplete -> fail 2026-03-31 08:53:34,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:34,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87a62a64fa9aff936f73 HTTP/1.1" 200 None 2026-03-31 08:53:34,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:34,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:53:34,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:34,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87a62a64fa9aff936f73 HTTP/1.1" 200 None 2026-03-31 08:53:34,360 - lava_callback - INFO - Completed processing callback for node 69cb87a62a64fa9aff936f73 INFO: 172.19.0.1:36370 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:53:37,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:53:37,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:45386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51288 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:55:10,854 - lava_callback - INFO - Received callback for node 69cb87f52a64fa9aff93706b INFO: 172.19.0.1:37982 - "POST /node/69cb87f52a64fa9aff93706b HTTP/1.0" 202 Accepted 2026-03-31 08:55:10,864 - lava_callback - INFO - Starting async job submit for node 69cb87f52a64fa9aff93706b 2026-03-31 08:55:10,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:10,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f52a64fa9aff93706b HTTP/1.1" 200 None 2026-03-31 08:55:10,920 - lava_callback - DEBUG - Processing node 69cb87f52a64fa9aff93706b: ltp-timers 2026-03-31 08:55:11,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:11,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:11,912 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69cb87f52a64fa9aff93706b/log.txt.gz 2026-03-31 08:55:12,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:12,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:12,597 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69cb87f52a64fa9aff93706b/lava-logs.yaml 2026-03-31 08:55:12,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:13,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:13,194 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69cb87f52a64fa9aff93706b/lava_callback.json.gz 2026-03-31 08:55:13,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:13,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f52a64fa9aff93706b HTTP/1.1" 200 None 2026-03-31 08:55:13,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:13,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:55:13,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:13,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87f52a64fa9aff93706b HTTP/1.1" 200 None 2026-03-31 08:55:13,336 - lava_callback - INFO - Completed processing callback for node 69cb87f52a64fa9aff93706b INFO: 172.19.0.1:37984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54916 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:55:27,243 - lava_callback - INFO - Received callback for node 69cb8bdc2a64fa9aff9372ab INFO: 172.19.0.1:54930 - "POST /node/69cb8bdc2a64fa9aff9372ab HTTP/1.0" 202 Accepted 2026-03-31 08:55:27,250 - lava_callback - INFO - Starting async job submit for node 69cb8bdc2a64fa9aff9372ab 2026-03-31 08:55:27,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:27,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdc2a64fa9aff9372ab HTTP/1.1" 200 None 2026-03-31 08:55:27,307 - lava_callback - DEBUG - Processing node 69cb8bdc2a64fa9aff9372ab: baseline-arm64 2026-03-31 08:55:27,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:27,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:27,767 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdc2a64fa9aff9372ab/log.txt.gz 2026-03-31 08:55:27,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:28,189 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:28,191 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdc2a64fa9aff9372ab/lava-logs.yaml 2026-03-31 08:55:28,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:28,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:28,409 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdc2a64fa9aff9372ab/lava_callback.json.gz 2026-03-31 08:55:28,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:28,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdc2a64fa9aff9372ab HTTP/1.1" 200 None 2026-03-31 08:55:28,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:28,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:55:28,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:28,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bdc2a64fa9aff9372ab HTTP/1.1" 200 None 2026-03-31 08:55:28,605 - lava_callback - INFO - Completed processing callback for node 69cb8bdc2a64fa9aff9372ab 2026-03-31 08:55:29,062 - lava_callback - INFO - Received callback for node 69cb8bfc2a64fa9aff93731b INFO: 172.19.0.1:54938 - "POST /node/69cb8bfc2a64fa9aff93731b HTTP/1.0" 202 Accepted 2026-03-31 08:55:29,071 - lava_callback - INFO - Starting async job submit for node 69cb8bfc2a64fa9aff93731b 2026-03-31 08:55:29,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:29,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfc2a64fa9aff93731b HTTP/1.1" 200 None 2026-03-31 08:55:29,091 - lava_callback - DEBUG - Processing node 69cb8bfc2a64fa9aff93731b: baseline-nfs-arm64 2026-03-31 08:55:29,093 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:55:29,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:29,218 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:29,219 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfc2a64fa9aff93731b/lava-logs.yaml 2026-03-31 08:55:29,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:29,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:29,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfc2a64fa9aff93731b/lava_callback.json.gz 2026-03-31 08:55:29,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:29,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfc2a64fa9aff93731b HTTP/1.1" 200 None 2026-03-31 08:55:29,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:29,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:55:29,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:29,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bfc2a64fa9aff93731b HTTP/1.1" 200 None 2026-03-31 08:55:29,491 - lava_callback - INFO - Completed processing callback for node 69cb8bfc2a64fa9aff93731b 2026-03-31 08:55:30,269 - lava_callback - INFO - Received callback for node 69cb8bea2a64fa9aff9372cc INFO: 172.19.0.1:44438 - "POST /node/69cb8bea2a64fa9aff9372cc HTTP/1.0" 202 Accepted 2026-03-31 08:55:30,279 - lava_callback - INFO - Starting async job submit for node 69cb8bea2a64fa9aff9372cc 2026-03-31 08:55:30,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:30,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bea2a64fa9aff9372cc HTTP/1.1" 200 None 2026-03-31 08:55:30,374 - lava_callback - DEBUG - Processing node 69cb8bea2a64fa9aff9372cc: baseline-arm64 2026-03-31 08:55:30,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:30,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:30,830 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bea2a64fa9aff9372cc/log.txt.gz 2026-03-31 08:55:31,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:31,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:31,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bea2a64fa9aff9372cc/lava-logs.yaml 2026-03-31 08:55:31,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:31,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:31,598 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bea2a64fa9aff9372cc/lava_callback.json.gz 2026-03-31 08:55:31,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:31,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bea2a64fa9aff9372cc HTTP/1.1" 200 None 2026-03-31 08:55:31,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:31,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:55:31,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:31,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bea2a64fa9aff9372cc HTTP/1.1" 200 None 2026-03-31 08:55:31,750 - lava_callback - INFO - Completed processing callback for node 69cb8bea2a64fa9aff9372cc INFO: 172.19.0.1:44446 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:55:37,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:37,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.19.0.1:37500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37502 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:55:50,191 - lava_callback - INFO - Received callback for node 69cb87f62a64fa9aff93706e INFO: 172.19.0.1:32848 - "POST /node/69cb87f62a64fa9aff93706e HTTP/1.0" 202 Accepted 2026-03-31 08:55:50,203 - lava_callback - INFO - Starting async job submit for node 69cb87f62a64fa9aff93706e 2026-03-31 08:55:50,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:50,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f62a64fa9aff93706e HTTP/1.1" 200 None 2026-03-31 08:55:50,278 - lava_callback - DEBUG - Processing node 69cb87f62a64fa9aff93706e: ltp-watchqueue 2026-03-31 08:55:50,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:51,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:51,076 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cb87f62a64fa9aff93706e/log.txt.gz 2026-03-31 08:55:51,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:51,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:51,683 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cb87f62a64fa9aff93706e/lava-logs.yaml 2026-03-31 08:55:51,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:55:51,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:55:51,841 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cb87f62a64fa9aff93706e/lava_callback.json.gz 2026-03-31 08:55:51,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:51,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87f62a64fa9aff93706e HTTP/1.1" 200 None 2026-03-31 08:55:51,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:51,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:55:51,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:55:51,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87f62a64fa9aff93706e HTTP/1.1" 200 None 2026-03-31 08:55:51,935 - lava_callback - INFO - Completed processing callback for node 69cb87f62a64fa9aff93706e INFO: 172.19.0.1:32860 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:06,443 - lava_callback - INFO - Received callback for node 69cb8bde2a64fa9aff9372b1 INFO: 172.19.0.1:41456 - "POST /node/69cb8bde2a64fa9aff9372b1 HTTP/1.0" 202 Accepted 2026-03-31 08:56:06,452 - lava_callback - INFO - Starting async job submit for node 69cb8bde2a64fa9aff9372b1 2026-03-31 08:56:06,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:41470 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:06,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bde2a64fa9aff9372b1 HTTP/1.1" 200 None 2026-03-31 08:56:06,543 - lava_callback - DEBUG - Processing node 69cb8bde2a64fa9aff9372b1: baseline-arm64 2026-03-31 08:56:06,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:07,082 - lava_callback - INFO - Received callback for node 69c8e9226a4c026e16348909 INFO: 172.19.0.1:41474 - "POST /node/69c8e9226a4c026e16348909 HTTP/1.0" 202 Accepted 2026-03-31 08:56:07,100 - lava_callback - INFO - Starting async job submit for node 69c8e9226a4c026e16348909 2026-03-31 08:56:07,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:07,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:07,129 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bde2a64fa9aff9372b1/log.txt.gz 2026-03-31 08:56:07,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9226a4c026e16348909 HTTP/1.1" 200 None 2026-03-31 08:56:07,145 - lava_callback - DEBUG - Processing node 69c8e9226a4c026e16348909: baseline-arm-baylibre 2026-03-31 08:56:07,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e9226a4c026e16348909 2026-03-31 08:56:07,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:07,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9226a4c026e16348909 HTTP/1.1" 200 None 2026-03-31 08:56:07,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:07,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 08:56:07,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:07,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e9226a4c026e16348909 HTTP/1.1" 200 None 2026-03-31 08:56:07,376 - lava_callback - ERROR - Error processing callback for node 69c8e9226a4c026e16348909: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:56:07,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:07,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:07,821 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bde2a64fa9aff9372b1/lava-logs.yaml 2026-03-31 08:56:07,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:07,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:07,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 08:56:08,006 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:08,009 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bde2a64fa9aff9372b1/lava_callback.json.gz 2026-03-31 08:56:08,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:08,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bde2a64fa9aff9372b1 HTTP/1.1" 200 None 2026-03-31 08:56:08,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:08,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:08,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:08,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bde2a64fa9aff9372b1 HTTP/1.1" 200 None 2026-03-31 08:56:08,120 - lava_callback - INFO - Completed processing callback for node 69cb8bde2a64fa9aff9372b1 2026-03-31 08:56:13,786 - lava_callback - INFO - Received callback for node 69cb8be62a64fa9aff9372c3 INFO: 172.19.0.1:34914 - "POST /node/69cb8be62a64fa9aff9372c3 HTTP/1.0" 202 Accepted 2026-03-31 08:56:13,796 - lava_callback - INFO - Starting async job submit for node 69cb8be62a64fa9aff9372c3 2026-03-31 08:56:13,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:13,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be62a64fa9aff9372c3 HTTP/1.1" 200 None 2026-03-31 08:56:13,864 - lava_callback - DEBUG - Processing node 69cb8be62a64fa9aff9372c3: baseline-arm64 2026-03-31 08:56:14,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:14,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:14,692 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be62a64fa9aff9372c3/log.txt.gz 2026-03-31 08:56:14,798 - lava_callback - INFO - Received callback for node 69cb8be22a64fa9aff9372ba INFO: 172.19.0.1:34916 - "POST /node/69cb8be22a64fa9aff9372ba HTTP/1.0" 202 Accepted 2026-03-31 08:56:14,829 - lava_callback - INFO - Starting async job submit for node 69cb8be22a64fa9aff9372ba 2026-03-31 08:56:14,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:15,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be22a64fa9aff9372ba HTTP/1.1" 200 None 2026-03-31 08:56:15,054 - lava_callback - DEBUG - Processing node 69cb8be22a64fa9aff9372ba: baseline-arm64 INFO: 172.19.0.1:34924 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 08:56:15,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:15,423 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:56:15,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:15,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:15,727 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be62a64fa9aff9372c3/lava-logs.yaml 2026-03-31 08:56:15,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:16,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:16,031 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be22a64fa9aff9372ba/lava-logs.yaml 2026-03-31 08:56:16,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:16,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:16,139 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be62a64fa9aff9372c3/lava_callback.json.gz 2026-03-31 08:56:16,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be62a64fa9aff9372c3 HTTP/1.1" 200 None 2026-03-31 08:56:16,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:16,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:16,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be22a64fa9aff9372ba/lava_callback.json.gz 2026-03-31 08:56:16,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be62a64fa9aff9372c3 HTTP/1.1" 200 None 2026-03-31 08:56:16,350 - lava_callback - INFO - Completed processing callback for node 69cb8be62a64fa9aff9372c3 2026-03-31 08:56:16,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be22a64fa9aff9372ba HTTP/1.1" 200 None 2026-03-31 08:56:16,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:16,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be22a64fa9aff9372ba HTTP/1.1" 200 None INFO: 172.19.0.1:34938 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:16,525 - lava_callback - INFO - Completed processing callback for node 69cb8be22a64fa9aff9372ba 2026-03-31 08:56:16,589 - lava_callback - INFO - Received callback for node 69cb8be02a64fa9aff9372b7 INFO: 172.19.0.1:34940 - "POST /node/69cb8be02a64fa9aff9372b7 HTTP/1.0" 202 Accepted 2026-03-31 08:56:16,602 - lava_callback - INFO - Starting async job submit for node 69cb8be02a64fa9aff9372b7 2026-03-31 08:56:16,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:16,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be02a64fa9aff9372b7 HTTP/1.1" 200 None 2026-03-31 08:56:16,695 - lava_callback - DEBUG - Processing node 69cb8be02a64fa9aff9372b7: baseline-arm64 2026-03-31 08:56:16,974 - lava_callback - WARNING - Failed to upload log 2026-03-31 08:56:17,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:17,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:17,450 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be02a64fa9aff9372b7/lava-logs.yaml 2026-03-31 08:56:17,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:17,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:17,582 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be02a64fa9aff9372b7/lava_callback.json.gz 2026-03-31 08:56:17,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:17,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be02a64fa9aff9372b7 HTTP/1.1" 200 None 2026-03-31 08:56:17,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:17,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:17,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:17,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be02a64fa9aff9372b7 HTTP/1.1" 200 None 2026-03-31 08:56:17,747 - lava_callback - INFO - Completed processing callback for node 69cb8be02a64fa9aff9372b7 2026-03-31 08:56:26,504 - lava_callback - INFO - Received callback for node 69cb8bdf2a64fa9aff9372b4 INFO: 172.19.0.1:37070 - "POST /node/69cb8bdf2a64fa9aff9372b4 HTTP/1.0" 202 Accepted 2026-03-31 08:56:26,513 - lava_callback - INFO - Starting async job submit for node 69cb8bdf2a64fa9aff9372b4 INFO: 172.19.0.1:37086 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:26,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:26,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdf2a64fa9aff9372b4 HTTP/1.1" 200 None 2026-03-31 08:56:26,560 - lava_callback - DEBUG - Processing node 69cb8bdf2a64fa9aff9372b4: baseline-arm64 2026-03-31 08:56:26,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:27,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:27,230 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdf2a64fa9aff9372b4/log.txt.gz 2026-03-31 08:56:27,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:27,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:27,547 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdf2a64fa9aff9372b4/lava-logs.yaml 2026-03-31 08:56:27,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:27,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:27,767 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdf2a64fa9aff9372b4/lava_callback.json.gz 2026-03-31 08:56:27,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:27,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdf2a64fa9aff9372b4 HTTP/1.1" 200 None 2026-03-31 08:56:27,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:27,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:27,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:27,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bdf2a64fa9aff9372b4 HTTP/1.1" 200 None 2026-03-31 08:56:27,864 - lava_callback - INFO - Completed processing callback for node 69cb8bdf2a64fa9aff9372b4 2026-03-31 08:56:28,615 - lava_callback - INFO - Received callback for node 69cb8be72a64fa9aff9372c6 INFO: 172.19.0.1:37090 - "POST /node/69cb8be72a64fa9aff9372c6 HTTP/1.0" 202 Accepted 2026-03-31 08:56:28,622 - lava_callback - INFO - Starting async job submit for node 69cb8be72a64fa9aff9372c6 2026-03-31 08:56:28,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:28,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be72a64fa9aff9372c6 HTTP/1.1" 200 None 2026-03-31 08:56:28,700 - lava_callback - DEBUG - Processing node 69cb8be72a64fa9aff9372c6: baseline-arm64 2026-03-31 08:56:28,903 - lava_callback - INFO - Received callback for node 69cb8bee2a64fa9aff9372d8 INFO: 172.19.0.1:37094 - "POST /node/69cb8bee2a64fa9aff9372d8 HTTP/1.0" 202 Accepted 2026-03-31 08:56:28,923 - lava_callback - INFO - Starting async job submit for node 69cb8bee2a64fa9aff9372d8 2026-03-31 08:56:29,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:29,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bee2a64fa9aff9372d8 HTTP/1.1" 200 None 2026-03-31 08:56:29,139 - lava_callback - DEBUG - Processing node 69cb8bee2a64fa9aff9372d8: baseline-arm64 2026-03-31 08:56:29,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:29,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:29,816 - lava_callback - INFO - Received callback for node 69cb8beb2a64fa9aff9372cf 2026-03-31 08:56:29,828 - lava_callback - INFO - Starting async job submit for node 69cb8beb2a64fa9aff9372cf INFO: 172.19.0.1:47666 - "POST /node/69cb8beb2a64fa9aff9372cf HTTP/1.0" 202 Accepted 2026-03-31 08:56:29,847 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be72a64fa9aff9372c6/log.txt.gz 2026-03-31 08:56:29,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:29,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:29,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8beb2a64fa9aff9372cf HTTP/1.1" 200 None 2026-03-31 08:56:29,956 - lava_callback - DEBUG - Processing node 69cb8beb2a64fa9aff9372cf: baseline-arm64 2026-03-31 08:56:30,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:30,138 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bee2a64fa9aff9372d8/log.txt.gz 2026-03-31 08:56:30,211 - lava_callback - INFO - Received callback for node 69cb8bf02a64fa9aff9372df 2026-03-31 08:56:30,313 - lava_callback - INFO - Starting async job submit for node 69cb8bf02a64fa9aff9372df INFO: 172.19.0.1:47674 - "POST /node/69cb8bf02a64fa9aff9372df HTTP/1.0" 202 Accepted 2026-03-31 08:56:30,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:30,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:30,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf02a64fa9aff9372df HTTP/1.1" 200 None 2026-03-31 08:56:30,620 - lava_callback - DEBUG - Processing node 69cb8bf02a64fa9aff9372df: baseline-arm64 2026-03-31 08:56:31,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:31,377 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be72a64fa9aff9372c6/lava-logs.yaml 2026-03-31 08:56:31,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:31,461 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8beb2a64fa9aff9372cf/log.txt.gz 2026-03-31 08:56:31,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:31,609 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bee2a64fa9aff9372d8/lava-logs.yaml 2026-03-31 08:56:31,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:31,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:31,703 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be72a64fa9aff9372c6/lava_callback.json.gz 2026-03-31 08:56:31,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:31,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be72a64fa9aff9372c6 HTTP/1.1" 200 None 2026-03-31 08:56:31,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:31,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:31,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:31,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be72a64fa9aff9372c6 HTTP/1.1" 200 None 2026-03-31 08:56:31,952 - lava_callback - INFO - Completed processing callback for node 69cb8be72a64fa9aff9372c6 2026-03-31 08:56:31,958 - lava_callback - INFO - Received callback for node 69cb8bdd2a64fa9aff9372ae INFO: 172.19.0.1:47678 - "POST /node/69cb8bdd2a64fa9aff9372ae HTTP/1.0" 202 Accepted 2026-03-31 08:56:31,984 - lava_callback - INFO - Starting async job submit for node 69cb8bdd2a64fa9aff9372ae 2026-03-31 08:56:32,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:32,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf02a64fa9aff9372df/log.txt.gz 2026-03-31 08:56:32,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdd2a64fa9aff9372ae HTTP/1.1" 200 None 2026-03-31 08:56:32,130 - lava_callback - DEBUG - Processing node 69cb8bdd2a64fa9aff9372ae: baseline-arm64 2026-03-31 08:56:32,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:32,189 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bee2a64fa9aff9372d8/lava_callback.json.gz 2026-03-31 08:56:32,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:32,456 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8beb2a64fa9aff9372cf/lava-logs.yaml 2026-03-31 08:56:32,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bee2a64fa9aff9372d8 HTTP/1.1" 200 None 2026-03-31 08:56:32,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:32,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:32,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:32,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:32,850 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8beb2a64fa9aff9372cf/lava_callback.json.gz 2026-03-31 08:56:32,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bee2a64fa9aff9372d8 HTTP/1.1" 200 None 2026-03-31 08:56:32,869 - lava_callback - INFO - Completed processing callback for node 69cb8bee2a64fa9aff9372d8 2026-03-31 08:56:32,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:32,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8beb2a64fa9aff9372cf HTTP/1.1" 200 None 2026-03-31 08:56:32,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:32,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:33,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:33,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8beb2a64fa9aff9372cf HTTP/1.1" 200 None 2026-03-31 08:56:33,051 - lava_callback - INFO - Completed processing callback for node 69cb8beb2a64fa9aff9372cf 2026-03-31 08:56:33,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:33,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdd2a64fa9aff9372ae/log.txt.gz 2026-03-31 08:56:33,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:33,257 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf02a64fa9aff9372df/lava-logs.yaml 2026-03-31 08:56:33,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:33,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:33,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:33,914 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf02a64fa9aff9372df/lava_callback.json.gz 2026-03-31 08:56:33,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:33,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf02a64fa9aff9372df HTTP/1.1" 200 None 2026-03-31 08:56:33,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:33,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:33,997 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdd2a64fa9aff9372ae/lava-logs.yaml 2026-03-31 08:56:34,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:34,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:34,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:34,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf02a64fa9aff9372df HTTP/1.1" 200 None 2026-03-31 08:56:34,046 - lava_callback - INFO - Completed processing callback for node 69cb8bf02a64fa9aff9372df 2026-03-31 08:56:34,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:34,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bdd2a64fa9aff9372ae/lava_callback.json.gz 2026-03-31 08:56:34,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:34,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bdd2a64fa9aff9372ae HTTP/1.1" 200 None 2026-03-31 08:56:34,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:34,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:34,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:34,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bdd2a64fa9aff9372ae HTTP/1.1" 200 None 2026-03-31 08:56:34,345 - lava_callback - INFO - Completed processing callback for node 69cb8bdd2a64fa9aff9372ae INFO: 172.19.0.1:47682 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:37,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:37,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.19.0.1:43064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43074 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:47,937 - lava_callback - INFO - Received callback for node 69cb8bf82a64fa9aff937315 INFO: 172.19.0.1:43082 - "POST /node/69cb8bf82a64fa9aff937315 HTTP/1.0" 202 Accepted 2026-03-31 08:56:47,945 - lava_callback - INFO - Starting async job submit for node 69cb8bf82a64fa9aff937315 2026-03-31 08:56:47,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:47,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf82a64fa9aff937315 HTTP/1.1" 200 None 2026-03-31 08:56:47,997 - lava_callback - DEBUG - Processing node 69cb8bf82a64fa9aff937315: baseline-nfs-arm64 2026-03-31 08:56:48,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:48,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:48,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf82a64fa9aff937315/log.txt.gz 2026-03-31 08:56:48,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:49,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:49,155 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf82a64fa9aff937315/lava-logs.yaml 2026-03-31 08:56:49,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:49,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:49,321 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf82a64fa9aff937315/lava_callback.json.gz 2026-03-31 08:56:49,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:49,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf82a64fa9aff937315 HTTP/1.1" 200 None 2026-03-31 08:56:49,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:49,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:49,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:49,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf82a64fa9aff937315 HTTP/1.1" 200 None 2026-03-31 08:56:49,429 - lava_callback - INFO - Completed processing callback for node 69cb8bf82a64fa9aff937315 2026-03-31 08:56:49,984 - lava_callback - INFO - Received callback for node 69cb8bf62a64fa9aff93730d INFO: 172.19.0.1:52554 - "POST /node/69cb8bf62a64fa9aff93730d HTTP/1.0" 202 Accepted 2026-03-31 08:56:49,990 - lava_callback - INFO - Starting async job submit for node 69cb8bf62a64fa9aff93730d 2026-03-31 08:56:50,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:50,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf62a64fa9aff93730d HTTP/1.1" 200 None 2026-03-31 08:56:50,042 - lava_callback - DEBUG - Processing node 69cb8bf62a64fa9aff93730d: baseline-nfs-arm64 2026-03-31 08:56:50,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:50,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:50,614 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf62a64fa9aff93730d/log.txt.gz 2026-03-31 08:56:50,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:51,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:51,028 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf62a64fa9aff93730d/lava-logs.yaml 2026-03-31 08:56:51,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:51,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:51,189 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf62a64fa9aff93730d/lava_callback.json.gz 2026-03-31 08:56:51,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:51,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf62a64fa9aff93730d HTTP/1.1" 200 None 2026-03-31 08:56:51,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:51,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:51,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:51,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf62a64fa9aff93730d HTTP/1.1" 200 None 2026-03-31 08:56:51,283 - lava_callback - INFO - Completed processing callback for node 69cb8bf62a64fa9aff93730d 2026-03-31 08:56:55,306 - lava_callback - INFO - Received callback for node 69cb8bfa2a64fa9aff937318 INFO: 172.19.0.1:52560 - "POST /node/69cb8bfa2a64fa9aff937318 HTTP/1.0" 202 Accepted 2026-03-31 08:56:55,313 - lava_callback - INFO - Starting async job submit for node 69cb8bfa2a64fa9aff937318 2026-03-31 08:56:55,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:55,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfa2a64fa9aff937318 HTTP/1.1" 200 None 2026-03-31 08:56:55,371 - lava_callback - DEBUG - Processing node 69cb8bfa2a64fa9aff937318: baseline-nfs-arm64 2026-03-31 08:56:55,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:56,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:56,127 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfa2a64fa9aff937318/log.txt.gz 2026-03-31 08:56:56,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:52574 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:56:56,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:56,687 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfa2a64fa9aff937318/lava-logs.yaml 2026-03-31 08:56:56,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:56:56,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:56:56,873 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfa2a64fa9aff937318/lava_callback.json.gz 2026-03-31 08:56:56,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:56,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfa2a64fa9aff937318 HTTP/1.1" 200 None 2026-03-31 08:56:56,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:56,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:56:56,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:56:56,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bfa2a64fa9aff937318 HTTP/1.1" 200 None 2026-03-31 08:56:56,971 - lava_callback - INFO - Completed processing callback for node 69cb8bfa2a64fa9aff937318 2026-03-31 08:57:02,409 - lava_callback - INFO - Received callback for node 69cb87cc2a64fa9aff93700c INFO: 172.19.0.1:48246 - "POST /node/69cb87cc2a64fa9aff93700c HTTP/1.0" 202 Accepted 2026-03-31 08:57:02,458 - lava_callback - INFO - Starting async job submit for node 69cb87cc2a64fa9aff93700c 2026-03-31 08:57:02,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:02,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87cc2a64fa9aff93700c HTTP/1.1" 200 None 2026-03-31 08:57:02,499 - lava_callback - DEBUG - Processing node 69cb87cc2a64fa9aff93700c: baseline-arm-pengutronix 2026-03-31 08:57:02,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:02,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:02,733 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cc2a64fa9aff93700c/log.txt.gz 2026-03-31 08:57:02,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:02,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:02,945 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cc2a64fa9aff93700c/lava-logs.yaml 2026-03-31 08:57:02,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:03,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:03,075 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb87cc2a64fa9aff93700c/lava_callback.json.gz DEBUG: Setup failed for 69cb87cc2a64fa9aff93700c. Transitting job node result: incomplete -> fail 2026-03-31 08:57:03,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:03,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87cc2a64fa9aff93700c HTTP/1.1" 200 None 2026-03-31 08:57:03,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:03,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 08:57:03,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:03,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87cc2a64fa9aff93700c HTTP/1.1" 200 None 2026-03-31 08:57:03,243 - lava_callback - INFO - Completed processing callback for node 69cb87cc2a64fa9aff93700c INFO: 172.19.0.1:48248 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:57:07,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:07,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:57:11,051 - lava_callback - INFO - Received callback for node 69cb8c102a64fa9aff93735a INFO: 172.19.0.1:57014 - "POST /node/69cb8c102a64fa9aff93735a HTTP/1.0" 202 Accepted 2026-03-31 08:57:11,059 - lava_callback - INFO - Starting async job submit for node 69cb8c102a64fa9aff93735a 2026-03-31 08:57:11,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:11,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c102a64fa9aff93735a HTTP/1.1" 200 None 2026-03-31 08:57:11,103 - lava_callback - DEBUG - Processing node 69cb8c102a64fa9aff93735a: baseline-arm64-broonie 2026-03-31 08:57:11,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:11,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:11,552 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c102a64fa9aff93735a/log.txt.gz 2026-03-31 08:57:11,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:11,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:11,950 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c102a64fa9aff93735a/lava-logs.yaml 2026-03-31 08:57:11,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:12,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:12,098 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c102a64fa9aff93735a/lava_callback.json.gz 2026-03-31 08:57:12,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:12,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c102a64fa9aff93735a HTTP/1.1" 200 None 2026-03-31 08:57:12,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:12,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:12,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:12,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c102a64fa9aff93735a HTTP/1.1" 200 None 2026-03-31 08:57:12,223 - lava_callback - INFO - Completed processing callback for node 69cb8c102a64fa9aff93735a 2026-03-31 08:57:13,608 - lava_callback - INFO - Received callback for node 69cb8c022a64fa9aff937331 INFO: 172.19.0.1:57020 - "POST /node/69cb8c022a64fa9aff937331 HTTP/1.0" 202 Accepted 2026-03-31 08:57:13,615 - lava_callback - INFO - Starting async job submit for node 69cb8c022a64fa9aff937331 2026-03-31 08:57:13,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:13,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c022a64fa9aff937331 HTTP/1.1" 200 None 2026-03-31 08:57:13,653 - lava_callback - DEBUG - Processing node 69cb8c022a64fa9aff937331: baseline-nfs-arm64 2026-03-31 08:57:14,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:14,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:14,166 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c022a64fa9aff937331/log.txt.gz 2026-03-31 08:57:14,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:14,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:14,596 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c022a64fa9aff937331/lava-logs.yaml 2026-03-31 08:57:14,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:14,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:14,775 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c022a64fa9aff937331/lava_callback.json.gz 2026-03-31 08:57:14,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:14,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c022a64fa9aff937331 HTTP/1.1" 200 None 2026-03-31 08:57:14,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:14,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:14,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:14,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c022a64fa9aff937331 HTTP/1.1" 200 None 2026-03-31 08:57:14,900 - lava_callback - INFO - Completed processing callback for node 69cb8c022a64fa9aff937331 2026-03-31 08:57:15,058 - lava_callback - INFO - Received callback for node 69cb8c2d2a64fa9aff9373bb INFO: 172.19.0.1:57034 - "POST /node/69cb8c2d2a64fa9aff9373bb HTTP/1.0" 202 Accepted 2026-03-31 08:57:15,068 - lava_callback - INFO - Starting async job submit for node 69cb8c2d2a64fa9aff9373bb 2026-03-31 08:57:15,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:15,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2d2a64fa9aff9373bb HTTP/1.1" 200 None 2026-03-31 08:57:15,122 - lava_callback - DEBUG - Processing node 69cb8c2d2a64fa9aff9373bb: baseline-arm64-foundriesio INFO: 172.19.0.1:57038 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 08:57:15,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:15,431 - lava_callback - INFO - Received callback for node 69cb8c2e2a64fa9aff9373be INFO: 172.19.0.1:57050 - "POST /node/69cb8c2e2a64fa9aff9373be HTTP/1.0" 202 Accepted 2026-03-31 08:57:15,443 - lava_callback - INFO - Starting async job submit for node 69cb8c2e2a64fa9aff9373be 2026-03-31 08:57:15,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:15,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2e2a64fa9aff9373be HTTP/1.1" 200 None 2026-03-31 08:57:15,499 - lava_callback - DEBUG - Processing node 69cb8c2e2a64fa9aff9373be: baseline-arm64-foundriesio 2026-03-31 08:57:15,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:15,578 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2d2a64fa9aff9373bb/log.txt.gz 2026-03-31 08:57:16,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:16,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:16,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:16,277 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2d2a64fa9aff9373bb/lava-logs.yaml 2026-03-31 08:57:16,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:16,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:16,297 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2e2a64fa9aff9373be/log.txt.gz 2026-03-31 08:57:16,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:16,507 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2d2a64fa9aff9373bb/lava_callback.json.gz INFO: 172.19.0.1:57062 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:57:16,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:16,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:16,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2d2a64fa9aff9373bb HTTP/1.1" 200 None 2026-03-31 08:57:16,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:16,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:16,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:16,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c2d2a64fa9aff9373bb HTTP/1.1" 200 None 2026-03-31 08:57:16,640 - lava_callback - INFO - Completed processing callback for node 69cb8c2d2a64fa9aff9373bb 2026-03-31 08:57:16,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:16,812 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2e2a64fa9aff9373be/lava-logs.yaml 2026-03-31 08:57:16,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:16,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:16,966 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cb8c2e2a64fa9aff9373be/lava_callback.json.gz 2026-03-31 08:57:16,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:17,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2e2a64fa9aff9373be HTTP/1.1" 200 None 2026-03-31 08:57:17,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:17,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:17,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:17,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c2e2a64fa9aff9373be HTTP/1.1" 200 None 2026-03-31 08:57:17,053 - lava_callback - INFO - Completed processing callback for node 69cb8c2e2a64fa9aff9373be 2026-03-31 08:57:19,575 - lava_callback - INFO - Received callback for node 69cb8c282a64fa9aff9373b2 INFO: 172.19.0.1:57078 - "POST /node/69cb8c282a64fa9aff9373b2 HTTP/1.0" 202 Accepted 2026-03-31 08:57:19,581 - lava_callback - INFO - Starting async job submit for node 69cb8c282a64fa9aff9373b2 2026-03-31 08:57:19,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:19,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c282a64fa9aff9373b2 HTTP/1.1" 200 None 2026-03-31 08:57:19,616 - lava_callback - DEBUG - Processing node 69cb8c282a64fa9aff9373b2: baseline-arm64-pengutronix 2026-03-31 08:57:19,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:19,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:19,906 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c282a64fa9aff9373b2/log.txt.gz 2026-03-31 08:57:20,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:20,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:20,257 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c282a64fa9aff9373b2/lava-logs.yaml 2026-03-31 08:57:20,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:20,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:20,435 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c282a64fa9aff9373b2/lava_callback.json.gz 2026-03-31 08:57:20,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:20,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c282a64fa9aff9373b2 HTTP/1.1" 200 None 2026-03-31 08:57:20,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:20,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:20,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:20,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c282a64fa9aff9373b2 HTTP/1.1" 200 None 2026-03-31 08:57:20,520 - lava_callback - INFO - Completed processing callback for node 69cb8c282a64fa9aff9373b2 2026-03-31 08:57:20,750 - lava_callback - INFO - Received callback for node 69cb8bf32a64fa9aff9372e9 INFO: 172.19.0.1:38982 - "POST /node/69cb8bf32a64fa9aff9372e9 HTTP/1.0" 202 Accepted 2026-03-31 08:57:20,757 - lava_callback - INFO - Starting async job submit for node 69cb8bf32a64fa9aff9372e9 2026-03-31 08:57:20,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:20,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf32a64fa9aff9372e9 HTTP/1.1" 200 None 2026-03-31 08:57:20,814 - lava_callback - DEBUG - Processing node 69cb8bf32a64fa9aff9372e9: baseline-nfs-arm64 2026-03-31 08:57:21,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:21,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:21,339 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf32a64fa9aff9372e9/log.txt.gz 2026-03-31 08:57:21,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:21,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:21,737 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf32a64fa9aff9372e9/lava-logs.yaml 2026-03-31 08:57:21,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:21,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:21,873 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf32a64fa9aff9372e9/lava_callback.json.gz 2026-03-31 08:57:21,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:21,902 - lava_callback - INFO - Received callback for node 69cb8c292a64fa9aff9373b5 INFO: 172.19.0.1:38994 - "POST /node/69cb8c292a64fa9aff9373b5 HTTP/1.0" 202 Accepted 2026-03-31 08:57:21,911 - lava_callback - INFO - Starting async job submit for node 69cb8c292a64fa9aff9373b5 2026-03-31 08:57:21,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:21,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf32a64fa9aff9372e9 HTTP/1.1" 200 None 2026-03-31 08:57:21,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:21,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c292a64fa9aff9373b5 HTTP/1.1" 200 None 2026-03-31 08:57:21,950 - lava_callback - DEBUG - Processing node 69cb8c292a64fa9aff9373b5: baseline-arm64-pengutronix 2026-03-31 08:57:21,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:22,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:22,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf32a64fa9aff9372e9 HTTP/1.1" 200 None 2026-03-31 08:57:22,102 - lava_callback - INFO - Completed processing callback for node 69cb8bf32a64fa9aff9372e9 2026-03-31 08:57:22,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:22,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:22,342 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c292a64fa9aff9373b5/log.txt.gz 2026-03-31 08:57:22,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:22,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:22,641 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c292a64fa9aff9373b5/lava-logs.yaml 2026-03-31 08:57:22,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:22,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:22,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c292a64fa9aff9373b5/lava_callback.json.gz 2026-03-31 08:57:22,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:22,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c292a64fa9aff9373b5 HTTP/1.1" 200 None 2026-03-31 08:57:22,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:22,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:22,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:22,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c292a64fa9aff9373b5 HTTP/1.1" 200 None 2026-03-31 08:57:22,890 - lava_callback - INFO - Completed processing callback for node 69cb8c292a64fa9aff9373b5 INFO: 172.19.0.1:38998 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:57:27,182 - lava_callback - INFO - Received callback for node 69c8e96a6a4c026e1634892f INFO: 172.19.0.1:39004 - "POST /node/69c8e96a6a4c026e1634892f HTTP/1.0" 202 Accepted 2026-03-31 08:57:27,188 - lava_callback - INFO - Starting async job submit for node 69c8e96a6a4c026e1634892f 2026-03-31 08:57:27,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:27,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e96a6a4c026e1634892f HTTP/1.1" 200 None 2026-03-31 08:57:27,205 - lava_callback - DEBUG - Processing node 69c8e96a6a4c026e1634892f: baseline-arm-baylibre 2026-03-31 08:57:27,205 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e96a6a4c026e1634892f 2026-03-31 08:57:27,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:27,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e96a6a4c026e1634892f HTTP/1.1" 200 None 2026-03-31 08:57:27,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:27,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 08:57:27,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:27,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e96a6a4c026e1634892f HTTP/1.1" 200 None 2026-03-31 08:57:27,279 - lava_callback - ERROR - Error processing callback for node 69c8e96a6a4c026e1634892f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:57:30,811 - lava_callback - INFO - Received callback for node 69cb8c182a64fa9aff937384 INFO: 172.19.0.1:52956 - "POST /node/69cb8c182a64fa9aff937384 HTTP/1.0" 202 Accepted 2026-03-31 08:57:30,817 - lava_callback - INFO - Starting async job submit for node 69cb8c182a64fa9aff937384 2026-03-31 08:57:30,834 - lava_callback - INFO - Received callback for node 69cb8c2b2a64fa9aff9373b8 INFO: 172.19.0.1:52968 - "POST /node/69cb8c2b2a64fa9aff9373b8 HTTP/1.0" 202 Accepted 2026-03-31 08:57:30,847 - lava_callback - INFO - Starting async job submit for node 69cb8c2b2a64fa9aff9373b8 2026-03-31 08:57:30,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:30,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:30,870 - lava_callback - INFO - Received callback for node 69cb8be42a64fa9aff9372c0 INFO: 172.19.0.1:52972 - "POST /node/69cb8be42a64fa9aff9372c0 HTTP/1.0" 202 Accepted 2026-03-31 08:57:30,876 - lava_callback - INFO - Starting async job submit for node 69cb8be42a64fa9aff9372c0 2026-03-31 08:57:30,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:30,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c182a64fa9aff937384 HTTP/1.1" 200 None 2026-03-31 08:57:30,909 - lava_callback - DEBUG - Processing node 69cb8c182a64fa9aff937384: baseline-arm64-broonie 2026-03-31 08:57:30,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2b2a64fa9aff9373b8 HTTP/1.1" 200 None 2026-03-31 08:57:30,932 - lava_callback - DEBUG - Processing node 69cb8c2b2a64fa9aff9373b8: baseline-arm64-pengutronix 2026-03-31 08:57:31,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be42a64fa9aff9372c0 HTTP/1.1" 200 None 2026-03-31 08:57:31,396 - lava_callback - DEBUG - Processing node 69cb8be42a64fa9aff9372c0: baseline-arm64 2026-03-31 08:57:31,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:31,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:31,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:31,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:31,713 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c182a64fa9aff937384/log.txt.gz 2026-03-31 08:57:31,724 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c2b2a64fa9aff9373b8/log.txt.gz 2026-03-31 08:57:31,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,219 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be42a64fa9aff9372c0/log.txt.gz 2026-03-31 08:57:32,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,385 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c2b2a64fa9aff9373b8/lava-logs.yaml 2026-03-31 08:57:32,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c182a64fa9aff937384/lava-logs.yaml 2026-03-31 08:57:32,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,608 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cb8c2b2a64fa9aff9373b8/lava_callback.json.gz 2026-03-31 08:57:32,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c2b2a64fa9aff9373b8 HTTP/1.1" 200 None 2026-03-31 08:57:32,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:32,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,716 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c182a64fa9aff937384/lava_callback.json.gz 2026-03-31 08:57:32,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c2b2a64fa9aff9373b8 HTTP/1.1" 200 None 2026-03-31 08:57:32,744 - lava_callback - INFO - Completed processing callback for node 69cb8c2b2a64fa9aff9373b8 2026-03-31 08:57:32,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:32,827 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be42a64fa9aff9372c0/lava-logs.yaml 2026-03-31 08:57:32,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c182a64fa9aff937384 HTTP/1.1" 200 None 2026-03-31 08:57:32,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:32,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:32,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:32,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c182a64fa9aff937384 HTTP/1.1" 200 None 2026-03-31 08:57:32,909 - lava_callback - INFO - Completed processing callback for node 69cb8c182a64fa9aff937384 2026-03-31 08:57:33,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:33,069 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be42a64fa9aff9372c0/lava_callback.json.gz 2026-03-31 08:57:33,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:33,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be42a64fa9aff9372c0 HTTP/1.1" 200 None 2026-03-31 08:57:33,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:33,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:33,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:33,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be42a64fa9aff9372c0 HTTP/1.1" 200 None 2026-03-31 08:57:33,244 - lava_callback - INFO - Received callback for node 69cb8c232a64fa9aff9373a0 INFO: 172.19.0.1:52976 - "POST /node/69cb8c232a64fa9aff9373a0 HTTP/1.0" 202 Accepted 2026-03-31 08:57:33,281 - lava_callback - INFO - Starting async job submit for node 69cb8c232a64fa9aff9373a0 2026-03-31 08:57:33,357 - lava_callback - INFO - Completed processing callback for node 69cb8be42a64fa9aff9372c0 2026-03-31 08:57:33,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:33,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c232a64fa9aff9373a0 HTTP/1.1" 200 None 2026-03-31 08:57:33,395 - lava_callback - DEBUG - Processing node 69cb8c232a64fa9aff9373a0: baseline-arm64-clabbe 2026-03-31 08:57:33,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:33,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:33,822 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cb8c232a64fa9aff9373a0/log.txt.gz 2026-03-31 08:57:33,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:34,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:34,125 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cb8c232a64fa9aff9373a0/lava-logs.yaml 2026-03-31 08:57:34,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:34,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:34,256 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cb8c232a64fa9aff9373a0/lava_callback.json.gz 2026-03-31 08:57:34,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:34,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c232a64fa9aff9373a0 HTTP/1.1" 200 None 2026-03-31 08:57:34,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:34,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:34,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:34,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c232a64fa9aff9373a0 HTTP/1.1" 200 None 2026-03-31 08:57:34,335 - lava_callback - INFO - Completed processing callback for node 69cb8c232a64fa9aff9373a0 INFO: 172.19.0.1:52990 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:57:37,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:37,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:57:39,455 - lava_callback - INFO - Received callback for node 69cb8bfd2a64fa9aff93731e INFO: 172.19.0.1:52996 - "POST /node/69cb8bfd2a64fa9aff93731e HTTP/1.0" 202 Accepted 2026-03-31 08:57:39,462 - lava_callback - INFO - Starting async job submit for node 69cb8bfd2a64fa9aff93731e 2026-03-31 08:57:39,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:39,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfd2a64fa9aff93731e HTTP/1.1" 200 None 2026-03-31 08:57:39,501 - lava_callback - DEBUG - Processing node 69cb8bfd2a64fa9aff93731e: baseline-nfs-arm64 2026-03-31 08:57:39,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:39,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:39,926 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfd2a64fa9aff93731e/log.txt.gz 2026-03-31 08:57:40,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:40,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:40,297 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfd2a64fa9aff93731e/lava-logs.yaml 2026-03-31 08:57:40,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:40,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:40,430 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfd2a64fa9aff93731e/lava_callback.json.gz 2026-03-31 08:57:40,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:40,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfd2a64fa9aff93731e HTTP/1.1" 200 None 2026-03-31 08:57:40,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:40,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:40,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:40,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bfd2a64fa9aff93731e HTTP/1.1" 200 None 2026-03-31 08:57:40,529 - lava_callback - INFO - Completed processing callback for node 69cb8bfd2a64fa9aff93731e INFO: 172.19.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56382 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:57:50,075 - lava_callback - INFO - Received callback for node 69cb8c132a64fa9aff937377 INFO: 172.19.0.1:37136 - "POST /node/69cb8c132a64fa9aff937377 HTTP/1.0" 202 Accepted 2026-03-31 08:57:50,082 - lava_callback - INFO - Starting async job submit for node 69cb8c132a64fa9aff937377 2026-03-31 08:57:50,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:50,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c132a64fa9aff937377 HTTP/1.1" 200 None 2026-03-31 08:57:50,121 - lava_callback - DEBUG - Processing node 69cb8c132a64fa9aff937377: baseline-arm64-broonie 2026-03-31 08:57:50,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:50,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:50,506 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c132a64fa9aff937377/log.txt.gz 2026-03-31 08:57:50,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:50,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:50,858 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c132a64fa9aff937377/lava-logs.yaml 2026-03-31 08:57:50,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:50,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:50,993 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c132a64fa9aff937377/lava_callback.json.gz 2026-03-31 08:57:51,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:51,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c132a64fa9aff937377 HTTP/1.1" 200 None 2026-03-31 08:57:51,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:51,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:51,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:51,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c132a64fa9aff937377 HTTP/1.1" 200 None 2026-03-31 08:57:51,089 - lava_callback - INFO - Completed processing callback for node 69cb8c132a64fa9aff937377 2026-03-31 08:57:51,536 - lava_callback - INFO - Received callback for node 69cb8c152a64fa9aff93737a INFO: 172.19.0.1:37152 - "POST /node/69cb8c152a64fa9aff93737a HTTP/1.0" 202 Accepted 2026-03-31 08:57:51,543 - lava_callback - INFO - Starting async job submit for node 69cb8c152a64fa9aff93737a 2026-03-31 08:57:51,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:51,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c152a64fa9aff93737a HTTP/1.1" 200 None 2026-03-31 08:57:51,589 - lava_callback - DEBUG - Processing node 69cb8c152a64fa9aff93737a: baseline-arm64-broonie 2026-03-31 08:57:51,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:52,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:52,038 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c152a64fa9aff93737a/log.txt.gz 2026-03-31 08:57:52,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:52,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:52,437 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c152a64fa9aff93737a/lava-logs.yaml 2026-03-31 08:57:52,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:52,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:52,599 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c152a64fa9aff93737a/lava_callback.json.gz 2026-03-31 08:57:52,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:52,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c152a64fa9aff93737a HTTP/1.1" 200 None 2026-03-31 08:57:52,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:52,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:52,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:52,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c152a64fa9aff93737a HTTP/1.1" 200 None 2026-03-31 08:57:52,707 - lava_callback - INFO - Completed processing callback for node 69cb8c152a64fa9aff93737a 2026-03-31 08:57:52,929 - lava_callback - INFO - Received callback for node 69cb8c1c2a64fa9aff93738e INFO: 172.19.0.1:37168 - "POST /node/69cb8c1c2a64fa9aff93738e HTTP/1.0" 202 Accepted 2026-03-31 08:57:52,935 - lava_callback - INFO - Starting async job submit for node 69cb8c1c2a64fa9aff93738e 2026-03-31 08:57:52,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:52,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1c2a64fa9aff93738e HTTP/1.1" 200 None 2026-03-31 08:57:52,975 - lava_callback - DEBUG - Processing node 69cb8c1c2a64fa9aff93738e: baseline-arm64-broonie 2026-03-31 08:57:53,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:53,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:53,355 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1c2a64fa9aff93738e/log.txt.gz 2026-03-31 08:57:53,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:53,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:53,670 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1c2a64fa9aff93738e/lava-logs.yaml 2026-03-31 08:57:53,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:57:53,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:57:53,811 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1c2a64fa9aff93738e/lava_callback.json.gz 2026-03-31 08:57:53,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:53,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1c2a64fa9aff93738e HTTP/1.1" 200 None 2026-03-31 08:57:53,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:53,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:57:53,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:57:53,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c1c2a64fa9aff93738e HTTP/1.1" 200 None 2026-03-31 08:57:53,902 - lava_callback - INFO - Completed processing callback for node 69cb8c1c2a64fa9aff93738e INFO: 172.19.0.1:37172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54584 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:58:07,149 - lava_callback - INFO - Received callback for node 69c8e98e6a4c026e1634893b INFO: 172.19.0.1:54592 - "POST /node/69c8e98e6a4c026e1634893b HTTP/1.0" 202 Accepted 2026-03-31 08:58:07,156 - lava_callback - INFO - Starting async job submit for node 69c8e98e6a4c026e1634893b 2026-03-31 08:58:07,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:07,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e98e6a4c026e1634893b HTTP/1.1" 200 None 2026-03-31 08:58:07,172 - lava_callback - DEBUG - Processing node 69c8e98e6a4c026e1634893b: baseline-arm-baylibre 2026-03-31 08:58:07,172 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e98e6a4c026e1634893b 2026-03-31 08:58:07,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:07,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e98e6a4c026e1634893b HTTP/1.1" 200 None 2026-03-31 08:58:07,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:07,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 08:58:07,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:07,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e98e6a4c026e1634893b HTTP/1.1" 200 None 2026-03-31 08:58:07,250 - lava_callback - ERROR - Error processing callback for node 69c8e98e6a4c026e1634893b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:58:07,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:07,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:58:10,019 - lava_callback - INFO - Received callback for node 69cb8c1a2a64fa9aff937389 INFO: 172.19.0.1:51080 - "POST /node/69cb8c1a2a64fa9aff937389 HTTP/1.0" 202 Accepted 2026-03-31 08:58:10,029 - lava_callback - INFO - Starting async job submit for node 69cb8c1a2a64fa9aff937389 2026-03-31 08:58:10,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:10,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1a2a64fa9aff937389 HTTP/1.1" 200 None 2026-03-31 08:58:10,073 - lava_callback - DEBUG - Processing node 69cb8c1a2a64fa9aff937389: baseline-arm64-broonie 2026-03-31 08:58:10,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:10,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:10,552 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1a2a64fa9aff937389/log.txt.gz 2026-03-31 08:58:10,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:10,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:10,954 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1a2a64fa9aff937389/lava-logs.yaml 2026-03-31 08:58:10,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:10,994 - lava_callback - INFO - Received callback for node 69cb87fb2a64fa9aff937088 INFO: 172.19.0.1:51086 - "POST /node/69cb87fb2a64fa9aff937088 HTTP/1.0" 202 Accepted 2026-03-31 08:58:11,005 - lava_callback - INFO - Starting async job submit for node 69cb87fb2a64fa9aff937088 2026-03-31 08:58:11,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:11,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87fb2a64fa9aff937088 HTTP/1.1" 200 None 2026-03-31 08:58:11,088 - lava_callback - DEBUG - Processing node 69cb87fb2a64fa9aff937088: ltp-smoke 2026-03-31 08:58:11,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:11,161 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1a2a64fa9aff937389/lava_callback.json.gz 2026-03-31 08:58:11,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:11,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1a2a64fa9aff937389 HTTP/1.1" 200 None 2026-03-31 08:58:11,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:11,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:58:11,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:11,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c1a2a64fa9aff937389 HTTP/1.1" 200 None 2026-03-31 08:58:11,602 - lava_callback - INFO - Completed processing callback for node 69cb8c1a2a64fa9aff937389 2026-03-31 08:58:11,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:11,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:11,809 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87fb2a64fa9aff937088/log.txt.gz 2026-03-31 08:58:12,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:12,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:12,277 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87fb2a64fa9aff937088/lava-logs.yaml 2026-03-31 08:58:12,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:12,406 - lava_callback - INFO - Received callback for node 69cb8c1e2a64fa9aff937391 INFO: 172.19.0.1:51088 - "POST /node/69cb8c1e2a64fa9aff937391 HTTP/1.0" 202 Accepted 2026-03-31 08:58:12,414 - lava_callback - INFO - Starting async job submit for node 69cb8c1e2a64fa9aff937391 2026-03-31 08:58:12,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:12,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:12,461 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb87fb2a64fa9aff937088/lava_callback.json.gz 2026-03-31 08:58:12,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1e2a64fa9aff937391 HTTP/1.1" 200 None 2026-03-31 08:58:12,478 - lava_callback - DEBUG - Processing node 69cb8c1e2a64fa9aff937391: baseline-arm64-broonie 2026-03-31 08:58:12,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:12,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb87fb2a64fa9aff937088 HTTP/1.1" 200 None 2026-03-31 08:58:12,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:12,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:12,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83462a64fa9aff936905 HTTP/1.1" 200 None 2026-03-31 08:58:12,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:12,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb87fb2a64fa9aff937088 HTTP/1.1" 200 None 2026-03-31 08:58:13,152 - lava_callback - INFO - Completed processing callback for node 69cb87fb2a64fa9aff937088 2026-03-31 08:58:13,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:13,180 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1e2a64fa9aff937391/log.txt.gz 2026-03-31 08:58:13,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:13,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:13,571 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1e2a64fa9aff937391/lava-logs.yaml 2026-03-31 08:58:13,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:13,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:13,743 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c1e2a64fa9aff937391/lava_callback.json.gz 2026-03-31 08:58:13,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:13,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c1e2a64fa9aff937391 HTTP/1.1" 200 None 2026-03-31 08:58:13,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:13,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:58:13,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:13,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c1e2a64fa9aff937391 HTTP/1.1" 200 None 2026-03-31 08:58:13,832 - lava_callback - INFO - Completed processing callback for node 69cb8c1e2a64fa9aff937391 INFO: 172.19.0.1:51094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41306 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:58:30,871 - lava_callback - INFO - Received callback for node 69cb8c112a64fa9aff93735d INFO: 172.19.0.1:39276 - "POST /node/69cb8c112a64fa9aff93735d HTTP/1.0" 202 Accepted 2026-03-31 08:58:30,876 - lava_callback - INFO - Starting async job submit for node 69cb8c112a64fa9aff93735d 2026-03-31 08:58:30,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:30,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c112a64fa9aff93735d HTTP/1.1" 200 None 2026-03-31 08:58:30,915 - lava_callback - DEBUG - Processing node 69cb8c112a64fa9aff93735d: baseline-arm64-broonie 2026-03-31 08:58:31,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:31,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:31,275 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c112a64fa9aff93735d/log.txt.gz 2026-03-31 08:58:31,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:31,599 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:31,601 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c112a64fa9aff93735d/lava-logs.yaml 2026-03-31 08:58:31,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:31,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:31,755 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c112a64fa9aff93735d/lava_callback.json.gz 2026-03-31 08:58:31,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:31,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c112a64fa9aff93735d HTTP/1.1" 200 None 2026-03-31 08:58:31,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:31,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:58:31,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:31,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c112a64fa9aff93735d HTTP/1.1" 200 None 2026-03-31 08:58:31,861 - lava_callback - INFO - Completed processing callback for node 69cb8c112a64fa9aff93735d INFO: 172.19.0.1:39284 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:58:37,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:37,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 08:58:39,394 - lava_callback - INFO - Received callback for node 69cb8be32a64fa9aff9372bd INFO: 172.19.0.1:39294 - "POST /node/69cb8be32a64fa9aff9372bd HTTP/1.0" 202 Accepted 2026-03-31 08:58:39,400 - lava_callback - INFO - Starting async job submit for node 69cb8be32a64fa9aff9372bd 2026-03-31 08:58:39,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:39,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be32a64fa9aff9372bd HTTP/1.1" 200 None 2026-03-31 08:58:39,434 - lava_callback - DEBUG - Processing node 69cb8be32a64fa9aff9372bd: baseline-arm64 2026-03-31 08:58:39,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:40,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:40,073 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be32a64fa9aff9372bd/log.txt.gz 2026-03-31 08:58:40,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:40,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:40,563 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be32a64fa9aff9372bd/lava-logs.yaml 2026-03-31 08:58:40,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:40,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:40,731 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be32a64fa9aff9372bd/lava_callback.json.gz 2026-03-31 08:58:40,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:40,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be32a64fa9aff9372bd HTTP/1.1" 200 None 2026-03-31 08:58:40,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:40,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:58:40,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:40,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be32a64fa9aff9372bd HTTP/1.1" 200 None 2026-03-31 08:58:40,835 - lava_callback - INFO - Completed processing callback for node 69cb8be32a64fa9aff9372bd INFO: 172.19.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52878 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:58:47,204 - lava_callback - INFO - Received callback for node 69cb8c002a64fa9aff937324 INFO: 172.19.0.1:52884 - "POST /node/69cb8c002a64fa9aff937324 HTTP/1.0" 202 Accepted 2026-03-31 08:58:47,211 - lava_callback - INFO - Starting async job submit for node 69cb8c002a64fa9aff937324 2026-03-31 08:58:47,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:47,244 - lava_callback - INFO - Received callback for node 69c8e9b66a4c026e16348947 INFO: 172.19.0.1:52898 - "POST /node/69c8e9b66a4c026e16348947 HTTP/1.0" 202 Accepted 2026-03-31 08:58:47,250 - lava_callback - INFO - Starting async job submit for node 69c8e9b66a4c026e16348947 2026-03-31 08:58:47,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:47,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c002a64fa9aff937324 HTTP/1.1" 200 None 2026-03-31 08:58:47,254 - lava_callback - DEBUG - Processing node 69cb8c002a64fa9aff937324: baseline-nfs-arm64 2026-03-31 08:58:47,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9b66a4c026e16348947 HTTP/1.1" 200 None 2026-03-31 08:58:47,318 - lava_callback - DEBUG - Processing node 69c8e9b66a4c026e16348947: baseline-arm-clabbe 2026-03-31 08:58:47,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e9b66a4c026e16348947 2026-03-31 08:58:47,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:47,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9b66a4c026e16348947 HTTP/1.1" 200 None 2026-03-31 08:58:47,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:47,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 08:58:47,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:47,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:47,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e9b66a4c026e16348947 HTTP/1.1" 200 None 2026-03-31 08:58:47,656 - lava_callback - ERROR - Error processing callback for node 69c8e9b66a4c026e16348947: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:58:47,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:47,892 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c002a64fa9aff937324/log.txt.gz 2026-03-31 08:58:48,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:48,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:48,333 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c002a64fa9aff937324/lava-logs.yaml 2026-03-31 08:58:48,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:58:48,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:58:48,513 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8c002a64fa9aff937324/lava_callback.json.gz 2026-03-31 08:58:48,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:48,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c002a64fa9aff937324 HTTP/1.1" 200 None 2026-03-31 08:58:48,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:48,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:58:48,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:58:48,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c002a64fa9aff937324 HTTP/1.1" 200 None 2026-03-31 08:58:48,608 - lava_callback - INFO - Completed processing callback for node 69cb8c002a64fa9aff937324 INFO: 172.19.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35032 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:59:07,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:07,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:43302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44714 - "GET / HTTP/1.0" 200 OK 2026-03-31 08:59:47,160 - lava_callback - INFO - Received callback for node 69cb8bf42a64fa9aff93730a INFO: 172.19.0.1:44722 - "POST /node/69cb8bf42a64fa9aff93730a HTTP/1.0" 202 Accepted 2026-03-31 08:59:47,166 - lava_callback - INFO - Starting async job submit for node 69cb8bf42a64fa9aff93730a 2026-03-31 08:59:47,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:47,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf42a64fa9aff93730a HTTP/1.1" 200 None 2026-03-31 08:59:47,206 - lava_callback - DEBUG - Processing node 69cb8bf42a64fa9aff93730a: baseline-nfs-arm64 2026-03-31 08:59:47,251 - lava_callback - INFO - Received callback for node 69c8e9fe6a4c026e16348963 INFO: 172.19.0.1:44732 - "POST /node/69c8e9fe6a4c026e16348963 HTTP/1.0" 202 Accepted 2026-03-31 08:59:47,272 - lava_callback - INFO - Starting async job submit for node 69c8e9fe6a4c026e16348963 2026-03-31 08:59:47,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:47,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9fe6a4c026e16348963 HTTP/1.1" 200 None 2026-03-31 08:59:47,378 - lava_callback - DEBUG - Processing node 69c8e9fe6a4c026e16348963: baseline-arm-clabbe 2026-03-31 08:59:47,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8e9fe6a4c026e16348963 2026-03-31 08:59:47,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:47,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8e9fe6a4c026e16348963 HTTP/1.1" 200 None 2026-03-31 08:59:47,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:47,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 08:59:47,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:47,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:47,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8e9fe6a4c026e16348963 HTTP/1.1" 200 None 2026-03-31 08:59:47,677 - lava_callback - ERROR - Error processing callback for node 69c8e9fe6a4c026e16348963: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 08:59:47,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:47,877 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf42a64fa9aff93730a/log.txt.gz 2026-03-31 08:59:48,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:48,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:48,433 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf42a64fa9aff93730a/lava-logs.yaml 2026-03-31 08:59:48,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:48,625 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:48,626 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf42a64fa9aff93730a/lava_callback.json.gz 2026-03-31 08:59:48,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:48,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf42a64fa9aff93730a HTTP/1.1" 200 None 2026-03-31 08:59:48,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:48,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:59:48,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:48,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf42a64fa9aff93730a HTTP/1.1" 200 None 2026-03-31 08:59:48,719 - lava_callback - INFO - Completed processing callback for node 69cb8bf42a64fa9aff93730a 2026-03-31 08:59:51,253 - lava_callback - INFO - Received callback for node 69cb8c172a64fa9aff93737d INFO: 172.19.0.1:36542 - "POST /node/69cb8c172a64fa9aff93737d HTTP/1.0" 202 Accepted 2026-03-31 08:59:51,259 - lava_callback - INFO - Starting async job submit for node 69cb8c172a64fa9aff93737d 2026-03-31 08:59:51,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:51,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c172a64fa9aff93737d HTTP/1.1" 200 None 2026-03-31 08:59:51,297 - lava_callback - DEBUG - Processing node 69cb8c172a64fa9aff93737d: baseline-arm64-broonie 2026-03-31 08:59:51,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:51,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:51,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c172a64fa9aff93737d/log.txt.gz 2026-03-31 08:59:51,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:52,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:52,022 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c172a64fa9aff93737d/lava-logs.yaml 2026-03-31 08:59:52,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 08:59:52,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 08:59:52,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cb8c172a64fa9aff93737d/lava_callback.json.gz 2026-03-31 08:59:52,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:52,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8c172a64fa9aff93737d HTTP/1.1" 200 None 2026-03-31 08:59:52,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:52,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 08:59:52,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 08:59:52,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8c172a64fa9aff93737d HTTP/1.1" 200 None 2026-03-31 08:59:52,310 - lava_callback - INFO - Completed processing callback for node 69cb8c172a64fa9aff93737d INFO: 172.19.0.1:36546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41314 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:00:07,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:00:08,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:34634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54506 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:00:27,232 - lava_callback - INFO - Received callback for node 69c8ea236a4c026e16348970 INFO: 172.19.0.1:54508 - "POST /node/69c8ea236a4c026e16348970 HTTP/1.0" 202 Accepted 2026-03-31 09:00:27,237 - lava_callback - INFO - Starting async job submit for node 69c8ea236a4c026e16348970 2026-03-31 09:00:27,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:00:27,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ea236a4c026e16348970 HTTP/1.1" 200 None 2026-03-31 09:00:27,254 - lava_callback - DEBUG - Processing node 69c8ea236a4c026e16348970: baseline-arm-clabbe 2026-03-31 09:00:27,255 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ea236a4c026e16348970 2026-03-31 09:00:27,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:00:27,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ea236a4c026e16348970 HTTP/1.1" 200 None 2026-03-31 09:00:27,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:00:27,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 09:00:27,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:00:27,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ea236a4c026e16348970 HTTP/1.1" 200 None 2026-03-31 09:00:27,322 - lava_callback - ERROR - Error processing callback for node 69c8ea236a4c026e16348970: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49304 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:01:07,296 - lava_callback - INFO - Received callback for node 69c8ea496a4c026e1634897d INFO: 172.19.0.1:49312 - "POST /node/69c8ea496a4c026e1634897d HTTP/1.0" 202 Accepted 2026-03-31 09:01:07,302 - lava_callback - INFO - Starting async job submit for node 69c8ea496a4c026e1634897d 2026-03-31 09:01:07,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:07,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ea496a4c026e1634897d HTTP/1.1" 200 None 2026-03-31 09:01:07,320 - lava_callback - DEBUG - Processing node 69c8ea496a4c026e1634897d: baseline-arm-clabbe 2026-03-31 09:01:07,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ea496a4c026e1634897d 2026-03-31 09:01:07,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:07,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ea496a4c026e1634897d HTTP/1.1" 200 None 2026-03-31 09:01:07,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:07,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 09:01:07,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:07,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ea496a4c026e1634897d HTTP/1.1" 200 None 2026-03-31 09:01:07,382 - lava_callback - ERROR - Error processing callback for node 69c8ea496a4c026e1634897d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 09:01:10,228 - lava_callback - INFO - Received callback for node 69cb8d402a64fa9aff93752a INFO: 172.19.0.1:59542 - "POST /node/69cb8d402a64fa9aff93752a HTTP/1.0" 202 Accepted 2026-03-31 09:01:10,234 - lava_callback - INFO - Starting async job submit for node 69cb8d402a64fa9aff93752a 2026-03-31 09:01:10,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:10,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8d402a64fa9aff93752a HTTP/1.1" 200 None 2026-03-31 09:01:10,251 - lava_callback - DEBUG - Processing node 69cb8d402a64fa9aff93752a: baseline-arm 2026-03-31 09:01:10,257 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:01:10,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:01:10,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:01:10,382 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb8d402a64fa9aff93752a/lava-logs.yaml 2026-03-31 09:01:10,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:01:10,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:01:10,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb8d402a64fa9aff93752a/lava_callback.json.gz 2026-03-31 09:01:10,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:10,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8d402a64fa9aff93752a HTTP/1.1" 200 None 2026-03-31 09:01:10,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:10,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 09:01:10,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:10,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8d402a64fa9aff93752a HTTP/1.1" 200 None 2026-03-31 09:01:10,566 - lava_callback - INFO - Completed processing callback for node 69cb8d402a64fa9aff93752a INFO: 172.19.0.1:59556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50748 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:01:38,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:01:38,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:41834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41276 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:02:42,723 - lava_callback - INFO - Received callback for node 69cb8bf22a64fa9aff9372e6 INFO: 172.19.0.1:50108 - "POST /node/69cb8bf22a64fa9aff9372e6 HTTP/1.0" 202 Accepted 2026-03-31 09:02:42,730 - lava_callback - INFO - Starting async job submit for node 69cb8bf22a64fa9aff9372e6 2026-03-31 09:02:42,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:02:42,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf22a64fa9aff9372e6 HTTP/1.1" 200 None 2026-03-31 09:02:42,785 - lava_callback - DEBUG - Processing node 69cb8bf22a64fa9aff9372e6: baseline-arm64 2026-03-31 09:02:43,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:02:43,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:02:43,309 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf22a64fa9aff9372e6/log.txt.gz 2026-03-31 09:02:43,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:02:43,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:02:43,765 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf22a64fa9aff9372e6/lava-logs.yaml 2026-03-31 09:02:43,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:02:43,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:02:43,901 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bf22a64fa9aff9372e6/lava_callback.json.gz DEBUG: Setup failed for 69cb8bf22a64fa9aff9372e6. Transitting job node result: incomplete -> fail 2026-03-31 09:02:43,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:02:43,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf22a64fa9aff9372e6 HTTP/1.1" 200 None 2026-03-31 09:02:43,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:02:44,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:02:44,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:02:44,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf22a64fa9aff9372e6 HTTP/1.1" 200 None 2026-03-31 09:02:44,124 - lava_callback - INFO - Completed processing callback for node 69cb8bf22a64fa9aff9372e6 INFO: 172.19.0.1:50120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50618 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:03:07,694 - lava_callback - INFO - Received callback for node 69cb8be82a64fa9aff9372c9 INFO: 172.19.0.1:50626 - "POST /node/69cb8be82a64fa9aff9372c9 HTTP/1.0" 202 Accepted 2026-03-31 09:03:07,701 - lava_callback - INFO - Starting async job submit for node 69cb8be82a64fa9aff9372c9 2026-03-31 09:03:07,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:07,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be82a64fa9aff9372c9 HTTP/1.1" 200 None 2026-03-31 09:03:07,750 - lava_callback - DEBUG - Processing node 69cb8be82a64fa9aff9372c9: baseline-arm64 2026-03-31 09:03:08,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:08,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 09:03:08,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:03:08,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:03:08,722 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be82a64fa9aff9372c9/log.txt.gz 2026-03-31 09:03:09,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:03:09,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:03:09,383 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be82a64fa9aff9372c9/lava-logs.yaml 2026-03-31 09:03:09,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:03:09,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:03:09,561 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8be82a64fa9aff9372c9/lava_callback.json.gz 2026-03-31 09:03:09,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:09,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8be82a64fa9aff9372c9 HTTP/1.1" 200 None 2026-03-31 09:03:09,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:09,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:03:09,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:09,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8be82a64fa9aff9372c9 HTTP/1.1" 200 None 2026-03-31 09:03:09,658 - lava_callback - INFO - Completed processing callback for node 69cb8be82a64fa9aff9372c9 INFO: 172.19.0.1:35938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32860 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:03:38,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:03:38,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:32862 - "GET /node/69b90f330ec88e58bad0d13c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47010 - "GET /node/69b912060ec88e58bad0d7b1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:47024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51444 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:07:27,636 - lava_callback - INFO - Received callback for node 69c8ebbe6a4c026e163489ca INFO: 172.19.0.1:51450 - "POST /node/69c8ebbe6a4c026e163489ca HTTP/1.0" 202 Accepted 2026-03-31 09:07:27,645 - lava_callback - INFO - Starting async job submit for node 69c8ebbe6a4c026e163489ca 2026-03-31 09:07:27,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:07:27,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ebbe6a4c026e163489ca HTTP/1.1" 200 None 2026-03-31 09:07:27,687 - lava_callback - DEBUG - Processing node 69c8ebbe6a4c026e163489ca: baseline-arm-clabbe 2026-03-31 09:07:27,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ebbe6a4c026e163489ca 2026-03-31 09:07:27,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:07:27,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ebbe6a4c026e163489ca HTTP/1.1" 200 None 2026-03-31 09:07:27,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:07:27,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 09:07:27,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:07:27,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ebbe6a4c026e163489ca HTTP/1.1" 200 None 2026-03-31 09:07:27,775 - lava_callback - ERROR - Error processing callback for node 69c8ebbe6a4c026e163489ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38338 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:15:20,180 - lava_callback - INFO - Received callback for node 69cb8bf72a64fa9aff937312 INFO: 172.19.0.1:35978 - "POST /node/69cb8bf72a64fa9aff937312 HTTP/1.0" 202 Accepted 2026-03-31 09:15:20,213 - lava_callback - INFO - Starting async job submit for node 69cb8bf72a64fa9aff937312 2026-03-31 09:15:20,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:15:20,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf72a64fa9aff937312 HTTP/1.1" 200 None 2026-03-31 09:15:20,404 - lava_callback - DEBUG - Processing node 69cb8bf72a64fa9aff937312: baseline-nfs-arm64 2026-03-31 09:15:21,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:15:21,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:15:21,698 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf72a64fa9aff937312/log.txt.gz 2026-03-31 09:15:22,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:15:22,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:15:22,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf72a64fa9aff937312/lava-logs.yaml 2026-03-31 09:15:22,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:15:22,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:15:22,809 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bf72a64fa9aff937312/lava_callback.json.gz DEBUG: Setup failed for 69cb8bf72a64fa9aff937312. Transitting job node result: incomplete -> fail 2026-03-31 09:15:22,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:15:22,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bf72a64fa9aff937312 HTTP/1.1" 200 None 2026-03-31 09:15:22,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:15:22,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:15:22,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:15:22,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bf72a64fa9aff937312 HTTP/1.1" 200 None 2026-03-31 09:15:23,053 - lava_callback - INFO - Completed processing callback for node 69cb8bf72a64fa9aff937312 INFO: 172.19.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40032 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:15:38,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:15:38,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38200 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:20:00,803 - lava_callback - INFO - Received callback for node 69cb8bed2a64fa9aff9372d5 INFO: 172.19.0.1:37416 - "POST /node/69cb8bed2a64fa9aff9372d5 HTTP/1.0" 202 Accepted 2026-03-31 09:20:00,810 - lava_callback - INFO - Starting async job submit for node 69cb8bed2a64fa9aff9372d5 2026-03-31 09:20:00,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:20:00,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bed2a64fa9aff9372d5 HTTP/1.1" 200 None 2026-03-31 09:20:00,851 - lava_callback - DEBUG - Processing node 69cb8bed2a64fa9aff9372d5: baseline-arm64 2026-03-31 09:20:01,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:20:01,537 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:20:01,539 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bed2a64fa9aff9372d5/log.txt.gz 2026-03-31 09:20:01,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:20:02,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:20:02,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bed2a64fa9aff9372d5/lava-logs.yaml 2026-03-31 09:20:02,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:20:02,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:20:02,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb8bed2a64fa9aff9372d5/lava_callback.json.gz 2026-03-31 09:20:02,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:20:02,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bed2a64fa9aff9372d5 HTTP/1.1" 200 None 2026-03-31 09:20:02,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:20:02,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:20:02,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:20:02,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bed2a64fa9aff9372d5 HTTP/1.1" 200 None 2026-03-31 09:20:02,417 - lava_callback - INFO - Completed processing callback for node 69cb8bed2a64fa9aff9372d5 INFO: 172.19.0.1:37422 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:20:08,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:20:08,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:57298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44396 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:21:10,498 - lava_callback - INFO - Received callback for node 69cb91ef2a64fa9aff93760b INFO: 172.19.0.1:46708 - "POST /node/69cb91ef2a64fa9aff93760b HTTP/1.0" 202 Accepted 2026-03-31 09:21:10,508 - lava_callback - INFO - Starting async job submit for node 69cb91ef2a64fa9aff93760b 2026-03-31 09:21:10,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:21:10,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb91ef2a64fa9aff93760b HTTP/1.1" 200 None 2026-03-31 09:21:10,537 - lava_callback - DEBUG - Processing node 69cb91ef2a64fa9aff93760b: baseline-nfs-arm64 2026-03-31 09:21:10,539 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:21:10,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:21:10,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:21:10,708 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb91ef2a64fa9aff93760b/lava-logs.yaml 2026-03-31 09:21:10,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:21:10,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:21:10,845 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb91ef2a64fa9aff93760b/lava_callback.json.gz 2026-03-31 09:21:10,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:21:10,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb91ef2a64fa9aff93760b HTTP/1.1" 200 None 2026-03-31 09:21:10,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:21:10,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:21:10,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:21:10,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb91ef2a64fa9aff93760b HTTP/1.1" 200 None 2026-03-31 09:21:10,928 - lava_callback - INFO - Completed processing callback for node 69cb91ef2a64fa9aff93760b INFO: 172.19.0.1:46714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60774 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:21:38,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:21:38,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36604 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:24:08,612 - lava_callback - INFO - Received callback for node 69c8efab6a4c026e16348ae6 INFO: 172.19.0.1:36606 - "POST /node/69c8efab6a4c026e16348ae6 HTTP/1.0" 202 Accepted 2026-03-31 09:24:08,618 - lava_callback - INFO - Starting async job submit for node 69c8efab6a4c026e16348ae6 2026-03-31 09:24:08,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:08,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8efab6a4c026e16348ae6 HTTP/1.1" 200 None 2026-03-31 09:24:08,640 - lava_callback - DEBUG - Processing node 69c8efab6a4c026e16348ae6: baseline-arm64-baylibre 2026-03-31 09:24:08,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8efab6a4c026e16348ae6 2026-03-31 09:24:08,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:08,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8efab6a4c026e16348ae6 HTTP/1.1" 200 None 2026-03-31 09:24:08,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:08,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:24:08,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:08,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8efab6a4c026e16348ae6 HTTP/1.1" 200 None 2026-03-31 09:24:08,753 - lava_callback - ERROR - Error processing callback for node 69c8efab6a4c026e16348ae6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 09:24:11,264 - lava_callback - INFO - Received callback for node 69cb8bfe2a64fa9aff937321 INFO: 172.19.0.1:42568 - "POST /node/69cb8bfe2a64fa9aff937321 HTTP/1.0" 202 Accepted 2026-03-31 09:24:11,272 - lava_callback - INFO - Starting async job submit for node 69cb8bfe2a64fa9aff937321 2026-03-31 09:24:11,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:11,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfe2a64fa9aff937321 HTTP/1.1" 200 None 2026-03-31 09:24:11,320 - lava_callback - DEBUG - Processing node 69cb8bfe2a64fa9aff937321: baseline-nfs-arm64 2026-03-31 09:24:11,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:24:12,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:24:12,090 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfe2a64fa9aff937321/log.txt.gz 2026-03-31 09:24:12,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:24:12,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:24:12,562 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfe2a64fa9aff937321/lava-logs.yaml 2026-03-31 09:24:12,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:24:12,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:24:12,746 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb8bfe2a64fa9aff937321/lava_callback.json.gz 2026-03-31 09:24:12,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:12,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb8bfe2a64fa9aff937321 HTTP/1.1" 200 None 2026-03-31 09:24:12,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:12,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:24:12,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:12,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb8bfe2a64fa9aff937321 HTTP/1.1" 200 None 2026-03-31 09:24:12,901 - lava_callback - INFO - Completed processing callback for node 69cb8bfe2a64fa9aff937321 INFO: 172.19.0.1:42584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49322 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:24:38,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:24:38,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43538 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:25:28,661 - lava_callback - INFO - Received callback for node 69c8effb6a4c026e16348af2 INFO: 172.19.0.1:43548 - "POST /node/69c8effb6a4c026e16348af2 HTTP/1.0" 202 Accepted 2026-03-31 09:25:28,667 - lava_callback - INFO - Starting async job submit for node 69c8effb6a4c026e16348af2 2026-03-31 09:25:28,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:25:28,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8effb6a4c026e16348af2 HTTP/1.1" 200 None 2026-03-31 09:25:28,683 - lava_callback - DEBUG - Processing node 69c8effb6a4c026e16348af2: baseline-arm64-baylibre 2026-03-31 09:25:28,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8effb6a4c026e16348af2 2026-03-31 09:25:28,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:25:28,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8effb6a4c026e16348af2 HTTP/1.1" 200 None 2026-03-31 09:25:28,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:25:28,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:25:28,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:25:28,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8effb6a4c026e16348af2 HTTP/1.1" 200 None 2026-03-31 09:25:28,760 - lava_callback - ERROR - Error processing callback for node 69c8effb6a4c026e16348af2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58600 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:27:24,217 - lava_callback - INFO - Received callback for node 69cb93142a64fa9aff937644 INFO: 172.19.0.1:60880 - "POST /node/69cb93142a64fa9aff937644 HTTP/1.0" 202 Accepted 2026-03-31 09:27:24,224 - lava_callback - INFO - Starting async job submit for node 69cb93142a64fa9aff937644 2026-03-31 09:27:24,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:24,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb93142a64fa9aff937644 HTTP/1.1" 200 None 2026-03-31 09:27:24,269 - lava_callback - DEBUG - Processing node 69cb93142a64fa9aff937644: baseline-arm64 2026-03-31 09:27:24,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:27:24,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:27:24,639 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93142a64fa9aff937644/log.txt.gz 2026-03-31 09:27:24,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:27:24,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:27:24,954 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93142a64fa9aff937644/lava-logs.yaml 2026-03-31 09:27:24,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:27:25,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:27:25,121 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93142a64fa9aff937644/lava_callback.json.gz 2026-03-31 09:27:25,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:25,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb93142a64fa9aff937644 HTTP/1.1" 200 None 2026-03-31 09:27:25,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:25,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:27:25,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:25,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb93142a64fa9aff937644 HTTP/1.1" 200 None 2026-03-31 09:27:25,263 - lava_callback - INFO - Completed processing callback for node 69cb93142a64fa9aff937644 INFO: 172.19.0.1:60884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38588 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:27:38,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:38,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51376 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:27:56,171 - lava_callback - INFO - Received callback for node 69cb935f2a64fa9aff93764c INFO: 172.19.0.1:33230 - "POST /node/69cb935f2a64fa9aff93764c HTTP/1.0" 202 Accepted 2026-03-31 09:27:56,177 - lava_callback - INFO - Starting async job submit for node 69cb935f2a64fa9aff93764c 2026-03-31 09:27:56,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:56,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb935f2a64fa9aff93764c HTTP/1.1" 200 None 2026-03-31 09:27:56,222 - lava_callback - DEBUG - Processing node 69cb935f2a64fa9aff93764c: baseline-arm64 2026-03-31 09:27:56,352 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:27:56,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:33232 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:27:56,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:27:56,603 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb935f2a64fa9aff93764c/lava-logs.yaml 2026-03-31 09:27:56,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:27:56,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:27:56,723 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb935f2a64fa9aff93764c/lava_callback.json.gz 2026-03-31 09:27:56,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:56,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb935f2a64fa9aff93764c HTTP/1.1" 200 None 2026-03-31 09:27:56,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:56,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:27:56,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:27:56,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb935f2a64fa9aff93764c HTTP/1.1" 200 None 2026-03-31 09:27:56,897 - lava_callback - INFO - Completed processing callback for node 69cb935f2a64fa9aff93764c INFO: 172.19.0.1:35108 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:28:08,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:28:08,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 09:28:08,845 - lava_callback - INFO - Received callback for node 69c8f0976a4c026e16348b0c INFO: 172.19.0.1:35110 - "POST /node/69c8f0976a4c026e16348b0c HTTP/1.0" 202 Accepted 2026-03-31 09:28:08,851 - lava_callback - INFO - Starting async job submit for node 69c8f0976a4c026e16348b0c 2026-03-31 09:28:08,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:28:08,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f0976a4c026e16348b0c HTTP/1.1" 200 None 2026-03-31 09:28:08,869 - lava_callback - DEBUG - Processing node 69c8f0976a4c026e16348b0c: baseline-arm64-baylibre 2026-03-31 09:28:08,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f0976a4c026e16348b0c 2026-03-31 09:28:08,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:28:08,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f0976a4c026e16348b0c HTTP/1.1" 200 None 2026-03-31 09:28:08,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:28:08,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:28:08,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:28:08,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f0976a4c026e16348b0c HTTP/1.1" 200 None 2026-03-31 09:28:08,926 - lava_callback - ERROR - Error processing callback for node 69c8f0976a4c026e16348b0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58146 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 09:29:16,414 - lava_callback - INFO - Received callback for node 69cb93b22a64fa9aff93765f INFO: 172.19.0.1:58150 - "POST /node/69cb93b22a64fa9aff93765f HTTP/1.0" 202 Accepted 2026-03-31 09:29:16,420 - lava_callback - INFO - Starting async job submit for node 69cb93b22a64fa9aff93765f 2026-03-31 09:29:16,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:16,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb93b22a64fa9aff93765f HTTP/1.1" 200 None 2026-03-31 09:29:16,457 - lava_callback - DEBUG - Processing node 69cb93b22a64fa9aff93765f: baseline-arm64 INFO: 172.19.0.1:58158 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:29:16,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:29:17,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:29:17,084 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93b22a64fa9aff93765f/log.txt.gz 2026-03-31 09:29:17,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:29:17,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:29:17,530 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93b22a64fa9aff93765f/lava-logs.yaml 2026-03-31 09:29:17,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:29:17,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:29:17,698 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb93b22a64fa9aff93765f/lava_callback.json.gz 2026-03-31 09:29:17,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:17,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb93b22a64fa9aff93765f HTTP/1.1" 200 None 2026-03-31 09:29:17,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:17,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:29:17,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:17,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb93b22a64fa9aff93765f HTTP/1.1" 200 None 2026-03-31 09:29:17,798 - lava_callback - INFO - Completed processing callback for node 69cb93b22a64fa9aff93765f INFO: 172.19.0.1:58950 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:29:28,887 - lava_callback - INFO - Received callback for node 69c8f0e46a4c026e16348b1b INFO: 172.19.0.1:58962 - "POST /node/69c8f0e46a4c026e16348b1b HTTP/1.0" 202 Accepted 2026-03-31 09:29:28,893 - lava_callback - INFO - Starting async job submit for node 69c8f0e46a4c026e16348b1b 2026-03-31 09:29:28,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:28,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f0e46a4c026e16348b1b HTTP/1.1" 200 None 2026-03-31 09:29:28,910 - lava_callback - DEBUG - Processing node 69c8f0e46a4c026e16348b1b: baseline-arm64-baylibre 2026-03-31 09:29:28,910 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f0e46a4c026e16348b1b 2026-03-31 09:29:28,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:28,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f0e46a4c026e16348b1b HTTP/1.1" 200 None 2026-03-31 09:29:28,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:28,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:29:28,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:28,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f0e46a4c026e16348b1b HTTP/1.1" 200 None 2026-03-31 09:29:28,962 - lava_callback - ERROR - Error processing callback for node 69c8f0e46a4c026e16348b1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39592 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:29:38,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:29:38,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:41246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56498 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:30:28,918 - lava_callback - INFO - Received callback for node 69c8f1326a4c026e16348b21 INFO: 172.19.0.1:56514 - "POST /node/69c8f1326a4c026e16348b21 HTTP/1.0" 202 Accepted 2026-03-31 09:30:28,924 - lava_callback - INFO - Starting async job submit for node 69c8f1326a4c026e16348b21 2026-03-31 09:30:28,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:30:28,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f1326a4c026e16348b21 HTTP/1.1" 200 None 2026-03-31 09:30:28,940 - lava_callback - DEBUG - Processing node 69c8f1326a4c026e16348b21: baseline-arm64-baylibre 2026-03-31 09:30:28,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f1326a4c026e16348b21 2026-03-31 09:30:28,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:30:28,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f1326a4c026e16348b21 HTTP/1.1" 200 None 2026-03-31 09:30:28,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:30:28,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:30:28,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:30:29,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f1326a4c026e16348b21 HTTP/1.1" 200 None 2026-03-31 09:30:29,022 - lava_callback - ERROR - Error processing callback for node 69c8f1326a4c026e16348b21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41000 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:31:49,129 - lava_callback - INFO - Received callback for node 69cb94472a64fa9aff937684 INFO: 172.19.0.1:41006 - "POST /node/69cb94472a64fa9aff937684 HTTP/1.0" 202 Accepted 2026-03-31 09:31:49,135 - lava_callback - INFO - Starting async job submit for node 69cb94472a64fa9aff937684 2026-03-31 09:31:49,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:49,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94472a64fa9aff937684 HTTP/1.1" 200 None 2026-03-31 09:31:49,182 - lava_callback - DEBUG - Processing node 69cb94472a64fa9aff937684: baseline-arm64 2026-03-31 09:31:49,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:31:49,474 - lava_callback - INFO - Received callback for node 69c8f17e6a4c026e16348b2d INFO: 172.19.0.1:41016 - "POST /node/69c8f17e6a4c026e16348b2d HTTP/1.0" 202 Accepted 2026-03-31 09:31:49,480 - lava_callback - INFO - Starting async job submit for node 69c8f17e6a4c026e16348b2d 2026-03-31 09:31:49,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:49,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f17e6a4c026e16348b2d HTTP/1.1" 200 None 2026-03-31 09:31:49,497 - lava_callback - DEBUG - Processing node 69c8f17e6a4c026e16348b2d: baseline-arm64-baylibre 2026-03-31 09:31:49,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f17e6a4c026e16348b2d 2026-03-31 09:31:49,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:49,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f17e6a4c026e16348b2d HTTP/1.1" 200 None 2026-03-31 09:31:49,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:49,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:31:49,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:49,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:31:49,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb94472a64fa9aff937684/log.txt.gz 2026-03-31 09:31:49,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f17e6a4c026e16348b2d HTTP/1.1" 200 None 2026-03-31 09:31:49,582 - lava_callback - ERROR - Error processing callback for node 69c8f17e6a4c026e16348b2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 09:31:49,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:31:49,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:31:49,990 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb94472a64fa9aff937684/lava-logs.yaml 2026-03-31 09:31:49,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:31:50,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:31:50,162 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb94472a64fa9aff937684/lava_callback.json.gz 2026-03-31 09:31:50,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:50,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94472a64fa9aff937684 HTTP/1.1" 200 None 2026-03-31 09:31:50,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:50,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:31:50,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:31:50,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb94472a64fa9aff937684 HTTP/1.1" 200 None 2026-03-31 09:31:50,281 - lava_callback - INFO - Completed processing callback for node 69cb94472a64fa9aff937684 INFO: 172.19.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43774 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:32:08,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:32:08,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:36354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57512 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:33:09,087 - lava_callback - INFO - Received callback for node 69c8f1cc6a4c026e16348b40 INFO: 172.19.0.1:57528 - "POST /node/69c8f1cc6a4c026e16348b40 HTTP/1.0" 202 Accepted 2026-03-31 09:33:09,093 - lava_callback - INFO - Starting async job submit for node 69c8f1cc6a4c026e16348b40 2026-03-31 09:33:09,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:33:09,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f1cc6a4c026e16348b40 HTTP/1.1" 200 None 2026-03-31 09:33:09,110 - lava_callback - DEBUG - Processing node 69c8f1cc6a4c026e16348b40: baseline-arm64-baylibre 2026-03-31 09:33:09,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f1cc6a4c026e16348b40 2026-03-31 09:33:09,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:33:09,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f1cc6a4c026e16348b40 HTTP/1.1" 200 None 2026-03-31 09:33:09,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:33:09,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:33:09,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:33:09,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f1cc6a4c026e16348b40 HTTP/1.1" 200 None 2026-03-31 09:33:09,180 - lava_callback - ERROR - Error processing callback for node 69c8f1cc6a4c026e16348b40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50532 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:34:29,157 - lava_callback - INFO - Received callback for node 69c8f2176a4c026e16348b4f INFO: 172.19.0.1:50536 - "POST /node/69c8f2176a4c026e16348b4f HTTP/1.0" 202 Accepted 2026-03-31 09:34:29,163 - lava_callback - INFO - Starting async job submit for node 69c8f2176a4c026e16348b4f 2026-03-31 09:34:29,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:34:29,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2176a4c026e16348b4f HTTP/1.1" 200 None 2026-03-31 09:34:29,179 - lava_callback - DEBUG - Processing node 69c8f2176a4c026e16348b4f: baseline-arm64-baylibre 2026-03-31 09:34:29,179 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f2176a4c026e16348b4f 2026-03-31 09:34:29,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:34:29,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2176a4c026e16348b4f HTTP/1.1" 200 None 2026-03-31 09:34:29,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:34:29,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:34:29,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:34:29,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f2176a4c026e16348b4f HTTP/1.1" 200 None 2026-03-31 09:34:29,248 - lava_callback - ERROR - Error processing callback for node 69c8f2176a4c026e16348b4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35784 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:35:07,518 - lava_callback - INFO - Received callback for node 69cb912c2a64fa9aff9375da INFO: 172.19.0.1:35788 - "POST /node/69cb912c2a64fa9aff9375da HTTP/1.0" 202 Accepted 2026-03-31 09:35:07,525 - lava_callback - INFO - Starting async job submit for node 69cb912c2a64fa9aff9375da 2026-03-31 09:35:07,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:07,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb912c2a64fa9aff9375da HTTP/1.1" 200 None 2026-03-31 09:35:07,594 - lava_callback - DEBUG - Processing node 69cb912c2a64fa9aff9375da: baseline-arm 2026-03-31 09:35:08,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:35:08,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:35:08,443 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb912c2a64fa9aff9375da/log.txt.gz 2026-03-31 09:35:08,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:35:09,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:35:09,099 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb912c2a64fa9aff9375da/lava-logs.yaml 2026-03-31 09:35:09,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:35:09,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:35:09,294 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb912c2a64fa9aff9375da/lava_callback.json.gz DEBUG: Setup failed for 69cb912c2a64fa9aff9375da. Transitting job node result: incomplete -> fail 2026-03-31 09:35:09,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:09,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb912c2a64fa9aff9375da HTTP/1.1" 200 None 2026-03-31 09:35:09,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:09,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 09:35:09,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:09,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb912c2a64fa9aff9375da HTTP/1.1" 200 None 2026-03-31 09:35:09,566 - lava_callback - INFO - Completed processing callback for node 69cb912c2a64fa9aff9375da INFO: 172.19.0.1:46332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52306 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:35:38,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:38,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42742 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:35:49,253 - lava_callback - INFO - Received callback for node 69c8f2626a4c026e16348b5e INFO: 172.19.0.1:42754 - "POST /node/69c8f2626a4c026e16348b5e HTTP/1.0" 202 Accepted 2026-03-31 09:35:49,258 - lava_callback - INFO - Starting async job submit for node 69c8f2626a4c026e16348b5e 2026-03-31 09:35:49,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:49,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2626a4c026e16348b5e HTTP/1.1" 200 None 2026-03-31 09:35:49,274 - lava_callback - DEBUG - Processing node 69c8f2626a4c026e16348b5e: baseline-arm64-baylibre 2026-03-31 09:35:49,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f2626a4c026e16348b5e 2026-03-31 09:35:49,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:49,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2626a4c026e16348b5e HTTP/1.1" 200 None 2026-03-31 09:35:49,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:49,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:35:49,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:35:49,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f2626a4c026e16348b5e HTTP/1.1" 200 None 2026-03-31 09:35:49,341 - lava_callback - ERROR - Error processing callback for node 69c8f2626a4c026e16348b5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46516 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:36:49,237 - lava_callback - INFO - Received callback for node 69c8f2ad6a4c026e16348b6f INFO: 172.19.0.1:46524 - "POST /node/69c8f2ad6a4c026e16348b6f HTTP/1.0" 202 Accepted 2026-03-31 09:36:49,244 - lava_callback - INFO - Starting async job submit for node 69c8f2ad6a4c026e16348b6f 2026-03-31 09:36:49,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:36:49,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2ad6a4c026e16348b6f HTTP/1.1" 200 None 2026-03-31 09:36:49,261 - lava_callback - DEBUG - Processing node 69c8f2ad6a4c026e16348b6f: baseline-arm64-baylibre 2026-03-31 09:36:49,261 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f2ad6a4c026e16348b6f 2026-03-31 09:36:49,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:36:49,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2ad6a4c026e16348b6f HTTP/1.1" 200 None 2026-03-31 09:36:49,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:36:49,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:36:49,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:36:49,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f2ad6a4c026e16348b6f HTTP/1.1" 200 None 2026-03-31 09:36:49,325 - lava_callback - ERROR - Error processing callback for node 69c8f2ad6a4c026e16348b6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:32802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58290 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:38:28,950 - lava_callback - INFO - Received callback for node 69c8f2ff6a4c026e16348b7b INFO: 172.19.0.1:58292 - "POST /node/69c8f2ff6a4c026e16348b7b HTTP/1.0" 202 Accepted 2026-03-31 09:38:28,956 - lava_callback - INFO - Starting async job submit for node 69c8f2ff6a4c026e16348b7b 2026-03-31 09:38:28,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:38:28,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2ff6a4c026e16348b7b HTTP/1.1" 200 None 2026-03-31 09:38:28,973 - lava_callback - DEBUG - Processing node 69c8f2ff6a4c026e16348b7b: baseline-arm64-clabbe 2026-03-31 09:38:28,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f2ff6a4c026e16348b7b 2026-03-31 09:38:28,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:38:28,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f2ff6a4c026e16348b7b HTTP/1.1" 200 None 2026-03-31 09:38:28,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:38:29,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:38:29,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:38:29,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f2ff6a4c026e16348b7b HTTP/1.1" 200 None 2026-03-31 09:38:29,034 - lava_callback - ERROR - Error processing callback for node 69c8f2ff6a4c026e16348b7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54524 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:39:48,974 - lava_callback - INFO - Received callback for node 69c8f34e6a4c026e16348b80 INFO: 172.19.0.1:54528 - "POST /node/69c8f34e6a4c026e16348b80 HTTP/1.0" 202 Accepted 2026-03-31 09:39:48,979 - lava_callback - INFO - Starting async job submit for node 69c8f34e6a4c026e16348b80 2026-03-31 09:39:48,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:39:48,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f34e6a4c026e16348b80 HTTP/1.1" 200 None 2026-03-31 09:39:48,997 - lava_callback - DEBUG - Processing node 69c8f34e6a4c026e16348b80: baseline-arm64-clabbe 2026-03-31 09:39:48,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f34e6a4c026e16348b80 2026-03-31 09:39:48,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:39:49,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f34e6a4c026e16348b80 HTTP/1.1" 200 None 2026-03-31 09:39:49,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:39:49,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:39:49,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:39:49,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f34e6a4c026e16348b80 HTTP/1.1" 200 None 2026-03-31 09:39:49,060 - lava_callback - ERROR - Error processing callback for node 69c8f34e6a4c026e16348b80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39458 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:40:49,043 - lava_callback - INFO - Received callback for node 69c8f39d6a4c026e16348b8c INFO: 172.19.0.1:39472 - "POST /node/69c8f39d6a4c026e16348b8c HTTP/1.0" 202 Accepted 2026-03-31 09:40:49,048 - lava_callback - INFO - Starting async job submit for node 69c8f39d6a4c026e16348b8c 2026-03-31 09:40:49,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:40:49,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f39d6a4c026e16348b8c HTTP/1.1" 200 None 2026-03-31 09:40:49,069 - lava_callback - DEBUG - Processing node 69c8f39d6a4c026e16348b8c: baseline-arm64-clabbe 2026-03-31 09:40:49,070 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f39d6a4c026e16348b8c 2026-03-31 09:40:49,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:40:49,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f39d6a4c026e16348b8c HTTP/1.1" 200 None 2026-03-31 09:40:49,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:40:49,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:40:49,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:40:49,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f39d6a4c026e16348b8c HTTP/1.1" 200 None 2026-03-31 09:40:49,145 - lava_callback - ERROR - Error processing callback for node 69c8f39d6a4c026e16348b8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41944 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:41:12,541 - lava_callback - INFO - Received callback for node 69cb96a82a64fa9aff93770c INFO: 172.19.0.1:36062 - "POST /node/69cb96a82a64fa9aff93770c HTTP/1.0" 202 Accepted 2026-03-31 09:41:12,547 - lava_callback - INFO - Starting async job submit for node 69cb96a82a64fa9aff93770c 2026-03-31 09:41:12,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:41:12,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb96a82a64fa9aff93770c HTTP/1.1" 200 None 2026-03-31 09:41:12,567 - lava_callback - DEBUG - Processing node 69cb96a82a64fa9aff93770c: baseline-nfs-arm64 2026-03-31 09:41:12,568 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:41:12,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:41:12,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:41:12,698 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb96a82a64fa9aff93770c/lava-logs.yaml 2026-03-31 09:41:12,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:41:12,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:41:12,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb96a82a64fa9aff93770c/lava_callback.json.gz 2026-03-31 09:41:12,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:41:12,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb96a82a64fa9aff93770c HTTP/1.1" 200 None 2026-03-31 09:41:12,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:41:12,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:41:12,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:41:12,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb96a82a64fa9aff93770c HTTP/1.1" 200 None 2026-03-31 09:41:12,893 - lava_callback - INFO - Completed processing callback for node 69cb96a82a64fa9aff93770c INFO: 172.19.0.1:36074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51990 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:41:38,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:41:38,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:48412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33016 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:42:09,117 - lava_callback - INFO - Received callback for node 69c8f3ea6a4c026e16348b92 INFO: 172.19.0.1:33020 - "POST /node/69c8f3ea6a4c026e16348b92 HTTP/1.0" 202 Accepted 2026-03-31 09:42:09,122 - lava_callback - INFO - Starting async job submit for node 69c8f3ea6a4c026e16348b92 2026-03-31 09:42:09,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:09,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f3ea6a4c026e16348b92 HTTP/1.1" 200 None 2026-03-31 09:42:09,143 - lava_callback - DEBUG - Processing node 69c8f3ea6a4c026e16348b92: baseline-arm64-clabbe 2026-03-31 09:42:09,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f3ea6a4c026e16348b92 2026-03-31 09:42:09,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:09,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f3ea6a4c026e16348b92 HTTP/1.1" 200 None 2026-03-31 09:42:09,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:09,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:42:09,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:09,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f3ea6a4c026e16348b92 HTTP/1.1" 200 None 2026-03-31 09:42:09,206 - lava_callback - ERROR - Error processing callback for node 69c8f3ea6a4c026e16348b92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32868 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:42:46,307 - lava_callback - INFO - Received callback for node 69cb954f2a64fa9aff9376d4 INFO: 172.19.0.1:60078 - "POST /node/69cb954f2a64fa9aff9376d4 HTTP/1.0" 202 Accepted 2026-03-31 09:42:46,315 - lava_callback - INFO - Starting async job submit for node 69cb954f2a64fa9aff9376d4 2026-03-31 09:42:46,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:46,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb954f2a64fa9aff9376d4 HTTP/1.1" 200 None 2026-03-31 09:42:46,403 - lava_callback - DEBUG - Processing node 69cb954f2a64fa9aff9376d4: baseline-arm64 INFO: 172.19.0.1:60094 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:42:46,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:42:46,936 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:42:46,938 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb954f2a64fa9aff9376d4/log.txt.gz 2026-03-31 09:42:47,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:42:47,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:42:47,370 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb954f2a64fa9aff9376d4/lava-logs.yaml 2026-03-31 09:42:47,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:42:47,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:42:47,524 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb954f2a64fa9aff9376d4/lava_callback.json.gz DEBUG: Setup failed for 69cb954f2a64fa9aff9376d4. Transitting job node result: incomplete -> fail 2026-03-31 09:42:47,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:47,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb954f2a64fa9aff9376d4 HTTP/1.1" 200 None 2026-03-31 09:42:47,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:47,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:42:47,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:42:47,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb954f2a64fa9aff9376d4 HTTP/1.1" 200 None 2026-03-31 09:42:47,736 - lava_callback - INFO - Completed processing callback for node 69cb954f2a64fa9aff9376d4 INFO: 172.19.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42894 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:43:08,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:43:08,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:48724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32972 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:43:29,144 - lava_callback - INFO - Received callback for node 69c8f4386a4c026e16348b9d INFO: 172.19.0.1:32988 - "POST /node/69c8f4386a4c026e16348b9d HTTP/1.0" 202 Accepted 2026-03-31 09:43:29,152 - lava_callback - INFO - Starting async job submit for node 69c8f4386a4c026e16348b9d 2026-03-31 09:43:29,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:43:29,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f4386a4c026e16348b9d HTTP/1.1" 200 None 2026-03-31 09:43:29,168 - lava_callback - DEBUG - Processing node 69c8f4386a4c026e16348b9d: baseline-arm64-clabbe 2026-03-31 09:43:29,169 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f4386a4c026e16348b9d 2026-03-31 09:43:29,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:43:29,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f4386a4c026e16348b9d HTTP/1.1" 200 None 2026-03-31 09:43:29,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:43:29,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:43:29,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:43:29,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f4386a4c026e16348b9d HTTP/1.1" 200 None 2026-03-31 09:43:29,243 - lava_callback - ERROR - Error processing callback for node 69c8f4386a4c026e16348b9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35868 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:44:49,237 - lava_callback - INFO - Received callback for node 69c8f4856a4c026e16348ba9 INFO: 172.19.0.1:35880 - "POST /node/69c8f4856a4c026e16348ba9 HTTP/1.0" 202 Accepted 2026-03-31 09:44:49,244 - lava_callback - INFO - Starting async job submit for node 69c8f4856a4c026e16348ba9 2026-03-31 09:44:49,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:44:49,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f4856a4c026e16348ba9 HTTP/1.1" 200 None 2026-03-31 09:44:49,266 - lava_callback - DEBUG - Processing node 69c8f4856a4c026e16348ba9: baseline-arm64-clabbe 2026-03-31 09:44:49,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f4856a4c026e16348ba9 2026-03-31 09:44:49,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:44:49,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f4856a4c026e16348ba9 HTTP/1.1" 200 None 2026-03-31 09:44:49,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:44:49,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 09:44:49,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:44:49,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f4856a4c026e16348ba9 HTTP/1.1" 200 None 2026-03-31 09:44:49,326 - lava_callback - ERROR - Error processing callback for node 69c8f4856a4c026e16348ba9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54006 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:48:29,991 - lava_callback - INFO - Received callback for node 69c8f5666a4c026e16348bc2 INFO: 172.19.0.1:46096 - "POST /node/69c8f5666a4c026e16348bc2 HTTP/1.0" 202 Accepted 2026-03-31 09:48:29,996 - lava_callback - INFO - Starting async job submit for node 69c8f5666a4c026e16348bc2 2026-03-31 09:48:29,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:48:30,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f5666a4c026e16348bc2 HTTP/1.1" 200 None 2026-03-31 09:48:30,012 - lava_callback - DEBUG - Processing node 69c8f5666a4c026e16348bc2: baseline-arm64-baylibre 2026-03-31 09:48:30,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f5666a4c026e16348bc2 2026-03-31 09:48:30,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:48:30,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f5666a4c026e16348bc2 HTTP/1.1" 200 None 2026-03-31 09:48:30,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:48:30,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:48:30,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:48:30,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f5666a4c026e16348bc2 HTTP/1.1" 200 None 2026-03-31 09:48:30,074 - lava_callback - ERROR - Error processing callback for node 69c8f5666a4c026e16348bc2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36014 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:49:12,958 - lava_callback - INFO - Received callback for node 69cb98662a64fa9aff93774c INFO: 172.19.0.1:47546 - "POST /node/69cb98662a64fa9aff93774c HTTP/1.0" 202 Accepted 2026-03-31 09:49:12,965 - lava_callback - INFO - Starting async job submit for node 69cb98662a64fa9aff93774c 2026-03-31 09:49:12,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:13,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb98662a64fa9aff93774c HTTP/1.1" 200 None 2026-03-31 09:49:13,009 - lava_callback - DEBUG - Processing node 69cb98662a64fa9aff93774c: baseline-arm64 2026-03-31 09:49:13,140 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:49:13,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:49:13,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:49:13,579 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb98662a64fa9aff93774c/lava-logs.yaml 2026-03-31 09:49:13,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:49:13,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:49:13,709 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb98662a64fa9aff93774c/lava_callback.json.gz 2026-03-31 09:49:13,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:13,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb98662a64fa9aff93774c HTTP/1.1" 200 None 2026-03-31 09:49:13,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:13,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:49:13,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:13,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb98662a64fa9aff93774c HTTP/1.1" 200 None 2026-03-31 09:49:13,861 - lava_callback - INFO - Completed processing callback for node 69cb98662a64fa9aff93774c INFO: 172.19.0.1:47562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47566 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:49:19,611 - lava_callback - INFO - Received callback for node 69cb94072a64fa9aff937670 INFO: 172.19.0.1:47572 - "POST /node/69cb94072a64fa9aff937670 HTTP/1.0" 202 Accepted 2026-03-31 09:49:19,657 - lava_callback - INFO - Starting async job submit for node 69cb94072a64fa9aff937670 2026-03-31 09:49:19,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:19,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94072a64fa9aff937670 HTTP/1.1" 200 None 2026-03-31 09:49:19,696 - lava_callback - DEBUG - Processing node 69cb94072a64fa9aff937670: baseline-arm-pengutronix 2026-03-31 09:49:19,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:49:19,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:49:19,930 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb94072a64fa9aff937670/log.txt.gz 2026-03-31 09:49:19,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:49:20,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:49:20,150 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb94072a64fa9aff937670/lava-logs.yaml 2026-03-31 09:49:20,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:49:20,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:49:20,275 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb94072a64fa9aff937670/lava_callback.json.gz DEBUG: Setup failed for 69cb94072a64fa9aff937670. Transitting job node result: incomplete -> fail 2026-03-31 09:49:20,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:20,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94072a64fa9aff937670 HTTP/1.1" 200 None 2026-03-31 09:49:20,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:20,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 09:49:20,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:20,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb94072a64fa9aff937670 HTTP/1.1" 200 None 2026-03-31 09:49:20,449 - lava_callback - INFO - Completed processing callback for node 69cb94072a64fa9aff937670 INFO: 172.19.0.1:35456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47872 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:49:38,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:38,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:47936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47952 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:49:50,002 - lava_callback - INFO - Received callback for node 69c8f5b26a4c026e16348bce INFO: 172.19.0.1:41364 - "POST /node/69c8f5b26a4c026e16348bce HTTP/1.0" 202 Accepted 2026-03-31 09:49:50,008 - lava_callback - INFO - Starting async job submit for node 69c8f5b26a4c026e16348bce 2026-03-31 09:49:50,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:50,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f5b26a4c026e16348bce HTTP/1.1" 200 None 2026-03-31 09:49:50,024 - lava_callback - DEBUG - Processing node 69c8f5b26a4c026e16348bce: baseline-arm64-baylibre 2026-03-31 09:49:50,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f5b26a4c026e16348bce 2026-03-31 09:49:50,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:50,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f5b26a4c026e16348bce HTTP/1.1" 200 None 2026-03-31 09:49:50,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:50,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:49:50,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:49:50,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f5b26a4c026e16348bce HTTP/1.1" 200 None 2026-03-31 09:49:50,079 - lava_callback - ERROR - Error processing callback for node 69c8f5b26a4c026e16348bce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48320 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:50:11,950 - lava_callback - INFO - Received callback for node 69cb96562a64fa9aff9376fe INFO: 172.19.0.1:59748 - "POST /node/69cb96562a64fa9aff9376fe HTTP/1.0" 202 Accepted 2026-03-31 09:50:11,956 - lava_callback - INFO - Starting async job submit for node 69cb96562a64fa9aff9376fe 2026-03-31 09:50:11,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:12,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb96562a64fa9aff9376fe HTTP/1.1" 200 None 2026-03-31 09:50:12,013 - lava_callback - DEBUG - Processing node 69cb96562a64fa9aff9376fe: baseline-arm64 2026-03-31 09:50:12,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:50:12,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:50:12,634 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb96562a64fa9aff9376fe/log.txt.gz 2026-03-31 09:50:12,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:50:13,094 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:50:13,096 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb96562a64fa9aff9376fe/lava-logs.yaml 2026-03-31 09:50:13,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:50:13,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:50:13,264 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb96562a64fa9aff9376fe/lava_callback.json.gz 2026-03-31 09:50:13,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:13,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb96562a64fa9aff9376fe HTTP/1.1" 200 None 2026-03-31 09:50:13,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:13,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:50:13,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:13,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb96562a64fa9aff9376fe HTTP/1.1" 200 None 2026-03-31 09:50:13,401 - lava_callback - INFO - Completed processing callback for node 69cb96562a64fa9aff9376fe INFO: 172.19.0.1:59758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57778 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:50:37,253 - lava_callback - INFO - Received callback for node 69cb98b02a64fa9aff93775b INFO: 172.19.0.1:57788 - "POST /node/69cb98b02a64fa9aff93775b HTTP/1.0" 202 Accepted 2026-03-31 09:50:37,260 - lava_callback - INFO - Starting async job submit for node 69cb98b02a64fa9aff93775b 2026-03-31 09:50:37,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:37,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb98b02a64fa9aff93775b HTTP/1.1" 200 None 2026-03-31 09:50:37,305 - lava_callback - DEBUG - Processing node 69cb98b02a64fa9aff93775b: baseline-arm64 2026-03-31 09:50:37,452 - lava_callback - WARNING - Failed to upload log 2026-03-31 09:50:37,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:50:37,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:50:37,702 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb98b02a64fa9aff93775b/lava-logs.yaml 2026-03-31 09:50:37,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:50:37,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:50:37,839 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb98b02a64fa9aff93775b/lava_callback.json.gz 2026-03-31 09:50:37,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:37,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb98b02a64fa9aff93775b HTTP/1.1" 200 None 2026-03-31 09:50:37,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:37,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:50:37,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:37,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb98b02a64fa9aff93775b HTTP/1.1" 200 None 2026-03-31 09:50:37,992 - lava_callback - INFO - Completed processing callback for node 69cb98b02a64fa9aff93775b 2026-03-31 09:50:38,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:50:38,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:37562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43976 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:51:42,888 - lava_callback - INFO - Received callback for node 69cb94b32a64fa9aff9376a2 INFO: 172.19.0.1:33346 - "POST /node/69cb94b32a64fa9aff9376a2 HTTP/1.0" 202 Accepted 2026-03-31 09:51:42,895 - lava_callback - INFO - Starting async job submit for node 69cb94b32a64fa9aff9376a2 2026-03-31 09:51:42,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:51:42,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94b32a64fa9aff9376a2 HTTP/1.1" 200 None 2026-03-31 09:51:42,968 - lava_callback - DEBUG - Processing node 69cb94b32a64fa9aff9376a2: ltp-smoke 2026-03-31 09:51:43,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:51:43,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:51:43,604 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb94b32a64fa9aff9376a2/log.txt.gz 2026-03-31 09:51:43,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:51:44,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:51:44,147 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb94b32a64fa9aff9376a2/lava-logs.yaml 2026-03-31 09:51:44,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:51:44,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:51:44,334 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cb94b32a64fa9aff9376a2/lava_callback.json.gz 2026-03-31 09:51:44,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:51:44,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb94b32a64fa9aff9376a2 HTTP/1.1" 200 None 2026-03-31 09:51:44,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:51:44,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83462a64fa9aff936905 HTTP/1.1" 200 None 2026-03-31 09:51:44,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:51:44,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb94b32a64fa9aff9376a2 HTTP/1.1" 200 None 2026-03-31 09:51:44,696 - lava_callback - INFO - Completed processing callback for node 69cb94b32a64fa9aff9376a2 INFO: 172.19.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53912 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:52:08,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:52:08,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:38016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55434 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:52:29,566 - lava_callback - INFO - Received callback for node 69c8f6526a4c026e16348be7 INFO: 172.19.0.1:55446 - "POST /node/69c8f6526a4c026e16348be7 HTTP/1.0" 202 Accepted 2026-03-31 09:52:29,576 - lava_callback - INFO - Starting async job submit for node 69c8f6526a4c026e16348be7 2026-03-31 09:52:29,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:52:29,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f6526a4c026e16348be7 HTTP/1.1" 200 None 2026-03-31 09:52:29,603 - lava_callback - DEBUG - Processing node 69c8f6526a4c026e16348be7: baseline-arm64-clabbe 2026-03-31 09:52:29,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f6526a4c026e16348be7 2026-03-31 09:52:29,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:52:29,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f6526a4c026e16348be7 HTTP/1.1" 200 None 2026-03-31 09:52:29,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:52:29,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:52:29,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:52:29,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f6526a4c026e16348be7 HTTP/1.1" 200 None 2026-03-31 09:52:29,695 - lava_callback - ERROR - Error processing callback for node 69c8f6526a4c026e16348be7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47610 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:53:49,555 - lava_callback - INFO - Received callback for node 69c8f69f6a4c026e16348bf0 INFO: 172.19.0.1:47622 - "POST /node/69c8f69f6a4c026e16348bf0 HTTP/1.0" 202 Accepted 2026-03-31 09:53:49,561 - lava_callback - INFO - Starting async job submit for node 69c8f69f6a4c026e16348bf0 2026-03-31 09:53:49,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:53:49,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f69f6a4c026e16348bf0 HTTP/1.1" 200 None 2026-03-31 09:53:49,584 - lava_callback - DEBUG - Processing node 69c8f69f6a4c026e16348bf0: baseline-arm64-clabbe 2026-03-31 09:53:49,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f69f6a4c026e16348bf0 2026-03-31 09:53:49,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:53:49,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f69f6a4c026e16348bf0 HTTP/1.1" 200 None 2026-03-31 09:53:49,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:53:49,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:53:49,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:53:49,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f69f6a4c026e16348bf0 HTTP/1.1" 200 None 2026-03-31 09:53:49,647 - lava_callback - ERROR - Error processing callback for node 69c8f69f6a4c026e16348bf0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40750 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:55:09,653 - lava_callback - INFO - Received callback for node 69c8f6ec6a4c026e16348bf6 INFO: 172.19.0.1:40756 - "POST /node/69c8f6ec6a4c026e16348bf6 HTTP/1.0" 202 Accepted 2026-03-31 09:55:09,658 - lava_callback - INFO - Starting async job submit for node 69c8f6ec6a4c026e16348bf6 2026-03-31 09:55:09,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:55:09,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f6ec6a4c026e16348bf6 HTTP/1.1" 200 None 2026-03-31 09:55:09,675 - lava_callback - DEBUG - Processing node 69c8f6ec6a4c026e16348bf6: baseline-arm64-clabbe 2026-03-31 09:55:09,675 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f6ec6a4c026e16348bf6 2026-03-31 09:55:09,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:55:09,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f6ec6a4c026e16348bf6 HTTP/1.1" 200 None 2026-03-31 09:55:09,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:55:09,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:55:09,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:55:09,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f6ec6a4c026e16348bf6 HTTP/1.1" 200 None 2026-03-31 09:55:09,738 - lava_callback - ERROR - Error processing callback for node 69c8f6ec6a4c026e16348bf6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43046 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:56:29,710 - lava_callback - INFO - Received callback for node 69c8f73a6a4c026e16348c02 INFO: 172.19.0.1:58058 - "POST /node/69c8f73a6a4c026e16348c02 HTTP/1.0" 202 Accepted 2026-03-31 09:56:29,715 - lava_callback - INFO - Starting async job submit for node 69c8f73a6a4c026e16348c02 2026-03-31 09:56:29,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:56:29,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f73a6a4c026e16348c02 HTTP/1.1" 200 None 2026-03-31 09:56:29,731 - lava_callback - DEBUG - Processing node 69c8f73a6a4c026e16348c02: baseline-arm64-clabbe 2026-03-31 09:56:29,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f73a6a4c026e16348c02 2026-03-31 09:56:29,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:56:29,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f73a6a4c026e16348c02 HTTP/1.1" 200 None 2026-03-31 09:56:29,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:56:29,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:56:29,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:56:29,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f73a6a4c026e16348c02 HTTP/1.1" 200 None 2026-03-31 09:56:29,790 - lava_callback - ERROR - Error processing callback for node 69c8f73a6a4c026e16348c02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57096 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:57:18,626 - lava_callback - INFO - Received callback for node 69cb95c32a64fa9aff9376ec INFO: 172.19.0.1:57108 - "POST /node/69cb95c32a64fa9aff9376ec HTTP/1.0" 202 Accepted 2026-03-31 09:57:18,635 - lava_callback - INFO - Starting async job submit for node 69cb95c32a64fa9aff9376ec 2026-03-31 09:57:18,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:57:18,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb95c32a64fa9aff9376ec HTTP/1.1" 200 None 2026-03-31 09:57:18,698 - lava_callback - DEBUG - Processing node 69cb95c32a64fa9aff9376ec: baseline-nfs-arm64 2026-03-31 09:57:19,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:57:19,914 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:57:19,916 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb95c32a64fa9aff9376ec/log.txt.gz 2026-03-31 09:57:20,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:57:20,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:57:20,808 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb95c32a64fa9aff9376ec/lava-logs.yaml 2026-03-31 09:57:20,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:57:21,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:57:21,037 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb95c32a64fa9aff9376ec/lava_callback.json.gz DEBUG: Setup failed for 69cb95c32a64fa9aff9376ec. Transitting job node result: incomplete -> fail 2026-03-31 09:57:21,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:57:21,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb95c32a64fa9aff9376ec HTTP/1.1" 200 None 2026-03-31 09:57:21,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:57:21,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:57:21,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:57:21,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb95c32a64fa9aff9376ec HTTP/1.1" 200 None 2026-03-31 09:57:21,332 - lava_callback - INFO - Completed processing callback for node 69cb95c32a64fa9aff9376ec INFO: 172.19.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56846 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:57:38,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:57:38,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52856 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:58:49,827 - lava_callback - INFO - Received callback for node 69c8f7d56a4c026e16348c1a INFO: 172.19.0.1:34128 - "POST /node/69c8f7d56a4c026e16348c1a HTTP/1.0" 202 Accepted 2026-03-31 09:58:49,832 - lava_callback - INFO - Starting async job submit for node 69c8f7d56a4c026e16348c1a 2026-03-31 09:58:49,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:58:49,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f7d56a4c026e16348c1a HTTP/1.1" 200 None 2026-03-31 09:58:49,849 - lava_callback - DEBUG - Processing node 69c8f7d56a4c026e16348c1a: baseline-arm64-clabbe 2026-03-31 09:58:49,849 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f7d56a4c026e16348c1a 2026-03-31 09:58:49,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:58:49,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f7d56a4c026e16348c1a HTTP/1.1" 200 None 2026-03-31 09:58:49,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:58:49,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 09:58:49,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:58:49,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f7d56a4c026e16348c1a HTTP/1.1" 200 None 2026-03-31 09:58:49,901 - lava_callback - ERROR - Error processing callback for node 69c8f7d56a4c026e16348c1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56582 - "GET / HTTP/1.0" 200 OK 2026-03-31 09:59:43,495 - lava_callback - INFO - Received callback for node 69cb97d72a64fa9aff93773c INFO: 172.19.0.1:43570 - "POST /node/69cb97d72a64fa9aff93773c HTTP/1.0" 202 Accepted 2026-03-31 09:59:43,501 - lava_callback - INFO - Starting async job submit for node 69cb97d72a64fa9aff93773c 2026-03-31 09:59:43,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:59:43,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb97d72a64fa9aff93773c HTTP/1.1" 200 None 2026-03-31 09:59:43,546 - lava_callback - DEBUG - Processing node 69cb97d72a64fa9aff93773c: baseline-nfs-arm64 2026-03-31 09:59:43,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:59:44,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:59:44,170 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb97d72a64fa9aff93773c/log.txt.gz 2026-03-31 09:59:44,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:59:44,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:59:44,717 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb97d72a64fa9aff93773c/lava-logs.yaml 2026-03-31 09:59:44,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 09:59:44,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 09:59:44,863 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb97d72a64fa9aff93773c/lava_callback.json.gz 2026-03-31 09:59:44,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:59:44,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb97d72a64fa9aff93773c HTTP/1.1" 200 None 2026-03-31 09:59:44,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:59:44,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 09:59:44,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 09:59:44,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb97d72a64fa9aff93773c HTTP/1.1" 200 None 2026-03-31 09:59:44,998 - lava_callback - INFO - Completed processing callback for node 69cb97d72a64fa9aff93773c INFO: 172.19.0.1:43574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40386 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:00:08,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:00:08,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:45896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48948 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:01:29,928 - lava_callback - INFO - Received callback for node 69c8f86e6a4c026e16348c2b INFO: 172.19.0.1:37278 - "POST /node/69c8f86e6a4c026e16348c2b HTTP/1.0" 202 Accepted 2026-03-31 10:01:29,934 - lava_callback - INFO - Starting async job submit for node 69c8f86e6a4c026e16348c2b 2026-03-31 10:01:29,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:01:29,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f86e6a4c026e16348c2b HTTP/1.1" 200 None 2026-03-31 10:01:29,952 - lava_callback - DEBUG - Processing node 69c8f86e6a4c026e16348c2b: baseline-arm64-clabbe 2026-03-31 10:01:29,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f86e6a4c026e16348c2b 2026-03-31 10:01:29,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:01:29,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f86e6a4c026e16348c2b HTTP/1.1" 200 None 2026-03-31 10:01:29,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:01:29,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 10:01:29,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:01:30,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f86e6a4c026e16348c2b HTTP/1.1" 200 None 2026-03-31 10:01:30,003 - lava_callback - ERROR - Error processing callback for node 69c8f86e6a4c026e16348c2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33402 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:07:12,376 - lava_callback - INFO - Received callback for node 69cb9cc62a64fa9aff9377fd INFO: 172.19.0.1:46768 - "POST /node/69cb9cc62a64fa9aff9377fd HTTP/1.0" 202 Accepted 2026-03-31 10:07:12,381 - lava_callback - INFO - Starting async job submit for node 69cb9cc62a64fa9aff9377fd 2026-03-31 10:07:12,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:12,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9cc62a64fa9aff9377fd HTTP/1.1" 200 None 2026-03-31 10:07:12,400 - lava_callback - DEBUG - Processing node 69cb9cc62a64fa9aff9377fd: baseline-nfs-arm64 2026-03-31 10:07:12,401 - lava_callback - WARNING - Failed to upload log 2026-03-31 10:07:12,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:07:12,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:07:12,555 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb9cc62a64fa9aff9377fd/lava-logs.yaml 2026-03-31 10:07:12,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:07:12,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:07:12,702 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cb9cc62a64fa9aff9377fd/lava_callback.json.gz 2026-03-31 10:07:12,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:12,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9cc62a64fa9aff9377fd HTTP/1.1" 200 None 2026-03-31 10:07:12,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:12,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:07:12,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:12,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb9cc62a64fa9aff9377fd HTTP/1.1" 200 None 2026-03-31 10:07:12,772 - lava_callback - INFO - Completed processing callback for node 69cb9cc62a64fa9aff9377fd INFO: 172.19.0.1:46770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37686 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:07:31,092 - lava_callback - INFO - Received callback for node 69c8f9de6a4c026e16348c5d INFO: 172.19.0.1:51652 - "POST /node/69c8f9de6a4c026e16348c5d HTTP/1.0" 202 Accepted 2026-03-31 10:07:31,098 - lava_callback - INFO - Starting async job submit for node 69c8f9de6a4c026e16348c5d 2026-03-31 10:07:31,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:31,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f9de6a4c026e16348c5d HTTP/1.1" 200 None 2026-03-31 10:07:31,118 - lava_callback - DEBUG - Processing node 69c8f9de6a4c026e16348c5d: baseline-arm-baylibre 2026-03-31 10:07:31,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8f9de6a4c026e16348c5d 2026-03-31 10:07:31,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:31,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8f9de6a4c026e16348c5d HTTP/1.1" 200 None 2026-03-31 10:07:31,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:31,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:07:31,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:31,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8f9de6a4c026e16348c5d HTTP/1.1" 200 None 2026-03-31 10:07:31,178 - lava_callback - ERROR - Error processing callback for node 69c8f9de6a4c026e16348c5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51662 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:07:38,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:07:38,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:46470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35610 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:08:10,980 - lava_callback - INFO - Received callback for node 69c8fa036a4c026e16348c69 INFO: 172.19.0.1:60280 - "POST /node/69c8fa036a4c026e16348c69 HTTP/1.0" 202 Accepted 2026-03-31 10:08:10,986 - lava_callback - INFO - Starting async job submit for node 69c8fa036a4c026e16348c69 2026-03-31 10:08:10,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:08:11,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa036a4c026e16348c69 HTTP/1.1" 200 None 2026-03-31 10:08:11,007 - lava_callback - DEBUG - Processing node 69c8fa036a4c026e16348c69: baseline-arm-baylibre 2026-03-31 10:08:11,007 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fa036a4c026e16348c69 2026-03-31 10:08:11,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:08:11,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa036a4c026e16348c69 HTTP/1.1" 200 None 2026-03-31 10:08:11,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:08:11,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:08:11,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:08:11,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fa036a4c026e16348c69 HTTP/1.1" 200 None 2026-03-31 10:08:11,060 - lava_callback - ERROR - Error processing callback for node 69c8fa036a4c026e16348c69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54848 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:10:11,155 - lava_callback - INFO - Received callback for node 69c8fa756a4c026e16348c78 INFO: 172.19.0.1:48692 - "POST /node/69c8fa756a4c026e16348c78 HTTP/1.0" 202 Accepted 2026-03-31 10:10:11,163 - lava_callback - INFO - Starting async job submit for node 69c8fa756a4c026e16348c78 2026-03-31 10:10:11,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:11,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa756a4c026e16348c78 HTTP/1.1" 200 None 2026-03-31 10:10:11,199 - lava_callback - DEBUG - Processing node 69c8fa756a4c026e16348c78: baseline-arm-baylibre 2026-03-31 10:10:11,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fa756a4c026e16348c78 2026-03-31 10:10:11,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:11,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa756a4c026e16348c78 HTTP/1.1" 200 None 2026-03-31 10:10:11,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:11,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:10:11,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:11,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fa756a4c026e16348c78 HTTP/1.1" 200 None 2026-03-31 10:10:11,274 - lava_callback - ERROR - Error processing callback for node 69c8fa756a4c026e16348c78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50416 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:10:51,169 - lava_callback - INFO - Received callback for node 69c8fa996a4c026e16348c85 INFO: 172.19.0.1:52778 - "POST /node/69c8fa996a4c026e16348c85 HTTP/1.0" 202 Accepted 2026-03-31 10:10:51,174 - lava_callback - INFO - Starting async job submit for node 69c8fa996a4c026e16348c85 2026-03-31 10:10:51,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:51,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa996a4c026e16348c85 HTTP/1.1" 200 None 2026-03-31 10:10:51,190 - lava_callback - DEBUG - Processing node 69c8fa996a4c026e16348c85: baseline-arm-baylibre 2026-03-31 10:10:51,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fa996a4c026e16348c85 2026-03-31 10:10:51,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:51,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fa996a4c026e16348c85 HTTP/1.1" 200 None 2026-03-31 10:10:51,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:51,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:10:51,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:10:51,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fa996a4c026e16348c85 HTTP/1.1" 200 None 2026-03-31 10:10:51,242 - lava_callback - ERROR - Error processing callback for node 69c8fa996a4c026e16348c85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53580 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:11:31,213 - lava_callback - INFO - Received callback for node 69c8fac46a4c026e16348c8c INFO: 172.19.0.1:45080 - "POST /node/69c8fac46a4c026e16348c8c HTTP/1.0" 202 Accepted 2026-03-31 10:11:31,219 - lava_callback - INFO - Starting async job submit for node 69c8fac46a4c026e16348c8c 2026-03-31 10:11:31,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:11:31,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fac46a4c026e16348c8c HTTP/1.1" 200 None 2026-03-31 10:11:31,235 - lava_callback - DEBUG - Processing node 69c8fac46a4c026e16348c8c: baseline-arm-baylibre 2026-03-31 10:11:31,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fac46a4c026e16348c8c 2026-03-31 10:11:31,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:11:31,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fac46a4c026e16348c8c HTTP/1.1" 200 None 2026-03-31 10:11:31,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:11:31,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:11:31,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:11:31,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fac46a4c026e16348c8c HTTP/1.1" 200 None 2026-03-31 10:11:31,300 - lava_callback - ERROR - Error processing callback for node 69c8fac46a4c026e16348c8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34392 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:12:11,220 - lava_callback - INFO - Received callback for node 69c8fae86a4c026e16348c95 INFO: 172.19.0.1:37888 - "POST /node/69c8fae86a4c026e16348c95 HTTP/1.0" 202 Accepted 2026-03-31 10:12:11,229 - lava_callback - INFO - Starting async job submit for node 69c8fae86a4c026e16348c95 2026-03-31 10:12:11,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:12:11,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fae86a4c026e16348c95 HTTP/1.1" 200 None 2026-03-31 10:12:11,244 - lava_callback - DEBUG - Processing node 69c8fae86a4c026e16348c95: baseline-arm-baylibre 2026-03-31 10:12:11,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fae86a4c026e16348c95 2026-03-31 10:12:11,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:12:11,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fae86a4c026e16348c95 HTTP/1.1" 200 None 2026-03-31 10:12:11,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:12:11,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:12:11,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:12:11,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fae86a4c026e16348c95 HTTP/1.1" 200 None 2026-03-31 10:12:11,308 - lava_callback - ERROR - Error processing callback for node 69c8fae86a4c026e16348c95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54996 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:13:30,519 - lava_callback - INFO - Received callback for node 69c8fb346a4c026e16348caa INFO: 172.19.0.1:54920 - "POST /node/69c8fb346a4c026e16348caa HTTP/1.0" 202 Accepted 2026-03-31 10:13:30,526 - lava_callback - INFO - Starting async job submit for node 69c8fb346a4c026e16348caa 2026-03-31 10:13:30,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:30,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb346a4c026e16348caa HTTP/1.1" 200 None 2026-03-31 10:13:30,549 - lava_callback - DEBUG - Processing node 69c8fb346a4c026e16348caa: baseline-arm-clabbe 2026-03-31 10:13:30,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fb346a4c026e16348caa 2026-03-31 10:13:30,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:30,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb346a4c026e16348caa HTTP/1.1" 200 None 2026-03-31 10:13:30,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:30,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:13:30,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:30,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fb346a4c026e16348caa HTTP/1.1" 200 None 2026-03-31 10:13:30,616 - lava_callback - ERROR - Error processing callback for node 69c8fb346a4c026e16348caa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39244 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:13:50,434 - lava_callback - INFO - Received callback for node 69c8fb5a6a4c026e16348cb1 INFO: 172.19.0.1:57644 - "POST /node/69c8fb5a6a4c026e16348cb1 HTTP/1.0" 202 Accepted 2026-03-31 10:13:50,440 - lava_callback - INFO - Starting async job submit for node 69c8fb5a6a4c026e16348cb1 2026-03-31 10:13:50,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:50,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb5a6a4c026e16348cb1 HTTP/1.1" 200 None 2026-03-31 10:13:50,462 - lava_callback - DEBUG - Processing node 69c8fb5a6a4c026e16348cb1: baseline-arm-clabbe 2026-03-31 10:13:50,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fb5a6a4c026e16348cb1 2026-03-31 10:13:50,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:50,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb5a6a4c026e16348cb1 HTTP/1.1" 200 None 2026-03-31 10:13:50,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:50,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:13:50,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:13:50,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fb5a6a4c026e16348cb1 HTTP/1.1" 200 None 2026-03-31 10:13:50,545 - lava_callback - ERROR - Error processing callback for node 69c8fb5a6a4c026e16348cb1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55538 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:14:30,504 - lava_callback - INFO - Received callback for node 69c8fb7f6a4c026e16348cc0 INFO: 172.19.0.1:53426 - "POST /node/69c8fb7f6a4c026e16348cc0 HTTP/1.0" 202 Accepted 2026-03-31 10:14:30,510 - lava_callback - INFO - Starting async job submit for node 69c8fb7f6a4c026e16348cc0 2026-03-31 10:14:30,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:14:30,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb7f6a4c026e16348cc0 HTTP/1.1" 200 None 2026-03-31 10:14:30,532 - lava_callback - DEBUG - Processing node 69c8fb7f6a4c026e16348cc0: baseline-arm-clabbe 2026-03-31 10:14:30,532 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fb7f6a4c026e16348cc0 2026-03-31 10:14:30,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:14:30,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fb7f6a4c026e16348cc0 HTTP/1.1" 200 None 2026-03-31 10:14:30,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:14:30,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:14:30,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:14:30,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fb7f6a4c026e16348cc0 HTTP/1.1" 200 None 2026-03-31 10:14:30,586 - lava_callback - ERROR - Error processing callback for node 69c8fb7f6a4c026e16348cc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60650 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:15:10,482 - lava_callback - INFO - Received callback for node 69c8fba56a4c026e16348ccc INFO: 172.19.0.1:40316 - "POST /node/69c8fba56a4c026e16348ccc HTTP/1.0" 202 Accepted 2026-03-31 10:15:10,488 - lava_callback - INFO - Starting async job submit for node 69c8fba56a4c026e16348ccc 2026-03-31 10:15:10,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:10,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fba56a4c026e16348ccc HTTP/1.1" 200 None 2026-03-31 10:15:10,516 - lava_callback - DEBUG - Processing node 69c8fba56a4c026e16348ccc: baseline-arm-clabbe 2026-03-31 10:15:10,516 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fba56a4c026e16348ccc 2026-03-31 10:15:10,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:10,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fba56a4c026e16348ccc HTTP/1.1" 200 None 2026-03-31 10:15:10,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:10,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:15:10,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:10,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fba56a4c026e16348ccc HTTP/1.1" 200 None 2026-03-31 10:15:10,566 - lava_callback - ERROR - Error processing callback for node 69c8fba56a4c026e16348ccc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57064 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:15:42,090 - lava_callback - INFO - Received callback for node 69cb9ac72a64fa9aff9377b6 INFO: 172.19.0.1:47218 - "POST /node/69cb9ac72a64fa9aff9377b6 HTTP/1.0" 202 Accepted 2026-03-31 10:15:42,097 - lava_callback - INFO - Starting async job submit for node 69cb9ac72a64fa9aff9377b6 2026-03-31 10:15:42,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:42,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9ac72a64fa9aff9377b6 HTTP/1.1" 200 None 2026-03-31 10:15:42,197 - lava_callback - DEBUG - Processing node 69cb9ac72a64fa9aff9377b6: baseline-arm 2026-03-31 10:15:42,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:15:43,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:15:43,050 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cb9ac72a64fa9aff9377b6/log.txt.gz 2026-03-31 10:15:43,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:15:43,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:15:43,621 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cb9ac72a64fa9aff9377b6/lava-logs.yaml 2026-03-31 10:15:43,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:15:43,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:15:43,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cb9ac72a64fa9aff9377b6/lava_callback.json.gz DEBUG: Setup failed for 69cb9ac72a64fa9aff9377b6. Transitting job node result: incomplete -> fail 2026-03-31 10:15:43,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:43,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9ac72a64fa9aff9377b6 HTTP/1.1" 200 None 2026-03-31 10:15:43,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:43,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 10:15:43,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:43,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb9ac72a64fa9aff9377b6 HTTP/1.1" 200 None 2026-03-31 10:15:44,067 - lava_callback - INFO - Completed processing callback for node 69cb9ac72a64fa9aff9377b6 INFO: 172.19.0.1:47230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47242 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:15:50,533 - lava_callback - INFO - Received callback for node 69c8fbc96a4c026e16348cd7 INFO: 172.19.0.1:36514 - "POST /node/69c8fbc96a4c026e16348cd7 HTTP/1.0" 202 Accepted 2026-03-31 10:15:50,539 - lava_callback - INFO - Starting async job submit for node 69c8fbc96a4c026e16348cd7 2026-03-31 10:15:50,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:50,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fbc96a4c026e16348cd7 HTTP/1.1" 200 None 2026-03-31 10:15:50,560 - lava_callback - DEBUG - Processing node 69c8fbc96a4c026e16348cd7: baseline-arm-clabbe 2026-03-31 10:15:50,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fbc96a4c026e16348cd7 2026-03-31 10:15:50,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:50,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fbc96a4c026e16348cd7 HTTP/1.1" 200 None 2026-03-31 10:15:50,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:50,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-31 10:15:50,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:15:50,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fbc96a4c026e16348cd7 HTTP/1.1" 200 None 2026-03-31 10:15:50,612 - lava_callback - ERROR - Error processing callback for node 69c8fbc96a4c026e16348cd7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45764 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:16:08,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:08,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 10:16:13,128 - lava_callback - INFO - Received callback for node 69cb9dab2a64fa9aff93781f INFO: 172.19.0.1:34294 - "POST /node/69cb9dab2a64fa9aff93781f HTTP/1.0" 202 Accepted 2026-03-31 10:16:13,136 - lava_callback - INFO - Starting async job submit for node 69cb9dab2a64fa9aff93781f 2026-03-31 10:16:13,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:13,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9dab2a64fa9aff93781f HTTP/1.1" 200 None 2026-03-31 10:16:13,224 - lava_callback - DEBUG - Processing node 69cb9dab2a64fa9aff93781f: baseline-arm64 2026-03-31 10:16:13,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:16:13,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:16:13,834 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9dab2a64fa9aff93781f/log.txt.gz 2026-03-31 10:16:14,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:16:14,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:16:14,242 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9dab2a64fa9aff93781f/lava-logs.yaml 2026-03-31 10:16:14,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:16:14,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:16:14,403 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9dab2a64fa9aff93781f/lava_callback.json.gz DEBUG: Setup failed for 69cb9dab2a64fa9aff93781f. Transitting job node result: incomplete -> fail 2026-03-31 10:16:14,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:14,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9dab2a64fa9aff93781f HTTP/1.1" 200 None 2026-03-31 10:16:14,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:14,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:16:14,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:14,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb9dab2a64fa9aff93781f HTTP/1.1" 200 None 2026-03-31 10:16:14,605 - lava_callback - INFO - Completed processing callback for node 69cb9dab2a64fa9aff93781f INFO: 172.19.0.1:34300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39842 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:16:38,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:16:38,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36316 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:18:23,423 - lava_callback - INFO - Received callback for node 69cb9f382a64fa9aff937860 INFO: 172.19.0.1:34100 - "POST /node/69cb9f382a64fa9aff937860 HTTP/1.0" 202 Accepted 2026-03-31 10:18:23,430 - lava_callback - INFO - Starting async job submit for node 69cb9f382a64fa9aff937860 2026-03-31 10:18:23,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:23,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9f382a64fa9aff937860 HTTP/1.1" 200 None 2026-03-31 10:18:23,478 - lava_callback - DEBUG - Processing node 69cb9f382a64fa9aff937860: baseline-arm64 2026-03-31 10:18:23,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:18:23,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:18:23,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9f382a64fa9aff937860/log.txt.gz 2026-03-31 10:18:23,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:18:24,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:18:24,109 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9f382a64fa9aff937860/lava-logs.yaml 2026-03-31 10:18:24,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:18:24,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:18:24,260 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cb9f382a64fa9aff937860/lava_callback.json.gz 2026-03-31 10:18:24,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:24,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9f382a64fa9aff937860 HTTP/1.1" 200 None 2026-03-31 10:18:24,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:24,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:18:24,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:24,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb9f382a64fa9aff937860 HTTP/1.1" 200 None 2026-03-31 10:18:24,427 - lava_callback - INFO - Completed processing callback for node 69cb9f382a64fa9aff937860 INFO: 172.19.0.1:34108 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:18:31,691 - lava_callback - INFO - Received callback for node 69c8fc666a4c026e16348cf8 INFO: 172.19.0.1:46846 - "POST /node/69c8fc666a4c026e16348cf8 HTTP/1.0" 202 Accepted 2026-03-31 10:18:31,700 - lava_callback - INFO - Starting async job submit for node 69c8fc666a4c026e16348cf8 2026-03-31 10:18:31,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:31,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fc666a4c026e16348cf8 HTTP/1.1" 200 None 2026-03-31 10:18:31,717 - lava_callback - DEBUG - Processing node 69c8fc666a4c026e16348cf8: baseline-arm64-baylibre 2026-03-31 10:18:31,717 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fc666a4c026e16348cf8 2026-03-31 10:18:31,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:31,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fc666a4c026e16348cf8 HTTP/1.1" 200 None 2026-03-31 10:18:31,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:31,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:18:31,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:31,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fc666a4c026e16348cf8 HTTP/1.1" 200 None 2026-03-31 10:18:31,781 - lava_callback - ERROR - Error processing callback for node 69c8fc666a4c026e16348cf8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46852 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:18:38,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:18:38,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:41978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39072 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:19:51,876 - lava_callback - INFO - Received callback for node 69c8fcb26a4c026e16348cfe INFO: 172.19.0.1:59510 - "POST /node/69c8fcb26a4c026e16348cfe HTTP/1.0" 202 Accepted 2026-03-31 10:19:51,884 - lava_callback - INFO - Starting async job submit for node 69c8fcb26a4c026e16348cfe 2026-03-31 10:19:51,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:19:51,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fcb26a4c026e16348cfe HTTP/1.1" 200 None 2026-03-31 10:19:51,900 - lava_callback - DEBUG - Processing node 69c8fcb26a4c026e16348cfe: baseline-arm64-baylibre 2026-03-31 10:19:51,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fcb26a4c026e16348cfe 2026-03-31 10:19:51,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:19:51,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fcb26a4c026e16348cfe HTTP/1.1" 200 None 2026-03-31 10:19:51,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:19:51,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:19:51,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:19:51,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fcb26a4c026e16348cfe HTTP/1.1" 200 None 2026-03-31 10:19:51,962 - lava_callback - ERROR - Error processing callback for node 69c8fcb26a4c026e16348cfe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50052 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:20:51,656 - lava_callback - INFO - Received callback for node 69c8fd006a4c026e16348d08 INFO: 172.19.0.1:41302 - "POST /node/69c8fd006a4c026e16348d08 HTTP/1.0" 202 Accepted 2026-03-31 10:20:51,661 - lava_callback - INFO - Starting async job submit for node 69c8fd006a4c026e16348d08 2026-03-31 10:20:51,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:20:51,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fd006a4c026e16348d08 HTTP/1.1" 200 None 2026-03-31 10:20:51,681 - lava_callback - DEBUG - Processing node 69c8fd006a4c026e16348d08: baseline-arm64-baylibre 2026-03-31 10:20:51,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fd006a4c026e16348d08 2026-03-31 10:20:51,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:20:51,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fd006a4c026e16348d08 HTTP/1.1" 200 None 2026-03-31 10:20:51,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:20:51,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:20:51,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:20:51,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fd006a4c026e16348d08 HTTP/1.1" 200 None 2026-03-31 10:20:51,745 - lava_callback - ERROR - Error processing callback for node 69c8fd006a4c026e16348d08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50702 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:22:11,762 - lava_callback - INFO - Received callback for node 69c8fd4c6a4c026e16348d12 INFO: 172.19.0.1:34770 - "POST /node/69c8fd4c6a4c026e16348d12 HTTP/1.0" 202 Accepted 2026-03-31 10:22:11,767 - lava_callback - INFO - Starting async job submit for node 69c8fd4c6a4c026e16348d12 2026-03-31 10:22:11,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:11,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fd4c6a4c026e16348d12 HTTP/1.1" 200 None 2026-03-31 10:22:11,804 - lava_callback - DEBUG - Processing node 69c8fd4c6a4c026e16348d12: baseline-arm64-baylibre 2026-03-31 10:22:11,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fd4c6a4c026e16348d12 2026-03-31 10:22:11,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:11,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fd4c6a4c026e16348d12 HTTP/1.1" 200 None 2026-03-31 10:22:11,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:11,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:22:11,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:11,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fd4c6a4c026e16348d12 HTTP/1.1" 200 None 2026-03-31 10:22:11,861 - lava_callback - ERROR - Error processing callback for node 69c8fd4c6a4c026e16348d12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36000 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:22:58,125 - lava_callback - INFO - Received callback for node 69cba04b2a64fa9aff937886 INFO: 172.19.0.1:36008 - "POST /node/69cba04b2a64fa9aff937886 HTTP/1.0" 202 Accepted 2026-03-31 10:22:58,132 - lava_callback - INFO - Starting async job submit for node 69cba04b2a64fa9aff937886 2026-03-31 10:22:58,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:58,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba04b2a64fa9aff937886 HTTP/1.1" 200 None 2026-03-31 10:22:58,176 - lava_callback - DEBUG - Processing node 69cba04b2a64fa9aff937886: baseline-arm64 2026-03-31 10:22:58,310 - lava_callback - WARNING - Failed to upload log 2026-03-31 10:22:58,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:22:58,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:22:58,611 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba04b2a64fa9aff937886/lava-logs.yaml 2026-03-31 10:22:58,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:22:58,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:22:58,738 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba04b2a64fa9aff937886/lava_callback.json.gz 2026-03-31 10:22:58,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:58,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba04b2a64fa9aff937886 HTTP/1.1" 200 None 2026-03-31 10:22:58,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:58,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:22:58,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:22:58,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba04b2a64fa9aff937886 HTTP/1.1" 200 None 2026-03-31 10:22:58,896 - lava_callback - INFO - Completed processing callback for node 69cba04b2a64fa9aff937886 INFO: 172.19.0.1:36714 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:23:08,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:23:08,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:37992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36306 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:24:52,006 - lava_callback - INFO - Received callback for node 69c8fde46a4c026e16348d25 INFO: 172.19.0.1:54982 - "POST /node/69c8fde46a4c026e16348d25 HTTP/1.0" 202 Accepted 2026-03-31 10:24:52,011 - lava_callback - INFO - Starting async job submit for node 69c8fde46a4c026e16348d25 2026-03-31 10:24:52,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:24:52,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fde46a4c026e16348d25 HTTP/1.1" 200 None 2026-03-31 10:24:52,028 - lava_callback - DEBUG - Processing node 69c8fde46a4c026e16348d25: baseline-arm64-baylibre 2026-03-31 10:24:52,028 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fde46a4c026e16348d25 2026-03-31 10:24:52,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:24:52,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fde46a4c026e16348d25 HTTP/1.1" 200 None 2026-03-31 10:24:52,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:24:52,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:24:52,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:24:52,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fde46a4c026e16348d25 HTTP/1.1" 200 None 2026-03-31 10:24:52,080 - lava_callback - ERROR - Error processing callback for node 69c8fde46a4c026e16348d25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49432 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:27:31,040 - lava_callback - INFO - Received callback for node 69c8fe806a4c026e16348d3c INFO: 172.19.0.1:34052 - "POST /node/69c8fe806a4c026e16348d3c HTTP/1.0" 202 Accepted 2026-03-31 10:27:31,046 - lava_callback - INFO - Starting async job submit for node 69c8fe806a4c026e16348d3c 2026-03-31 10:27:31,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:27:31,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fe806a4c026e16348d3c HTTP/1.1" 200 None 2026-03-31 10:27:31,062 - lava_callback - DEBUG - Processing node 69c8fe806a4c026e16348d3c: baseline-arm64-clabbe 2026-03-31 10:27:31,062 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fe806a4c026e16348d3c 2026-03-31 10:27:31,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:27:31,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fe806a4c026e16348d3c HTTP/1.1" 200 None 2026-03-31 10:27:31,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:27:31,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:27:31,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:27:31,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fe806a4c026e16348d3c HTTP/1.1" 200 None 2026-03-31 10:27:31,121 - lava_callback - ERROR - Error processing callback for node 69c8fe806a4c026e16348d3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56590 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:28:31,114 - lava_callback - INFO - Received callback for node 69c8fecb6a4c026e16348d44 INFO: 172.19.0.1:46098 - "POST /node/69c8fecb6a4c026e16348d44 HTTP/1.0" 202 Accepted 2026-03-31 10:28:31,119 - lava_callback - INFO - Starting async job submit for node 69c8fecb6a4c026e16348d44 2026-03-31 10:28:31,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:28:31,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fecb6a4c026e16348d44 HTTP/1.1" 200 None 2026-03-31 10:28:31,140 - lava_callback - DEBUG - Processing node 69c8fecb6a4c026e16348d44: baseline-arm64-clabbe 2026-03-31 10:28:31,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8fecb6a4c026e16348d44 2026-03-31 10:28:31,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:28:31,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8fecb6a4c026e16348d44 HTTP/1.1" 200 None 2026-03-31 10:28:31,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:28:31,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:28:31,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:28:31,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8fecb6a4c026e16348d44 HTTP/1.1" 200 None 2026-03-31 10:28:31,193 - lava_callback - ERROR - Error processing callback for node 69c8fecb6a4c026e16348d44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37322 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:29:51,179 - lava_callback - INFO - Received callback for node 69c8ff176a4c026e16348d57 INFO: 172.19.0.1:44062 - "POST /node/69c8ff176a4c026e16348d57 HTTP/1.0" 202 Accepted 2026-03-31 10:29:51,184 - lava_callback - INFO - Starting async job submit for node 69c8ff176a4c026e16348d57 2026-03-31 10:29:51,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:29:51,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ff176a4c026e16348d57 HTTP/1.1" 200 None 2026-03-31 10:29:51,200 - lava_callback - DEBUG - Processing node 69c8ff176a4c026e16348d57: baseline-arm64-clabbe 2026-03-31 10:29:51,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ff176a4c026e16348d57 2026-03-31 10:29:51,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:29:51,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ff176a4c026e16348d57 HTTP/1.1" 200 None 2026-03-31 10:29:51,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:29:51,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:29:51,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:29:51,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ff176a4c026e16348d57 HTTP/1.1" 200 None 2026-03-31 10:29:51,252 - lava_callback - ERROR - Error processing callback for node 69c8ff176a4c026e16348d57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47198 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:31:11,230 - lava_callback - INFO - Received callback for node 69c8ff646a4c026e16348d5c INFO: 172.19.0.1:42018 - "POST /node/69c8ff646a4c026e16348d5c HTTP/1.0" 202 Accepted 2026-03-31 10:31:11,236 - lava_callback - INFO - Starting async job submit for node 69c8ff646a4c026e16348d5c 2026-03-31 10:31:11,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:31:11,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ff646a4c026e16348d5c HTTP/1.1" 200 None 2026-03-31 10:31:11,262 - lava_callback - DEBUG - Processing node 69c8ff646a4c026e16348d5c: baseline-arm64-clabbe 2026-03-31 10:31:11,262 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ff646a4c026e16348d5c 2026-03-31 10:31:11,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:31:11,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ff646a4c026e16348d5c HTTP/1.1" 200 None 2026-03-31 10:31:11,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:31:11,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:31:11,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:31:11,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ff646a4c026e16348d5c HTTP/1.1" 200 None 2026-03-31 10:31:11,325 - lava_callback - ERROR - Error processing callback for node 69c8ff646a4c026e16348d5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52790 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:32:33,858 - lava_callback - INFO - Received callback for node 69c8ffb46a4c026e16348d66 INFO: 172.19.0.1:56866 - "POST /node/69c8ffb46a4c026e16348d66 HTTP/1.0" 202 Accepted 2026-03-31 10:32:33,864 - lava_callback - INFO - Starting async job submit for node 69c8ffb46a4c026e16348d66 2026-03-31 10:32:33,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:32:33,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ffb46a4c026e16348d66 HTTP/1.1" 200 None 2026-03-31 10:32:33,880 - lava_callback - DEBUG - Processing node 69c8ffb46a4c026e16348d66: baseline-arm64-clabbe 2026-03-31 10:32:33,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c8ffb46a4c026e16348d66 2026-03-31 10:32:33,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:32:33,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8ffb46a4c026e16348d66 HTTP/1.1" 200 None 2026-03-31 10:32:33,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:32:33,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:32:33,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:32:33,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8ffb46a4c026e16348d66 HTTP/1.1" 200 None 2026-03-31 10:32:33,949 - lava_callback - ERROR - Error processing callback for node 69c8ffb46a4c026e16348d66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33126 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33130 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:33:51,352 - lava_callback - INFO - Received callback for node 69c900026a4c026e16348d6f INFO: 172.19.0.1:34772 - "POST /node/69c900026a4c026e16348d6f HTTP/1.0" 202 Accepted 2026-03-31 10:33:51,358 - lava_callback - INFO - Starting async job submit for node 69c900026a4c026e16348d6f 2026-03-31 10:33:51,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:33:51,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c900026a4c026e16348d6f HTTP/1.1" 200 None 2026-03-31 10:33:51,376 - lava_callback - DEBUG - Processing node 69c900026a4c026e16348d6f: baseline-arm64-clabbe 2026-03-31 10:33:51,377 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c900026a4c026e16348d6f 2026-03-31 10:33:51,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:33:51,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c900026a4c026e16348d6f HTTP/1.1" 200 None 2026-03-31 10:33:51,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:33:51,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-31 10:33:51,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:33:51,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c900026a4c026e16348d6f HTTP/1.1" 200 None 2026-03-31 10:33:51,449 - lava_callback - ERROR - Error processing callback for node 69c900026a4c026e16348d6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37254 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:39:13,533 - lava_callback - INFO - Received callback for node 69cba0002a64fa9aff93787d INFO: 172.19.0.1:46750 - "POST /node/69cba0002a64fa9aff93787d HTTP/1.0" 202 Accepted 2026-03-31 10:39:13,540 - lava_callback - INFO - Starting async job submit for node 69cba0002a64fa9aff93787d 2026-03-31 10:39:13,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:39:13,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba0002a64fa9aff93787d HTTP/1.1" 200 None 2026-03-31 10:39:13,590 - lava_callback - DEBUG - Processing node 69cba0002a64fa9aff93787d: baseline-arm64 2026-03-31 10:39:14,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:39:14,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:39:14,232 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba0002a64fa9aff93787d/log.txt.gz 2026-03-31 10:39:14,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:39:14,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:39:14,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba0002a64fa9aff93787d/lava-logs.yaml 2026-03-31 10:39:14,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:39:14,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:39:14,978 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba0002a64fa9aff93787d/lava_callback.json.gz 2026-03-31 10:39:15,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:39:15,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba0002a64fa9aff93787d HTTP/1.1" 200 None 2026-03-31 10:39:15,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:39:15,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:39:15,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:39:15,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba0002a64fa9aff93787d HTTP/1.1" 200 None INFO: 172.19.0.1:46762 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 10:39:15,168 - lava_callback - INFO - Completed processing callback for node 69cba0002a64fa9aff93787d INFO: 172.19.0.1:46778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39256 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:39:38,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:39:38,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:47484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37514 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:41:04,209 - lava_callback - INFO - Received callback for node 69cba0982a64fa9aff937892 INFO: 172.19.0.1:56340 - "POST /node/69cba0982a64fa9aff937892 HTTP/1.0" 202 Accepted 2026-03-31 10:41:04,216 - lava_callback - INFO - Starting async job submit for node 69cba0982a64fa9aff937892 2026-03-31 10:41:04,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:04,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba0982a64fa9aff937892 HTTP/1.1" 200 None 2026-03-31 10:41:04,290 - lava_callback - DEBUG - Processing node 69cba0982a64fa9aff937892: ltp-smoke 2026-03-31 10:41:04,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:04,967 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:04,969 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba0982a64fa9aff937892/log.txt.gz 2026-03-31 10:41:05,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:05,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:05,593 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba0982a64fa9aff937892/lava-logs.yaml 2026-03-31 10:41:05,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:05,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:05,787 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba0982a64fa9aff937892/lava_callback.json.gz 2026-03-31 10:41:05,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:05,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba0982a64fa9aff937892 HTTP/1.1" 200 None 2026-03-31 10:41:05,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:05,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83462a64fa9aff936905 HTTP/1.1" 200 None 2026-03-31 10:41:05,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:05,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba0982a64fa9aff937892 HTTP/1.1" 200 None 2026-03-31 10:41:06,101 - lava_callback - INFO - Completed processing callback for node 69cba0982a64fa9aff937892 INFO: 172.19.0.1:56352 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:41:08,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:08,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:56974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52172 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:41:40,902 - lava_callback - INFO - Received callback for node 69cb9f912a64fa9aff93786e INFO: 172.19.0.1:33744 - "POST /node/69cb9f912a64fa9aff93786e HTTP/1.0" 202 Accepted 2026-03-31 10:41:40,951 - lava_callback - INFO - Starting async job submit for node 69cb9f912a64fa9aff93786e 2026-03-31 10:41:40,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:40,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9f912a64fa9aff93786e HTTP/1.1" 200 None 2026-03-31 10:41:40,992 - lava_callback - DEBUG - Processing node 69cb9f912a64fa9aff93786e: baseline-arm-pengutronix 2026-03-31 10:41:41,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:41,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:41,268 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb9f912a64fa9aff93786e/log.txt.gz 2026-03-31 10:41:41,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:41,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:41,523 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb9f912a64fa9aff93786e/lava-logs.yaml 2026-03-31 10:41:41,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:41:41,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:41:41,647 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cb9f912a64fa9aff93786e/lava_callback.json.gz DEBUG: Setup failed for 69cb9f912a64fa9aff93786e. Transitting job node result: incomplete -> fail 2026-03-31 10:41:41,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:41,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb9f912a64fa9aff93786e HTTP/1.1" 200 None 2026-03-31 10:41:41,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:41,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 10:41:41,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:41:41,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb9f912a64fa9aff93786e HTTP/1.1" 200 None 2026-03-31 10:41:41,830 - lava_callback - INFO - Completed processing callback for node 69cb9f912a64fa9aff93786e INFO: 172.19.0.1:33750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51958 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:42:02,449 - lava_callback - INFO - Received callback for node 69cba2772a64fa9aff9378ce INFO: 172.19.0.1:49576 - "POST /node/69cba2772a64fa9aff9378ce HTTP/1.0" 202 Accepted 2026-03-31 10:42:02,456 - lava_callback - INFO - Starting async job submit for node 69cba2772a64fa9aff9378ce 2026-03-31 10:42:02,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:02,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba2772a64fa9aff9378ce HTTP/1.1" 200 None 2026-03-31 10:42:02,500 - lava_callback - DEBUG - Processing node 69cba2772a64fa9aff9378ce: baseline-nfs-arm64 2026-03-31 10:42:03,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:03,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:03,180 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba2772a64fa9aff9378ce/log.txt.gz 2026-03-31 10:42:03,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:03,702 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:03,705 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba2772a64fa9aff9378ce/lava-logs.yaml 2026-03-31 10:42:03,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:03,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:03,910 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba2772a64fa9aff9378ce/lava_callback.json.gz 2026-03-31 10:42:03,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:03,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba2772a64fa9aff9378ce HTTP/1.1" 200 None 2026-03-31 10:42:03,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:04,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:42:04,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:04,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba2772a64fa9aff9378ce HTTP/1.1" 200 None 2026-03-31 10:42:04,083 - lava_callback - INFO - Completed processing callback for node 69cba2772a64fa9aff9378ce INFO: 172.19.0.1:49580 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:42:08,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:08,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:59838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38622 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:42:31,971 - lava_callback - INFO - Received callback for node 69cba33c2a64fa9aff9378e6 INFO: 172.19.0.1:54048 - "POST /node/69cba33c2a64fa9aff9378e6 HTTP/1.0" 202 Accepted 2026-03-31 10:42:31,979 - lava_callback - INFO - Starting async job submit for node 69cba33c2a64fa9aff9378e6 2026-03-31 10:42:32,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:32,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba33c2a64fa9aff9378e6 HTTP/1.1" 200 None 2026-03-31 10:42:32,030 - lava_callback - DEBUG - Processing node 69cba33c2a64fa9aff9378e6: baseline-arm64 2026-03-31 10:42:32,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:32,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:32,543 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba33c2a64fa9aff9378e6/log.txt.gz 2026-03-31 10:42:32,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:33,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:33,018 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba33c2a64fa9aff9378e6/lava-logs.yaml 2026-03-31 10:42:33,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:42:33,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:42:33,197 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba33c2a64fa9aff9378e6/lava_callback.json.gz DEBUG: Setup failed for 69cba33c2a64fa9aff9378e6. Transitting job node result: incomplete -> fail 2026-03-31 10:42:33,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:33,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba33c2a64fa9aff9378e6 HTTP/1.1" 200 None 2026-03-31 10:42:33,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:33,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:42:33,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:33,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba33c2a64fa9aff9378e6 HTTP/1.1" 200 None 2026-03-31 10:42:33,479 - lava_callback - INFO - Completed processing callback for node 69cba33c2a64fa9aff9378e6 INFO: 172.19.0.1:54060 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:42:38,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:42:38,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39966 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:46:11,918 - lava_callback - INFO - Received callback for node 69c902e66a4c026e16348def INFO: 172.19.0.1:39110 - "POST /node/69c902e66a4c026e16348def HTTP/1.0" 202 Accepted 2026-03-31 10:46:11,923 - lava_callback - INFO - Starting async job submit for node 69c902e66a4c026e16348def 2026-03-31 10:46:11,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:11,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c902e66a4c026e16348def HTTP/1.1" 200 None 2026-03-31 10:46:11,949 - lava_callback - DEBUG - Processing node 69c902e66a4c026e16348def: baseline-arm-clabbe 2026-03-31 10:46:11,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c902e66a4c026e16348def 2026-03-31 10:46:11,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:11,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c902e66a4c026e16348def HTTP/1.1" 200 None 2026-03-31 10:46:11,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:12,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 10:46:12,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:12,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c902e66a4c026e16348def HTTP/1.1" 200 None 2026-03-31 10:46:12,031 - lava_callback - ERROR - Error processing callback for node 69c902e66a4c026e16348def: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44758 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:46:51,881 - lava_callback - INFO - Received callback for node 69c9030c6a4c026e16348dfd INFO: 172.19.0.1:40188 - "POST /node/69c9030c6a4c026e16348dfd HTTP/1.0" 202 Accepted 2026-03-31 10:46:51,886 - lava_callback - INFO - Starting async job submit for node 69c9030c6a4c026e16348dfd 2026-03-31 10:46:51,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:51,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9030c6a4c026e16348dfd HTTP/1.1" 200 None 2026-03-31 10:46:51,903 - lava_callback - DEBUG - Processing node 69c9030c6a4c026e16348dfd: baseline-arm-clabbe 2026-03-31 10:46:51,903 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9030c6a4c026e16348dfd 2026-03-31 10:46:51,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:51,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9030c6a4c026e16348dfd HTTP/1.1" 200 None 2026-03-31 10:46:51,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:51,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 10:46:51,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:46:51,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9030c6a4c026e16348dfd HTTP/1.1" 200 None 2026-03-31 10:46:51,975 - lava_callback - ERROR - Error processing callback for node 69c9030c6a4c026e16348dfd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48970 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:47:31,930 - lava_callback - INFO - Received callback for node 69c9033b6a4c026e16348e08 INFO: 172.19.0.1:33360 - "POST /node/69c9033b6a4c026e16348e08 HTTP/1.0" 202 Accepted 2026-03-31 10:47:31,935 - lava_callback - INFO - Starting async job submit for node 69c9033b6a4c026e16348e08 2026-03-31 10:47:31,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:47:31,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9033b6a4c026e16348e08 HTTP/1.1" 200 None 2026-03-31 10:47:31,955 - lava_callback - DEBUG - Processing node 69c9033b6a4c026e16348e08: baseline-arm-clabbe 2026-03-31 10:47:31,955 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9033b6a4c026e16348e08 2026-03-31 10:47:31,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:47:31,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9033b6a4c026e16348e08 HTTP/1.1" 200 None 2026-03-31 10:47:31,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:47:32,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 10:47:32,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:47:32,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9033b6a4c026e16348e08 HTTP/1.1" 200 None 2026-03-31 10:47:32,033 - lava_callback - ERROR - Error processing callback for node 69c9033b6a4c026e16348e08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55282 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:48:12,045 - lava_callback - INFO - Received callback for node 69c903616a4c026e16348e15 INFO: 172.19.0.1:47734 - "POST /node/69c903616a4c026e16348e15 HTTP/1.0" 202 Accepted 2026-03-31 10:48:12,050 - lava_callback - INFO - Starting async job submit for node 69c903616a4c026e16348e15 2026-03-31 10:48:12,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:12,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c903616a4c026e16348e15 HTTP/1.1" 200 None 2026-03-31 10:48:12,088 - lava_callback - DEBUG - Processing node 69c903616a4c026e16348e15: baseline-arm-clabbe 2026-03-31 10:48:12,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c903616a4c026e16348e15 2026-03-31 10:48:12,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:12,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c903616a4c026e16348e15 HTTP/1.1" 200 None 2026-03-31 10:48:12,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:12,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 10:48:12,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:12,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c903616a4c026e16348e15 HTTP/1.1" 200 None 2026-03-31 10:48:12,142 - lava_callback - ERROR - Error processing callback for node 69c903616a4c026e16348e15: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40380 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:48:40,884 - lava_callback - INFO - Received callback for node 69cba1da2a64fa9aff9378b8 INFO: 172.19.0.1:33700 - "POST /node/69cba1da2a64fa9aff9378b8 HTTP/1.0" 202 Accepted 2026-03-31 10:48:40,891 - lava_callback - INFO - Starting async job submit for node 69cba1da2a64fa9aff9378b8 2026-03-31 10:48:40,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:40,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba1da2a64fa9aff9378b8 HTTP/1.1" 200 None 2026-03-31 10:48:40,954 - lava_callback - DEBUG - Processing node 69cba1da2a64fa9aff9378b8: baseline-nfs-arm64 2026-03-31 10:48:41,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:48:42,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:48:42,317 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba1da2a64fa9aff9378b8/log.txt.gz 2026-03-31 10:48:42,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:48:43,149 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:48:43,152 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba1da2a64fa9aff9378b8/lava-logs.yaml 2026-03-31 10:48:43,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:48:43,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:48:43,373 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba1da2a64fa9aff9378b8/lava_callback.json.gz DEBUG: Setup failed for 69cba1da2a64fa9aff9378b8. Transitting job node result: incomplete -> fail 2026-03-31 10:48:43,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:43,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba1da2a64fa9aff9378b8 HTTP/1.1" 200 None 2026-03-31 10:48:43,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:43,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:48:43,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:48:43,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba1da2a64fa9aff9378b8 HTTP/1.1" 200 None 2026-03-31 10:48:43,601 - lava_callback - INFO - Completed processing callback for node 69cba1da2a64fa9aff9378b8 INFO: 172.19.0.1:33702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47568 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:49:08,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:08,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:53304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53314 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:49:17,428 - lava_callback - INFO - Received callback for node 69cba4592a64fa9aff9378f1 INFO: 172.19.0.1:53320 - "POST /node/69cba4592a64fa9aff9378f1 HTTP/1.0" 202 Accepted 2026-03-31 10:49:17,435 - lava_callback - INFO - Starting async job submit for node 69cba4592a64fa9aff9378f1 2026-03-31 10:49:17,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:17,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4592a64fa9aff9378f1 HTTP/1.1" 200 None 2026-03-31 10:49:17,483 - lava_callback - DEBUG - Processing node 69cba4592a64fa9aff9378f1: baseline-arm64 2026-03-31 10:49:17,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:49:18,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:49:18,120 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba4592a64fa9aff9378f1/log.txt.gz 2026-03-31 10:49:18,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:49:18,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:49:18,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba4592a64fa9aff9378f1/lava-logs.yaml 2026-03-31 10:49:18,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:49:18,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:49:18,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cba4592a64fa9aff9378f1/lava_callback.json.gz 2026-03-31 10:49:18,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:18,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4592a64fa9aff9378f1 HTTP/1.1" 200 None 2026-03-31 10:49:18,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:18,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:49:18,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:18,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba4592a64fa9aff9378f1 HTTP/1.1" 200 None 2026-03-31 10:49:18,921 - lava_callback - INFO - Completed processing callback for node 69cba4592a64fa9aff9378f1 INFO: 172.19.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43994 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:49:38,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:49:38,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:59208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45224 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:50:12,036 - lava_callback - INFO - Received callback for node 69c903ce6a4c026e16348e27 INFO: 172.19.0.1:37948 - "POST /node/69c903ce6a4c026e16348e27 HTTP/1.0" 202 Accepted 2026-03-31 10:50:12,041 - lava_callback - INFO - Starting async job submit for node 69c903ce6a4c026e16348e27 2026-03-31 10:50:12,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:12,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c903ce6a4c026e16348e27 HTTP/1.1" 200 None 2026-03-31 10:50:12,058 - lava_callback - DEBUG - Processing node 69c903ce6a4c026e16348e27: baseline-arm-clabbe 2026-03-31 10:50:12,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c903ce6a4c026e16348e27 2026-03-31 10:50:12,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:12,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c903ce6a4c026e16348e27 HTTP/1.1" 200 None 2026-03-31 10:50:12,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:12,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-03-31 10:50:12,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:12,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c903ce6a4c026e16348e27 HTTP/1.1" 200 None 2026-03-31 10:50:12,134 - lava_callback - ERROR - Error processing callback for node 69c903ce6a4c026e16348e27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37712 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:50:30,655 - lava_callback - INFO - Received callback for node 69cba3132a64fa9aff9378dd INFO: 172.19.0.1:52974 - "POST /node/69cba3132a64fa9aff9378dd HTTP/1.0" 202 Accepted 2026-03-31 10:50:30,662 - lava_callback - INFO - Starting async job submit for node 69cba3132a64fa9aff9378dd 2026-03-31 10:50:30,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:30,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba3132a64fa9aff9378dd HTTP/1.1" 200 None 2026-03-31 10:50:30,731 - lava_callback - DEBUG - Processing node 69cba3132a64fa9aff9378dd: baseline-arm 2026-03-31 10:50:31,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:50:31,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:50:31,538 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cba3132a64fa9aff9378dd/log.txt.gz 2026-03-31 10:50:31,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:50:32,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:50:32,341 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cba3132a64fa9aff9378dd/lava-logs.yaml 2026-03-31 10:50:32,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:50:32,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:50:32,528 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cba3132a64fa9aff9378dd/lava_callback.json.gz DEBUG: Setup failed for 69cba3132a64fa9aff9378dd. Transitting job node result: incomplete -> fail 2026-03-31 10:50:32,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:32,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba3132a64fa9aff9378dd HTTP/1.1" 200 None 2026-03-31 10:50:32,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:32,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 10:50:32,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:32,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba3132a64fa9aff9378dd HTTP/1.1" 200 None 2026-03-31 10:50:32,808 - lava_callback - INFO - Completed processing callback for node 69cba3132a64fa9aff9378dd INFO: 172.19.0.1:52986 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:50:38,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:50:38,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37136 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:53:05,656 - lava_callback - INFO - Received callback for node 69cba5142a64fa9aff937918 INFO: 172.19.0.1:40748 - "POST /node/69cba5142a64fa9aff937918 HTTP/1.0" 202 Accepted 2026-03-31 10:53:05,662 - lava_callback - INFO - Starting async job submit for node 69cba5142a64fa9aff937918 2026-03-31 10:53:05,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:53:05,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba5142a64fa9aff937918 HTTP/1.1" 200 None 2026-03-31 10:53:05,714 - lava_callback - DEBUG - Processing node 69cba5142a64fa9aff937918: baseline-nfs-arm64 2026-03-31 10:53:06,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:53:06,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:53:06,352 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba5142a64fa9aff937918/log.txt.gz INFO: 172.19.0.1:40756 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:53:06,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:53:07,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:53:07,020 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba5142a64fa9aff937918/lava-logs.yaml 2026-03-31 10:53:07,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:53:07,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:53:07,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba5142a64fa9aff937918/lava_callback.json.gz 2026-03-31 10:53:07,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:53:07,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba5142a64fa9aff937918 HTTP/1.1" 200 None 2026-03-31 10:53:07,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:53:07,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 10:53:07,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:53:07,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba5142a64fa9aff937918 HTTP/1.1" 200 None 2026-03-31 10:53:07,392 - lava_callback - INFO - Completed processing callback for node 69cba5142a64fa9aff937918 2026-03-31 10:53:08,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:53:09,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:37726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34366 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:57:33,740 - lava_callback - INFO - Received callback for node 69c9059a6a4c026e16348e53 INFO: 172.19.0.1:36362 - "POST /node/69c9059a6a4c026e16348e53 HTTP/1.0" 202 Accepted 2026-03-31 10:57:33,745 - lava_callback - INFO - Starting async job submit for node 69c9059a6a4c026e16348e53 2026-03-31 10:57:33,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:57:33,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9059a6a4c026e16348e53 HTTP/1.1" 200 None 2026-03-31 10:57:33,762 - lava_callback - DEBUG - Processing node 69c9059a6a4c026e16348e53: baseline-arm64-baylibre 2026-03-31 10:57:33,763 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9059a6a4c026e16348e53 2026-03-31 10:57:33,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:57:33,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9059a6a4c026e16348e53 HTTP/1.1" 200 None 2026-03-31 10:57:33,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:57:33,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 10:57:33,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:57:33,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9059a6a4c026e16348e53 HTTP/1.1" 200 None 2026-03-31 10:57:33,829 - lava_callback - ERROR - Error processing callback for node 69c9059a6a4c026e16348e53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44714 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:58:44,727 - lava_callback - INFO - Received callback for node 69cba4c32a64fa9aff9378fe INFO: 172.19.0.1:50942 - "POST /node/69cba4c32a64fa9aff9378fe HTTP/1.0" 202 Accepted 2026-03-31 10:58:44,734 - lava_callback - INFO - Starting async job submit for node 69cba4c32a64fa9aff9378fe 2026-03-31 10:58:44,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:44,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4c32a64fa9aff9378fe HTTP/1.1" 200 None 2026-03-31 10:58:44,812 - lava_callback - DEBUG - Processing node 69cba4c32a64fa9aff9378fe: ltp-smoke 2026-03-31 10:58:45,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:58:45,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:58:45,454 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba4c32a64fa9aff9378fe/log.txt.gz 2026-03-31 10:58:45,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:58:45,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:58:45,958 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba4c32a64fa9aff9378fe/lava-logs.yaml 2026-03-31 10:58:45,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 10:58:46,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 10:58:46,130 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cba4c32a64fa9aff9378fe/lava_callback.json.gz 2026-03-31 10:58:46,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:46,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4c32a64fa9aff9378fe HTTP/1.1" 200 None 2026-03-31 10:58:46,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:46,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83462a64fa9aff936905 HTTP/1.1" 200 None 2026-03-31 10:58:46,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:46,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba4c32a64fa9aff9378fe HTTP/1.1" 200 None 2026-03-31 10:58:46,457 - lava_callback - INFO - Completed processing callback for node 69cba4c32a64fa9aff9378fe INFO: 172.19.0.1:50952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50962 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:58:53,798 - lava_callback - INFO - Received callback for node 69c905e86a4c026e16348e65 INFO: 172.19.0.1:48918 - "POST /node/69c905e86a4c026e16348e65 HTTP/1.0" 202 Accepted 2026-03-31 10:58:53,807 - lava_callback - INFO - Starting async job submit for node 69c905e86a4c026e16348e65 2026-03-31 10:58:53,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:53,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c905e86a4c026e16348e65 HTTP/1.1" 200 None 2026-03-31 10:58:53,832 - lava_callback - DEBUG - Processing node 69c905e86a4c026e16348e65: baseline-arm64-baylibre 2026-03-31 10:58:53,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c905e86a4c026e16348e65 2026-03-31 10:58:53,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:53,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c905e86a4c026e16348e65 HTTP/1.1" 200 None 2026-03-31 10:58:53,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:53,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 10:58:53,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:58:53,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c905e86a4c026e16348e65 HTTP/1.1" 200 None 2026-03-31 10:58:53,908 - lava_callback - ERROR - Error processing callback for node 69c905e86a4c026e16348e65: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60728 - "GET / HTTP/1.0" 200 OK 2026-03-31 10:59:09,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 10:59:09,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:50704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50122 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:00:53,891 - lava_callback - INFO - Received callback for node 69c906516a4c026e16348e74 INFO: 172.19.0.1:46600 - "POST /node/69c906516a4c026e16348e74 HTTP/1.0" 202 Accepted 2026-03-31 11:00:53,897 - lava_callback - INFO - Starting async job submit for node 69c906516a4c026e16348e74 2026-03-31 11:00:53,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:00:53,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c906516a4c026e16348e74 HTTP/1.1" 200 None 2026-03-31 11:00:53,927 - lava_callback - DEBUG - Processing node 69c906516a4c026e16348e74: baseline-arm64-baylibre 2026-03-31 11:00:53,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c906516a4c026e16348e74 2026-03-31 11:00:53,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:00:53,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c906516a4c026e16348e74 HTTP/1.1" 200 None 2026-03-31 11:00:53,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:00:53,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:00:53,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:00:53,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c906516a4c026e16348e74 HTTP/1.1" 200 None 2026-03-31 11:00:53,995 - lava_callback - ERROR - Error processing callback for node 69c906516a4c026e16348e74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57218 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:01:22,040 - lava_callback - INFO - Received callback for node 69cba4f22a64fa9aff93790b INFO: 172.19.0.1:48122 - "POST /node/69cba4f22a64fa9aff93790b HTTP/1.0" 202 Accepted 2026-03-31 11:01:22,089 - lava_callback - INFO - Starting async job submit for node 69cba4f22a64fa9aff93790b 2026-03-31 11:01:22,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:01:22,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4f22a64fa9aff93790b HTTP/1.1" 200 None 2026-03-31 11:01:22,143 - lava_callback - DEBUG - Processing node 69cba4f22a64fa9aff93790b: baseline-arm-pengutronix 2026-03-31 11:01:22,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:01:22,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:01:22,409 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cba4f22a64fa9aff93790b/log.txt.gz 2026-03-31 11:01:22,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:01:22,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:01:22,660 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cba4f22a64fa9aff93790b/lava-logs.yaml 2026-03-31 11:01:22,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:01:22,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:01:22,802 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cba4f22a64fa9aff93790b/lava_callback.json.gz DEBUG: Setup failed for 69cba4f22a64fa9aff93790b. Transitting job node result: incomplete -> fail 2026-03-31 11:01:22,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:01:22,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba4f22a64fa9aff93790b HTTP/1.1" 200 None 2026-03-31 11:01:22,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:01:22,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-03-31 11:01:22,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:01:22,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba4f22a64fa9aff93790b HTTP/1.1" 200 None 2026-03-31 11:01:23,005 - lava_callback - INFO - Completed processing callback for node 69cba4f22a64fa9aff93790b INFO: 172.19.0.1:48138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46960 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:01:39,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:01:39,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:32896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56140 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:03:14,007 - lava_callback - INFO - Received callback for node 69c906e86a4c026e16348e8b INFO: 172.19.0.1:45236 - "POST /node/69c906e86a4c026e16348e8b HTTP/1.0" 202 Accepted 2026-03-31 11:03:14,013 - lava_callback - INFO - Starting async job submit for node 69c906e86a4c026e16348e8b 2026-03-31 11:03:14,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:03:14,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c906e86a4c026e16348e8b HTTP/1.1" 200 None 2026-03-31 11:03:14,040 - lava_callback - DEBUG - Processing node 69c906e86a4c026e16348e8b: baseline-arm64-baylibre 2026-03-31 11:03:14,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c906e86a4c026e16348e8b 2026-03-31 11:03:14,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:03:14,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c906e86a4c026e16348e8b HTTP/1.1" 200 None 2026-03-31 11:03:14,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:03:14,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:03:14,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:03:14,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c906e86a4c026e16348e8b HTTP/1.1" 200 None 2026-03-31 11:03:14,107 - lava_callback - ERROR - Error processing callback for node 69c906e86a4c026e16348e8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58550 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:08:34,309 - lava_callback - INFO - Received callback for node 69c908236a4c026e16348eba INFO: 172.19.0.1:50476 - "POST /node/69c908236a4c026e16348eba HTTP/1.0" 202 Accepted 2026-03-31 11:08:34,315 - lava_callback - INFO - Starting async job submit for node 69c908236a4c026e16348eba 2026-03-31 11:08:34,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:08:34,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c908236a4c026e16348eba HTTP/1.1" 200 None 2026-03-31 11:08:34,331 - lava_callback - DEBUG - Processing node 69c908236a4c026e16348eba: baseline-arm64-baylibre 2026-03-31 11:08:34,331 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c908236a4c026e16348eba 2026-03-31 11:08:34,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:08:34,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c908236a4c026e16348eba HTTP/1.1" 200 None 2026-03-31 11:08:34,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:08:34,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:08:34,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:08:34,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c908236a4c026e16348eba HTTP/1.1" 200 None 2026-03-31 11:08:34,387 - lava_callback - ERROR - Error processing callback for node 69c908236a4c026e16348eba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45908 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:09:01,550 - lava_callback - INFO - Received callback for node 69cba6942a64fa9aff93793e INFO: 172.19.0.1:57724 - "POST /node/69cba6942a64fa9aff93793e HTTP/1.0" 202 Accepted 2026-03-31 11:09:01,557 - lava_callback - INFO - Starting async job submit for node 69cba6942a64fa9aff93793e 2026-03-31 11:09:01,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:09:01,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba6942a64fa9aff93793e HTTP/1.1" 200 None 2026-03-31 11:09:01,620 - lava_callback - DEBUG - Processing node 69cba6942a64fa9aff93793e: baseline-nfs-arm64 2026-03-31 11:09:02,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:09:02,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:09:02,812 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba6942a64fa9aff93793e/log.txt.gz 2026-03-31 11:09:03,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:09:03,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:09:03,648 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba6942a64fa9aff93793e/lava-logs.yaml 2026-03-31 11:09:03,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 11:09:03,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 11:09:03,865 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cba6942a64fa9aff93793e/lava_callback.json.gz DEBUG: Setup failed for 69cba6942a64fa9aff93793e. Transitting job node result: incomplete -> fail 2026-03-31 11:09:03,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:09:03,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cba6942a64fa9aff93793e HTTP/1.1" 200 None 2026-03-31 11:09:03,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:09:03,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-03-31 11:09:03,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:09:04,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cba6942a64fa9aff93793e HTTP/1.1" 200 None 2026-03-31 11:09:04,102 - lava_callback - INFO - Completed processing callback for node 69cba6942a64fa9aff93793e INFO: 172.19.0.1:57726 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:09:09,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:09:09,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:58400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58956 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:11:14,495 - lava_callback - INFO - Received callback for node 69c908be6a4c026e16348ed2 INFO: 172.19.0.1:32994 - "POST /node/69c908be6a4c026e16348ed2 HTTP/1.0" 202 Accepted 2026-03-31 11:11:14,500 - lava_callback - INFO - Starting async job submit for node 69c908be6a4c026e16348ed2 2026-03-31 11:11:14,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:11:14,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c908be6a4c026e16348ed2 HTTP/1.1" 200 None 2026-03-31 11:11:14,525 - lava_callback - DEBUG - Processing node 69c908be6a4c026e16348ed2: baseline-arm64-baylibre 2026-03-31 11:11:14,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c908be6a4c026e16348ed2 2026-03-31 11:11:14,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:11:14,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c908be6a4c026e16348ed2 HTTP/1.1" 200 None 2026-03-31 11:11:14,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:11:14,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:11:14,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:11:14,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c908be6a4c026e16348ed2 HTTP/1.1" 200 None 2026-03-31 11:11:14,628 - lava_callback - ERROR - Error processing callback for node 69c908be6a4c026e16348ed2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:32996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43274 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:12:34,510 - lava_callback - INFO - Received callback for node 69c9091e6a4c026e16348edc INFO: 172.19.0.1:60974 - "POST /node/69c9091e6a4c026e16348edc HTTP/1.0" 202 Accepted 2026-03-31 11:12:34,515 - lava_callback - INFO - Starting async job submit for node 69c9091e6a4c026e16348edc 2026-03-31 11:12:34,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:12:34,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9091e6a4c026e16348edc HTTP/1.1" 200 None 2026-03-31 11:12:34,538 - lava_callback - DEBUG - Processing node 69c9091e6a4c026e16348edc: baseline-arm64-baylibre 2026-03-31 11:12:34,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9091e6a4c026e16348edc 2026-03-31 11:12:34,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:12:34,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9091e6a4c026e16348edc HTTP/1.1" 200 None 2026-03-31 11:12:34,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:12:34,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:12:34,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:12:34,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9091e6a4c026e16348edc HTTP/1.1" 200 None 2026-03-31 11:12:34,622 - lava_callback - ERROR - Error processing callback for node 69c9091e6a4c026e16348edc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39572 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:14:13,103 - lava_callback - INFO - Received callback for node 69c909726a4c026e16348ee9 INFO: 172.19.0.1:42008 - "POST /node/69c909726a4c026e16348ee9 HTTP/1.0" 202 Accepted 2026-03-31 11:14:13,110 - lava_callback - INFO - Starting async job submit for node 69c909726a4c026e16348ee9 2026-03-31 11:14:13,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:14:13,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c909726a4c026e16348ee9 HTTP/1.1" 200 None 2026-03-31 11:14:13,126 - lava_callback - DEBUG - Processing node 69c909726a4c026e16348ee9: baseline-arm64-clabbe 2026-03-31 11:14:13,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c909726a4c026e16348ee9 2026-03-31 11:14:13,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:14:13,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c909726a4c026e16348ee9 HTTP/1.1" 200 None 2026-03-31 11:14:13,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:14:13,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:14:13,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:14:13,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c909726a4c026e16348ee9 HTTP/1.1" 200 None 2026-03-31 11:14:13,188 - lava_callback - ERROR - Error processing callback for node 69c909726a4c026e16348ee9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46656 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:15:14,760 - lava_callback - INFO - Received callback for node 69c909b86a4c026e16348ef7 INFO: 172.19.0.1:58230 - "POST /node/69c909b86a4c026e16348ef7 HTTP/1.0" 202 Accepted 2026-03-31 11:15:14,766 - lava_callback - INFO - Starting async job submit for node 69c909b86a4c026e16348ef7 2026-03-31 11:15:14,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:15:14,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c909b86a4c026e16348ef7 HTTP/1.1" 200 None 2026-03-31 11:15:14,784 - lava_callback - DEBUG - Processing node 69c909b86a4c026e16348ef7: baseline-arm64-baylibre 2026-03-31 11:15:14,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c909b86a4c026e16348ef7 2026-03-31 11:15:14,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:15:14,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c909b86a4c026e16348ef7 HTTP/1.1" 200 None 2026-03-31 11:15:14,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:15:14,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:15:14,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:15:14,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c909b86a4c026e16348ef7 HTTP/1.1" 200 None 2026-03-31 11:15:14,857 - lava_callback - ERROR - Error processing callback for node 69c909b86a4c026e16348ef7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52208 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:16:33,239 - lava_callback - INFO - Received callback for node 69c90a0a6a4c026e16348f09 INFO: 172.19.0.1:57062 - "POST /node/69c90a0a6a4c026e16348f09 HTTP/1.0" 202 Accepted 2026-03-31 11:16:33,244 - lava_callback - INFO - Starting async job submit for node 69c90a0a6a4c026e16348f09 2026-03-31 11:16:33,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:16:33,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a0a6a4c026e16348f09 HTTP/1.1" 200 None 2026-03-31 11:16:33,263 - lava_callback - DEBUG - Processing node 69c90a0a6a4c026e16348f09: baseline-arm64-clabbe 2026-03-31 11:16:33,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90a0a6a4c026e16348f09 2026-03-31 11:16:33,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:16:33,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a0a6a4c026e16348f09 HTTP/1.1" 200 None 2026-03-31 11:16:33,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:16:33,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:16:33,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:16:33,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90a0a6a4c026e16348f09 HTTP/1.1" 200 None 2026-03-31 11:16:33,316 - lava_callback - ERROR - Error processing callback for node 69c90a0a6a4c026e16348f09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52786 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:17:53,228 - lava_callback - INFO - Received callback for node 69c90a576a4c026e16348f16 INFO: 172.19.0.1:35206 - "POST /node/69c90a576a4c026e16348f16 HTTP/1.0" 202 Accepted 2026-03-31 11:17:53,234 - lava_callback - INFO - Starting async job submit for node 69c90a576a4c026e16348f16 2026-03-31 11:17:53,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:17:53,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a576a4c026e16348f16 HTTP/1.1" 200 None 2026-03-31 11:17:53,256 - lava_callback - DEBUG - Processing node 69c90a576a4c026e16348f16: baseline-arm64-clabbe 2026-03-31 11:17:53,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90a576a4c026e16348f16 2026-03-31 11:17:53,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:17:53,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a576a4c026e16348f16 HTTP/1.1" 200 None 2026-03-31 11:17:53,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:17:53,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:17:53,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:17:53,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90a576a4c026e16348f16 HTTP/1.1" 200 None 2026-03-31 11:17:53,317 - lava_callback - ERROR - Error processing callback for node 69c90a576a4c026e16348f16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54408 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:19:15,025 - lava_callback - INFO - Received callback for node 69c90a9f6a4c026e16348f20 INFO: 172.19.0.1:51910 - "POST /node/69c90a9f6a4c026e16348f20 HTTP/1.0" 202 Accepted 2026-03-31 11:19:15,030 - lava_callback - INFO - Starting async job submit for node 69c90a9f6a4c026e16348f20 2026-03-31 11:19:15,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:19:15,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a9f6a4c026e16348f20 HTTP/1.1" 200 None 2026-03-31 11:19:15,049 - lava_callback - DEBUG - Processing node 69c90a9f6a4c026e16348f20: baseline-arm64-baylibre 2026-03-31 11:19:15,050 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90a9f6a4c026e16348f20 2026-03-31 11:19:15,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:19:15,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90a9f6a4c026e16348f20 HTTP/1.1" 200 None 2026-03-31 11:19:15,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:19:15,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:19:15,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:19:15,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90a9f6a4c026e16348f20 HTTP/1.1" 200 None 2026-03-31 11:19:15,120 - lava_callback - ERROR - Error processing callback for node 69c90a9f6a4c026e16348f20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47164 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:20:33,327 - lava_callback - INFO - Received callback for node 69c90af06a4c026e16348f30 INFO: 172.19.0.1:49188 - "POST /node/69c90af06a4c026e16348f30 HTTP/1.0" 202 Accepted 2026-03-31 11:20:33,333 - lava_callback - INFO - Starting async job submit for node 69c90af06a4c026e16348f30 2026-03-31 11:20:33,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:20:33,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90af06a4c026e16348f30 HTTP/1.1" 200 None 2026-03-31 11:20:33,349 - lava_callback - DEBUG - Processing node 69c90af06a4c026e16348f30: baseline-arm64-clabbe 2026-03-31 11:20:33,349 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90af06a4c026e16348f30 2026-03-31 11:20:33,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:20:33,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90af06a4c026e16348f30 HTTP/1.1" 200 None 2026-03-31 11:20:33,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:20:33,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:20:33,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:20:33,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90af06a4c026e16348f30 HTTP/1.1" 200 None 2026-03-31 11:20:33,410 - lava_callback - ERROR - Error processing callback for node 69c90af06a4c026e16348f30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46150 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:21:53,400 - lava_callback - INFO - Received callback for node 69c90b3d6a4c026e16348f3c INFO: 172.19.0.1:58520 - "POST /node/69c90b3d6a4c026e16348f3c HTTP/1.0" 202 Accepted 2026-03-31 11:21:53,405 - lava_callback - INFO - Starting async job submit for node 69c90b3d6a4c026e16348f3c 2026-03-31 11:21:53,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:21:53,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90b3d6a4c026e16348f3c HTTP/1.1" 200 None 2026-03-31 11:21:53,444 - lava_callback - DEBUG - Processing node 69c90b3d6a4c026e16348f3c: baseline-arm64-clabbe 2026-03-31 11:21:53,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90b3d6a4c026e16348f3c 2026-03-31 11:21:53,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:21:53,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90b3d6a4c026e16348f3c HTTP/1.1" 200 None 2026-03-31 11:21:53,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:21:53,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:21:53,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:21:53,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90b3d6a4c026e16348f3c HTTP/1.1" 200 None 2026-03-31 11:21:53,525 - lava_callback - ERROR - Error processing callback for node 69c90b3d6a4c026e16348f3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42722 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:23:13,628 - lava_callback - INFO - Received callback for node 69c90b8b6a4c026e16348f4b INFO: 172.19.0.1:33052 - "POST /node/69c90b8b6a4c026e16348f4b HTTP/1.0" 202 Accepted 2026-03-31 11:23:13,634 - lava_callback - INFO - Starting async job submit for node 69c90b8b6a4c026e16348f4b 2026-03-31 11:23:13,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:23:13,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90b8b6a4c026e16348f4b HTTP/1.1" 200 None 2026-03-31 11:23:13,664 - lava_callback - DEBUG - Processing node 69c90b8b6a4c026e16348f4b: baseline-arm64-clabbe 2026-03-31 11:23:13,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90b8b6a4c026e16348f4b 2026-03-31 11:23:13,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:23:13,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90b8b6a4c026e16348f4b HTTP/1.1" 200 None 2026-03-31 11:23:13,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:23:13,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:23:13,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:23:13,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90b8b6a4c026e16348f4b HTTP/1.1" 200 None 2026-03-31 11:23:13,755 - lava_callback - ERROR - Error processing callback for node 69c90b8b6a4c026e16348f4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47538 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:24:13,454 - lava_callback - INFO - Received callback for node 69c90bd86a4c026e16348f57 INFO: 172.19.0.1:41740 - "POST /node/69c90bd86a4c026e16348f57 HTTP/1.0" 202 Accepted 2026-03-31 11:24:13,459 - lava_callback - INFO - Starting async job submit for node 69c90bd86a4c026e16348f57 2026-03-31 11:24:13,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:24:13,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90bd86a4c026e16348f57 HTTP/1.1" 200 None 2026-03-31 11:24:13,476 - lava_callback - DEBUG - Processing node 69c90bd86a4c026e16348f57: baseline-arm64-clabbe 2026-03-31 11:24:13,476 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90bd86a4c026e16348f57 2026-03-31 11:24:13,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:24:13,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90bd86a4c026e16348f57 HTTP/1.1" 200 None 2026-03-31 11:24:13,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:24:13,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:24:13,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:24:13,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90bd86a4c026e16348f57 HTTP/1.1" 200 None 2026-03-31 11:24:13,536 - lava_callback - ERROR - Error processing callback for node 69c90bd86a4c026e16348f57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46824 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:25:33,573 - lava_callback - INFO - Received callback for node 69c90c256a4c026e16348f61 INFO: 172.19.0.1:38340 - "POST /node/69c90c256a4c026e16348f61 HTTP/1.0" 202 Accepted 2026-03-31 11:25:33,578 - lava_callback - INFO - Starting async job submit for node 69c90c256a4c026e16348f61 2026-03-31 11:25:33,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:25:33,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90c256a4c026e16348f61 HTTP/1.1" 200 None 2026-03-31 11:25:33,599 - lava_callback - DEBUG - Processing node 69c90c256a4c026e16348f61: baseline-arm64-clabbe 2026-03-31 11:25:33,599 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90c256a4c026e16348f61 2026-03-31 11:25:33,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:25:33,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90c256a4c026e16348f61 HTTP/1.1" 200 None 2026-03-31 11:25:33,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:25:33,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:25:33,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:25:33,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90c256a4c026e16348f61 HTTP/1.1" 200 None 2026-03-31 11:25:33,654 - lava_callback - ERROR - Error processing callback for node 69c90c256a4c026e16348f61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51936 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:26:53,627 - lava_callback - INFO - Received callback for node 69c90c726a4c026e16348f6f INFO: 172.19.0.1:37954 - "POST /node/69c90c726a4c026e16348f6f HTTP/1.0" 202 Accepted 2026-03-31 11:26:53,634 - lava_callback - INFO - Starting async job submit for node 69c90c726a4c026e16348f6f 2026-03-31 11:26:53,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:26:53,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90c726a4c026e16348f6f HTTP/1.1" 200 None 2026-03-31 11:26:53,652 - lava_callback - DEBUG - Processing node 69c90c726a4c026e16348f6f: baseline-arm64-clabbe 2026-03-31 11:26:53,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90c726a4c026e16348f6f 2026-03-31 11:26:53,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:26:53,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90c726a4c026e16348f6f HTTP/1.1" 200 None 2026-03-31 11:26:53,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:26:53,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:26:53,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:26:53,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90c726a4c026e16348f6f HTTP/1.1" 200 None 2026-03-31 11:26:53,728 - lava_callback - ERROR - Error processing callback for node 69c90c726a4c026e16348f6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60536 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:28:13,641 - lava_callback - INFO - Received callback for node 69c90cbf6a4c026e16348f78 INFO: 172.19.0.1:37832 - "POST /node/69c90cbf6a4c026e16348f78 HTTP/1.0" 202 Accepted 2026-03-31 11:28:13,647 - lava_callback - INFO - Starting async job submit for node 69c90cbf6a4c026e16348f78 2026-03-31 11:28:13,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:28:13,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90cbf6a4c026e16348f78 HTTP/1.1" 200 None 2026-03-31 11:28:13,663 - lava_callback - DEBUG - Processing node 69c90cbf6a4c026e16348f78: baseline-arm64-clabbe 2026-03-31 11:28:13,663 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90cbf6a4c026e16348f78 2026-03-31 11:28:13,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:28:13,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90cbf6a4c026e16348f78 HTTP/1.1" 200 None 2026-03-31 11:28:13,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:28:13,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:28:13,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:28:13,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90cbf6a4c026e16348f78 HTTP/1.1" 200 None 2026-03-31 11:28:13,725 - lava_callback - ERROR - Error processing callback for node 69c90cbf6a4c026e16348f78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38100 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:29:33,667 - lava_callback - INFO - Received callback for node 69c90d0a6a4c026e16348f86 INFO: 172.19.0.1:46382 - "POST /node/69c90d0a6a4c026e16348f86 HTTP/1.0" 202 Accepted 2026-03-31 11:29:33,673 - lava_callback - INFO - Starting async job submit for node 69c90d0a6a4c026e16348f86 2026-03-31 11:29:33,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:29:33,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90d0a6a4c026e16348f86 HTTP/1.1" 200 None 2026-03-31 11:29:33,697 - lava_callback - DEBUG - Processing node 69c90d0a6a4c026e16348f86: baseline-arm64-clabbe 2026-03-31 11:29:33,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90d0a6a4c026e16348f86 2026-03-31 11:29:33,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:29:33,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90d0a6a4c026e16348f86 HTTP/1.1" 200 None 2026-03-31 11:29:33,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:29:33,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:29:33,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:29:33,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90d0a6a4c026e16348f86 HTTP/1.1" 200 None 2026-03-31 11:29:33,761 - lava_callback - ERROR - Error processing callback for node 69c90d0a6a4c026e16348f86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48708 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:30:53,720 - lava_callback - INFO - Received callback for node 69c90d586a4c026e16348f8f INFO: 172.19.0.1:54524 - "POST /node/69c90d586a4c026e16348f8f HTTP/1.0" 202 Accepted 2026-03-31 11:30:53,725 - lava_callback - INFO - Starting async job submit for node 69c90d586a4c026e16348f8f 2026-03-31 11:30:53,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:30:53,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90d586a4c026e16348f8f HTTP/1.1" 200 None 2026-03-31 11:30:53,744 - lava_callback - DEBUG - Processing node 69c90d586a4c026e16348f8f: baseline-arm64-clabbe 2026-03-31 11:30:53,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90d586a4c026e16348f8f 2026-03-31 11:30:53,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:30:53,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90d586a4c026e16348f8f HTTP/1.1" 200 None 2026-03-31 11:30:53,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:30:53,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:30:53,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:30:53,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90d586a4c026e16348f8f HTTP/1.1" 200 None 2026-03-31 11:30:53,819 - lava_callback - ERROR - Error processing callback for node 69c90d586a4c026e16348f8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57186 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:31:53,758 - lava_callback - INFO - Received callback for node 69c90da56a4c026e16348f9e INFO: 172.19.0.1:45350 - "POST /node/69c90da56a4c026e16348f9e HTTP/1.0" 202 Accepted 2026-03-31 11:31:53,763 - lava_callback - INFO - Starting async job submit for node 69c90da56a4c026e16348f9e 2026-03-31 11:31:53,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:31:53,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90da56a4c026e16348f9e HTTP/1.1" 200 None 2026-03-31 11:31:53,780 - lava_callback - DEBUG - Processing node 69c90da56a4c026e16348f9e: baseline-arm64-clabbe 2026-03-31 11:31:53,780 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90da56a4c026e16348f9e 2026-03-31 11:31:53,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:31:53,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90da56a4c026e16348f9e HTTP/1.1" 200 None 2026-03-31 11:31:53,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:31:53,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:31:53,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:31:53,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90da56a4c026e16348f9e HTTP/1.1" 200 None 2026-03-31 11:31:53,857 - lava_callback - ERROR - Error processing callback for node 69c90da56a4c026e16348f9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39772 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:33:13,812 - lava_callback - INFO - Received callback for node 69c90df26a4c026e16348faa INFO: 172.19.0.1:41602 - "POST /node/69c90df26a4c026e16348faa HTTP/1.0" 202 Accepted 2026-03-31 11:33:13,817 - lava_callback - INFO - Starting async job submit for node 69c90df26a4c026e16348faa 2026-03-31 11:33:13,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:33:13,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90df26a4c026e16348faa HTTP/1.1" 200 None 2026-03-31 11:33:13,833 - lava_callback - DEBUG - Processing node 69c90df26a4c026e16348faa: baseline-arm64-clabbe 2026-03-31 11:33:13,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90df26a4c026e16348faa 2026-03-31 11:33:13,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:33:13,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90df26a4c026e16348faa HTTP/1.1" 200 None 2026-03-31 11:33:13,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:33:13,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:33:13,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:33:13,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90df26a4c026e16348faa HTTP/1.1" 200 None 2026-03-31 11:33:13,894 - lava_callback - ERROR - Error processing callback for node 69c90df26a4c026e16348faa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43708 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:34:33,874 - lava_callback - INFO - Received callback for node 69c90e3c6a4c026e16348fb7 INFO: 172.19.0.1:39752 - "POST /node/69c90e3c6a4c026e16348fb7 HTTP/1.0" 202 Accepted 2026-03-31 11:34:33,883 - lava_callback - INFO - Starting async job submit for node 69c90e3c6a4c026e16348fb7 2026-03-31 11:34:33,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:34:33,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90e3c6a4c026e16348fb7 HTTP/1.1" 200 None 2026-03-31 11:34:33,900 - lava_callback - DEBUG - Processing node 69c90e3c6a4c026e16348fb7: baseline-arm64-clabbe 2026-03-31 11:34:33,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90e3c6a4c026e16348fb7 2026-03-31 11:34:33,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:34:33,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90e3c6a4c026e16348fb7 HTTP/1.1" 200 None 2026-03-31 11:34:33,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:34:33,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:34:33,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:34:33,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90e3c6a4c026e16348fb7 HTTP/1.1" 200 None 2026-03-31 11:34:33,961 - lava_callback - ERROR - Error processing callback for node 69c90e3c6a4c026e16348fb7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52792 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:35:53,930 - lava_callback - INFO - Received callback for node 69c90e886a4c026e16348fc5 INFO: 172.19.0.1:54156 - "POST /node/69c90e886a4c026e16348fc5 HTTP/1.0" 202 Accepted 2026-03-31 11:35:53,935 - lava_callback - INFO - Starting async job submit for node 69c90e886a4c026e16348fc5 2026-03-31 11:35:53,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:35:53,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90e886a4c026e16348fc5 HTTP/1.1" 200 None 2026-03-31 11:35:53,952 - lava_callback - DEBUG - Processing node 69c90e886a4c026e16348fc5: baseline-arm64-clabbe 2026-03-31 11:35:53,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90e886a4c026e16348fc5 2026-03-31 11:35:53,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:35:53,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90e886a4c026e16348fc5 HTTP/1.1" 200 None 2026-03-31 11:35:53,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:35:53,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:35:53,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:35:54,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90e886a4c026e16348fc5 HTTP/1.1" 200 None 2026-03-31 11:35:54,009 - lava_callback - ERROR - Error processing callback for node 69c90e886a4c026e16348fc5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56350 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:36:55,917 - lava_callback - INFO - Received callback for node 69c90ecf6a4c026e16348fcf INFO: 172.19.0.1:58672 - "POST /node/69c90ecf6a4c026e16348fcf HTTP/1.0" 202 Accepted 2026-03-31 11:36:55,922 - lava_callback - INFO - Starting async job submit for node 69c90ecf6a4c026e16348fcf 2026-03-31 11:36:55,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:36:55,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90ecf6a4c026e16348fcf HTTP/1.1" 200 None 2026-03-31 11:36:55,938 - lava_callback - DEBUG - Processing node 69c90ecf6a4c026e16348fcf: baseline-arm64-baylibre 2026-03-31 11:36:55,939 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90ecf6a4c026e16348fcf 2026-03-31 11:36:55,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:36:55,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90ecf6a4c026e16348fcf HTTP/1.1" 200 None 2026-03-31 11:36:55,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:36:55,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:36:55,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:36:55,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90ecf6a4c026e16348fcf HTTP/1.1" 200 None 2026-03-31 11:36:56,000 - lava_callback - ERROR - Error processing callback for node 69c90ecf6a4c026e16348fcf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52772 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:38:14,042 - lava_callback - INFO - Received callback for node 69c90f206a4c026e16348fde INFO: 172.19.0.1:33634 - "POST /node/69c90f206a4c026e16348fde HTTP/1.0" 202 Accepted 2026-03-31 11:38:14,049 - lava_callback - INFO - Starting async job submit for node 69c90f206a4c026e16348fde 2026-03-31 11:38:14,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:38:14,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90f206a4c026e16348fde HTTP/1.1" 200 None 2026-03-31 11:38:14,091 - lava_callback - DEBUG - Processing node 69c90f206a4c026e16348fde: baseline-arm64-clabbe 2026-03-31 11:38:14,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90f206a4c026e16348fde 2026-03-31 11:38:14,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:38:14,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90f206a4c026e16348fde HTTP/1.1" 200 None 2026-03-31 11:38:14,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:38:14,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:38:14,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:38:14,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90f206a4c026e16348fde HTTP/1.1" 200 None 2026-03-31 11:38:14,152 - lava_callback - ERROR - Error processing callback for node 69c90f206a4c026e16348fde: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56960 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57232 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:39:34,079 - lava_callback - INFO - Received callback for node 69c90f6c6a4c026e16348fe9 INFO: 172.19.0.1:40122 - "POST /node/69c90f6c6a4c026e16348fe9 HTTP/1.0" 202 Accepted 2026-03-31 11:39:34,089 - lava_callback - INFO - Starting async job submit for node 69c90f6c6a4c026e16348fe9 2026-03-31 11:39:34,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:39:34,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90f6c6a4c026e16348fe9 HTTP/1.1" 200 None 2026-03-31 11:39:34,114 - lava_callback - DEBUG - Processing node 69c90f6c6a4c026e16348fe9: baseline-arm64-clabbe 2026-03-31 11:39:34,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90f6c6a4c026e16348fe9 2026-03-31 11:39:34,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:39:34,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90f6c6a4c026e16348fe9 HTTP/1.1" 200 None 2026-03-31 11:39:34,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:39:34,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:39:34,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:39:34,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90f6c6a4c026e16348fe9 HTTP/1.1" 200 None 2026-03-31 11:39:34,190 - lava_callback - ERROR - Error processing callback for node 69c90f6c6a4c026e16348fe9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34848 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:40:54,177 - lava_callback - INFO - Received callback for node 69c90fb86a4c026e16348ff9 INFO: 172.19.0.1:45438 - "POST /node/69c90fb86a4c026e16348ff9 HTTP/1.0" 202 Accepted 2026-03-31 11:40:54,183 - lava_callback - INFO - Starting async job submit for node 69c90fb86a4c026e16348ff9 2026-03-31 11:40:54,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:40:54,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90fb86a4c026e16348ff9 HTTP/1.1" 200 None 2026-03-31 11:40:54,199 - lava_callback - DEBUG - Processing node 69c90fb86a4c026e16348ff9: baseline-arm64-clabbe 2026-03-31 11:40:54,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c90fb86a4c026e16348ff9 2026-03-31 11:40:54,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:40:54,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c90fb86a4c026e16348ff9 HTTP/1.1" 200 None 2026-03-31 11:40:54,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:40:54,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:40:54,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:40:54,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c90fb86a4c026e16348ff9 HTTP/1.1" 200 None 2026-03-31 11:40:54,251 - lava_callback - ERROR - Error processing callback for node 69c90fb86a4c026e16348ff9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51628 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:42:14,200 - lava_callback - INFO - Received callback for node 69c910026a4c026e16349003 INFO: 172.19.0.1:51518 - "POST /node/69c910026a4c026e16349003 HTTP/1.0" 202 Accepted 2026-03-31 11:42:14,209 - lava_callback - INFO - Starting async job submit for node 69c910026a4c026e16349003 2026-03-31 11:42:14,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:42:14,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910026a4c026e16349003 HTTP/1.1" 200 None 2026-03-31 11:42:14,225 - lava_callback - DEBUG - Processing node 69c910026a4c026e16349003: baseline-arm64-clabbe 2026-03-31 11:42:14,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c910026a4c026e16349003 2026-03-31 11:42:14,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:42:14,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910026a4c026e16349003 HTTP/1.1" 200 None 2026-03-31 11:42:14,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:42:14,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-31 11:42:14,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:42:14,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c910026a4c026e16349003 HTTP/1.1" 200 None 2026-03-31 11:42:14,300 - lava_callback - ERROR - Error processing callback for node 69c910026a4c026e16349003: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52650 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:43:14,200 - lava_callback - INFO - Received callback for node 69c9104d6a4c026e16349010 INFO: 172.19.0.1:47674 - "POST /node/69c9104d6a4c026e16349010 HTTP/1.0" 202 Accepted 2026-03-31 11:43:14,208 - lava_callback - INFO - Starting async job submit for node 69c9104d6a4c026e16349010 2026-03-31 11:43:14,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:43:14,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9104d6a4c026e16349010 HTTP/1.1" 200 None 2026-03-31 11:43:14,225 - lava_callback - DEBUG - Processing node 69c9104d6a4c026e16349010: baseline-arm64-clabbe 2026-03-31 11:43:14,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9104d6a4c026e16349010 2026-03-31 11:43:14,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:43:14,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9104d6a4c026e16349010 HTTP/1.1" 200 None 2026-03-31 11:43:14,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:43:14,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:43:14,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:43:14,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9104d6a4c026e16349010 HTTP/1.1" 200 None 2026-03-31 11:43:14,280 - lava_callback - ERROR - Error processing callback for node 69c9104d6a4c026e16349010: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36052 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:44:34,255 - lava_callback - INFO - Received callback for node 69c910986a4c026e1634901b INFO: 172.19.0.1:33056 - "POST /node/69c910986a4c026e1634901b HTTP/1.0" 202 Accepted 2026-03-31 11:44:34,260 - lava_callback - INFO - Starting async job submit for node 69c910986a4c026e1634901b 2026-03-31 11:44:34,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:44:34,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910986a4c026e1634901b HTTP/1.1" 200 None 2026-03-31 11:44:34,278 - lava_callback - DEBUG - Processing node 69c910986a4c026e1634901b: baseline-arm64-clabbe 2026-03-31 11:44:34,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c910986a4c026e1634901b 2026-03-31 11:44:34,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:44:34,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910986a4c026e1634901b HTTP/1.1" 200 None 2026-03-31 11:44:34,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:44:34,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:44:34,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:44:34,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c910986a4c026e1634901b HTTP/1.1" 200 None 2026-03-31 11:44:34,344 - lava_callback - ERROR - Error processing callback for node 69c910986a4c026e1634901b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37616 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:45:54,322 - lava_callback - INFO - Received callback for node 69c910e46a4c026e1634902a INFO: 172.19.0.1:55956 - "POST /node/69c910e46a4c026e1634902a HTTP/1.0" 202 Accepted 2026-03-31 11:45:54,328 - lava_callback - INFO - Starting async job submit for node 69c910e46a4c026e1634902a 2026-03-31 11:45:54,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:45:54,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910e46a4c026e1634902a HTTP/1.1" 200 None 2026-03-31 11:45:54,343 - lava_callback - DEBUG - Processing node 69c910e46a4c026e1634902a: baseline-arm64-clabbe 2026-03-31 11:45:54,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c910e46a4c026e1634902a 2026-03-31 11:45:54,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:45:54,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c910e46a4c026e1634902a HTTP/1.1" 200 None 2026-03-31 11:45:54,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:45:54,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:45:54,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:45:54,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c910e46a4c026e1634902a HTTP/1.1" 200 None 2026-03-31 11:45:54,408 - lava_callback - ERROR - Error processing callback for node 69c910e46a4c026e1634902a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34192 - "GET / HTTP/1.0" 200 OK 2026-03-31 11:47:14,409 - lava_callback - INFO - Received callback for node 69c911306a4c026e16349034 INFO: 172.19.0.1:49590 - "POST /node/69c911306a4c026e16349034 HTTP/1.0" 202 Accepted 2026-03-31 11:47:14,415 - lava_callback - INFO - Starting async job submit for node 69c911306a4c026e16349034 2026-03-31 11:47:14,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:47:14,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c911306a4c026e16349034 HTTP/1.1" 200 None 2026-03-31 11:47:14,431 - lava_callback - DEBUG - Processing node 69c911306a4c026e16349034: baseline-arm64-clabbe 2026-03-31 11:47:14,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c911306a4c026e16349034 2026-03-31 11:47:14,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:47:14,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c911306a4c026e16349034 HTTP/1.1" 200 None 2026-03-31 11:47:14,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:47:14,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-31 11:47:14,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 11:47:14,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c911306a4c026e16349034 HTTP/1.1" 200 None 2026-03-31 11:47:14,491 - lava_callback - ERROR - Error processing callback for node 69c911306a4c026e16349034: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42716 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:02:28,469 - lava_callback - INFO - Received callback for node 69cb86a72a64fa9aff936f0c INFO: 172.19.0.1:42720 - "POST /node/69cb86a72a64fa9aff936f0c HTTP/1.0" 202 Accepted 2026-03-31 12:02:28,475 - lava_callback - INFO - Starting async job submit for node 69cb86a72a64fa9aff936f0c 2026-03-31 12:02:28,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:02:28,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a72a64fa9aff936f0c HTTP/1.1" 200 None 2026-03-31 12:02:28,530 - lava_callback - DEBUG - Processing node 69cb86a72a64fa9aff936f0c: baseline-x86 2026-03-31 12:02:28,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 12:02:28,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 12:02:28,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a72a64fa9aff936f0c/log.txt.gz 2026-03-31 12:02:29,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 12:02:29,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 12:02:29,377 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a72a64fa9aff936f0c/lava-logs.yaml 2026-03-31 12:02:29,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 12:02:29,532 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 12:02:29,534 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cb86a72a64fa9aff936f0c/lava_callback.json.gz 2026-03-31 12:02:29,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:02:29,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb86a72a64fa9aff936f0c HTTP/1.1" 200 None 2026-03-31 12:02:29,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:02:29,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83602a64fa9aff936925 HTTP/1.1" 200 None 2026-03-31 12:02:29,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:02:29,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb86a72a64fa9aff936f0c HTTP/1.1" 200 None 2026-03-31 12:02:29,616 - lava_callback - INFO - Completed processing callback for node 69cb86a72a64fa9aff936f0c INFO: 172.19.0.1:34026 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:02:39,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:02:39,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:60410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48604 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59588 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:36:39,221 - lava_callback - INFO - Received callback for node 69c91cce6a4c026e1634906b INFO: 172.19.0.1:59590 - "POST /node/69c91cce6a4c026e1634906b HTTP/1.0" 202 Accepted 2026-03-31 12:36:39,227 - lava_callback - INFO - Starting async job submit for node 69c91cce6a4c026e1634906b 2026-03-31 12:36:39,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:36:39,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91cce6a4c026e1634906b HTTP/1.1" 200 None 2026-03-31 12:36:39,245 - lava_callback - DEBUG - Processing node 69c91cce6a4c026e1634906b: baseline-arm-baylibre 2026-03-31 12:36:39,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91cce6a4c026e1634906b 2026-03-31 12:36:39,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:36:39,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91cce6a4c026e1634906b HTTP/1.1" 200 None 2026-03-31 12:36:39,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:36:39,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:36:39,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:36:39,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91cce6a4c026e1634906b HTTP/1.1" 200 None 2026-03-31 12:36:39,325 - lava_callback - ERROR - Error processing callback for node 69c91cce6a4c026e1634906b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36470 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:37:19,149 - lava_callback - INFO - Received callback for node 69c91cf36a4c026e1634907a INFO: 172.19.0.1:36482 - "POST /node/69c91cf36a4c026e1634907a HTTP/1.0" 202 Accepted 2026-03-31 12:37:19,155 - lava_callback - INFO - Starting async job submit for node 69c91cf36a4c026e1634907a 2026-03-31 12:37:19,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:19,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91cf36a4c026e1634907a HTTP/1.1" 200 None 2026-03-31 12:37:19,171 - lava_callback - DEBUG - Processing node 69c91cf36a4c026e1634907a: baseline-arm-baylibre 2026-03-31 12:37:19,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91cf36a4c026e1634907a 2026-03-31 12:37:19,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:19,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91cf36a4c026e1634907a HTTP/1.1" 200 None 2026-03-31 12:37:19,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:19,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:37:19,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:19,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91cf36a4c026e1634907a HTTP/1.1" 200 None 2026-03-31 12:37:19,243 - lava_callback - ERROR - Error processing callback for node 69c91cf36a4c026e1634907a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53386 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:37:59,265 - lava_callback - INFO - Received callback for node 69c91d176a4c026e16349087 INFO: 172.19.0.1:53390 - "POST /node/69c91d176a4c026e16349087 HTTP/1.0" 202 Accepted 2026-03-31 12:37:59,270 - lava_callback - INFO - Starting async job submit for node 69c91d176a4c026e16349087 2026-03-31 12:37:59,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:59,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d176a4c026e16349087 HTTP/1.1" 200 None 2026-03-31 12:37:59,287 - lava_callback - DEBUG - Processing node 69c91d176a4c026e16349087: baseline-arm-baylibre 2026-03-31 12:37:59,288 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91d176a4c026e16349087 2026-03-31 12:37:59,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:59,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d176a4c026e16349087 HTTP/1.1" 200 None 2026-03-31 12:37:59,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:59,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:37:59,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:37:59,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91d176a4c026e16349087 HTTP/1.1" 200 None 2026-03-31 12:37:59,351 - lava_callback - ERROR - Error processing callback for node 69c91d176a4c026e16349087: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51062 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:38:39,245 - lava_callback - INFO - Received callback for node 69c91d3c6a4c026e16349090 INFO: 172.19.0.1:51074 - "POST /node/69c91d3c6a4c026e16349090 HTTP/1.0" 202 Accepted 2026-03-31 12:38:39,254 - lava_callback - INFO - Starting async job submit for node 69c91d3c6a4c026e16349090 2026-03-31 12:38:39,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:38:39,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d3c6a4c026e16349090 HTTP/1.1" 200 None 2026-03-31 12:38:39,273 - lava_callback - DEBUG - Processing node 69c91d3c6a4c026e16349090: baseline-arm-baylibre 2026-03-31 12:38:39,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91d3c6a4c026e16349090 2026-03-31 12:38:39,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:38:39,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d3c6a4c026e16349090 HTTP/1.1" 200 None 2026-03-31 12:38:39,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:38:39,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:38:39,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:38:39,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91d3c6a4c026e16349090 HTTP/1.1" 200 None 2026-03-31 12:38:39,334 - lava_callback - ERROR - Error processing callback for node 69c91d3c6a4c026e16349090: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38820 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:39:19,305 - lava_callback - INFO - Received callback for node 69c91d646a4c026e1634909b INFO: 172.19.0.1:38830 - "POST /node/69c91d646a4c026e1634909b HTTP/1.0" 202 Accepted 2026-03-31 12:39:19,310 - lava_callback - INFO - Starting async job submit for node 69c91d646a4c026e1634909b 2026-03-31 12:39:19,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:19,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d646a4c026e1634909b HTTP/1.1" 200 None 2026-03-31 12:39:19,326 - lava_callback - DEBUG - Processing node 69c91d646a4c026e1634909b: baseline-arm-baylibre 2026-03-31 12:39:19,326 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91d646a4c026e1634909b 2026-03-31 12:39:19,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:19,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d646a4c026e1634909b HTTP/1.1" 200 None 2026-03-31 12:39:19,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:19,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:39:19,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:19,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91d646a4c026e1634909b HTTP/1.1" 200 None 2026-03-31 12:39:19,386 - lava_callback - ERROR - Error processing callback for node 69c91d646a4c026e1634909b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33012 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:39:39,305 - lava_callback - INFO - Received callback for node 69c91d886a4c026e163490ad INFO: 172.19.0.1:33020 - "POST /node/69c91d886a4c026e163490ad HTTP/1.0" 202 Accepted 2026-03-31 12:39:39,311 - lava_callback - INFO - Starting async job submit for node 69c91d886a4c026e163490ad 2026-03-31 12:39:39,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:39,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d886a4c026e163490ad HTTP/1.1" 200 None 2026-03-31 12:39:39,327 - lava_callback - DEBUG - Processing node 69c91d886a4c026e163490ad: baseline-arm-baylibre 2026-03-31 12:39:39,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91d886a4c026e163490ad 2026-03-31 12:39:39,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:39,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91d886a4c026e163490ad HTTP/1.1" 200 None 2026-03-31 12:39:39,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:39,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:39:39,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:39:39,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91d886a4c026e163490ad HTTP/1.1" 200 None 2026-03-31 12:39:39,410 - lava_callback - ERROR - Error processing callback for node 69c91d886a4c026e163490ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60332 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:40:19,322 - lava_callback - INFO - Received callback for node 69c91dad6a4c026e163490b4 INFO: 172.19.0.1:60338 - "POST /node/69c91dad6a4c026e163490b4 HTTP/1.0" 202 Accepted 2026-03-31 12:40:19,331 - lava_callback - INFO - Starting async job submit for node 69c91dad6a4c026e163490b4 2026-03-31 12:40:19,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:19,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91dad6a4c026e163490b4 HTTP/1.1" 200 None 2026-03-31 12:40:19,353 - lava_callback - DEBUG - Processing node 69c91dad6a4c026e163490b4: baseline-arm-baylibre 2026-03-31 12:40:19,353 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91dad6a4c026e163490b4 2026-03-31 12:40:19,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:19,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91dad6a4c026e163490b4 HTTP/1.1" 200 None 2026-03-31 12:40:19,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:19,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:40:19,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:19,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91dad6a4c026e163490b4 HTTP/1.1" 200 None 2026-03-31 12:40:19,468 - lava_callback - ERROR - Error processing callback for node 69c91dad6a4c026e163490b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50104 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:40:59,390 - lava_callback - INFO - Received callback for node 69c91dd36a4c026e163490c3 INFO: 172.19.0.1:50114 - "POST /node/69c91dd36a4c026e163490c3 HTTP/1.0" 202 Accepted 2026-03-31 12:40:59,395 - lava_callback - INFO - Starting async job submit for node 69c91dd36a4c026e163490c3 2026-03-31 12:40:59,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:59,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91dd36a4c026e163490c3 HTTP/1.1" 200 None 2026-03-31 12:40:59,412 - lava_callback - DEBUG - Processing node 69c91dd36a4c026e163490c3: baseline-arm-baylibre 2026-03-31 12:40:59,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91dd36a4c026e163490c3 2026-03-31 12:40:59,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:59,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91dd36a4c026e163490c3 HTTP/1.1" 200 None 2026-03-31 12:40:59,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:59,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:40:59,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:40:59,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91dd36a4c026e163490c3 HTTP/1.1" 200 None 2026-03-31 12:40:59,479 - lava_callback - ERROR - Error processing callback for node 69c91dd36a4c026e163490c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44688 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:41:36,961 - lava_callback - INFO - Received callback for node 69c91df96a4c026e163490d0 INFO: 172.19.0.1:44704 - "POST /node/69c91df96a4c026e163490d0 HTTP/1.0" 202 Accepted 2026-03-31 12:41:36,970 - lava_callback - INFO - Starting async job submit for node 69c91df96a4c026e163490d0 2026-03-31 12:41:36,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:41:36,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91df96a4c026e163490d0 HTTP/1.1" 200 None 2026-03-31 12:41:36,988 - lava_callback - DEBUG - Processing node 69c91df96a4c026e163490d0: baseline-arm-clabbe 2026-03-31 12:41:36,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91df96a4c026e163490d0 2026-03-31 12:41:36,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:41:37,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91df96a4c026e163490d0 HTTP/1.1" 200 None 2026-03-31 12:41:37,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:41:37,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:41:37,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:41:37,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91df96a4c026e163490d0 HTTP/1.1" 200 None 2026-03-31 12:41:37,046 - lava_callback - ERROR - Error processing callback for node 69c91df96a4c026e163490d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46570 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:42:19,548 - lava_callback - INFO - Received callback for node 69c91e1c6a4c026e163490dc INFO: 172.19.0.1:46582 - "POST /node/69c91e1c6a4c026e163490dc HTTP/1.0" 202 Accepted 2026-03-31 12:42:19,554 - lava_callback - INFO - Starting async job submit for node 69c91e1c6a4c026e163490dc 2026-03-31 12:42:19,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:19,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e1c6a4c026e163490dc HTTP/1.1" 200 None 2026-03-31 12:42:19,571 - lava_callback - DEBUG - Processing node 69c91e1c6a4c026e163490dc: baseline-arm-baylibre 2026-03-31 12:42:19,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91e1c6a4c026e163490dc 2026-03-31 12:42:19,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:19,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e1c6a4c026e163490dc HTTP/1.1" 200 None 2026-03-31 12:42:19,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:19,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:42:19,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:19,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91e1c6a4c026e163490dc HTTP/1.1" 200 None 2026-03-31 12:42:19,637 - lava_callback - ERROR - Error processing callback for node 69c91e1c6a4c026e163490dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43344 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:42:59,464 - lava_callback - INFO - Received callback for node 69c91e406a4c026e163490ed INFO: 172.19.0.1:43346 - "POST /node/69c91e406a4c026e163490ed HTTP/1.0" 202 Accepted 2026-03-31 12:42:59,470 - lava_callback - INFO - Starting async job submit for node 69c91e406a4c026e163490ed 2026-03-31 12:42:59,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:59,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e406a4c026e163490ed HTTP/1.1" 200 None 2026-03-31 12:42:59,486 - lava_callback - DEBUG - Processing node 69c91e406a4c026e163490ed: baseline-arm-baylibre 2026-03-31 12:42:59,486 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91e406a4c026e163490ed 2026-03-31 12:42:59,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:59,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e406a4c026e163490ed HTTP/1.1" 200 None 2026-03-31 12:42:59,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:59,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:42:59,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:42:59,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91e406a4c026e163490ed HTTP/1.1" 200 None 2026-03-31 12:42:59,537 - lava_callback - ERROR - Error processing callback for node 69c91e406a4c026e163490ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55140 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:43:19,494 - lava_callback - INFO - Received callback for node 69c91e646a4c026e163490f4 INFO: 172.19.0.1:55156 - "POST /node/69c91e646a4c026e163490f4 HTTP/1.0" 202 Accepted 2026-03-31 12:43:19,500 - lava_callback - INFO - Starting async job submit for node 69c91e646a4c026e163490f4 2026-03-31 12:43:19,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:19,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e646a4c026e163490f4 HTTP/1.1" 200 None 2026-03-31 12:43:19,538 - lava_callback - DEBUG - Processing node 69c91e646a4c026e163490f4: baseline-arm-baylibre 2026-03-31 12:43:19,539 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91e646a4c026e163490f4 2026-03-31 12:43:19,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:19,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e646a4c026e163490f4 HTTP/1.1" 200 None 2026-03-31 12:43:19,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:19,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:43:19,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:19,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91e646a4c026e163490f4 HTTP/1.1" 200 None 2026-03-31 12:43:19,600 - lava_callback - ERROR - Error processing callback for node 69c91e646a4c026e163490f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42336 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:43:59,560 - lava_callback - INFO - Received callback for node 69c91e886a4c026e16349106 INFO: 172.19.0.1:42352 - "POST /node/69c91e886a4c026e16349106 HTTP/1.0" 202 Accepted 2026-03-31 12:43:59,566 - lava_callback - INFO - Starting async job submit for node 69c91e886a4c026e16349106 2026-03-31 12:43:59,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:59,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e886a4c026e16349106 HTTP/1.1" 200 None 2026-03-31 12:43:59,583 - lava_callback - DEBUG - Processing node 69c91e886a4c026e16349106: baseline-arm-baylibre 2026-03-31 12:43:59,583 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91e886a4c026e16349106 2026-03-31 12:43:59,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:59,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91e886a4c026e16349106 HTTP/1.1" 200 None 2026-03-31 12:43:59,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:59,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:43:59,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:43:59,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91e886a4c026e16349106 HTTP/1.1" 200 None 2026-03-31 12:43:59,662 - lava_callback - ERROR - Error processing callback for node 69c91e886a4c026e16349106: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48734 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:44:37,132 - lava_callback - INFO - Received callback for node 69c91eae6a4c026e16349114 INFO: 172.19.0.1:48740 - "POST /node/69c91eae6a4c026e16349114 HTTP/1.0" 202 Accepted 2026-03-31 12:44:37,137 - lava_callback - INFO - Starting async job submit for node 69c91eae6a4c026e16349114 2026-03-31 12:44:37,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:44:37,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91eae6a4c026e16349114 HTTP/1.1" 200 None 2026-03-31 12:44:37,177 - lava_callback - DEBUG - Processing node 69c91eae6a4c026e16349114: baseline-arm-clabbe 2026-03-31 12:44:37,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91eae6a4c026e16349114 2026-03-31 12:44:37,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:44:37,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91eae6a4c026e16349114 HTTP/1.1" 200 None 2026-03-31 12:44:37,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:44:37,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:44:37,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:44:37,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91eae6a4c026e16349114 HTTP/1.1" 200 None 2026-03-31 12:44:37,408 - lava_callback - ERROR - Error processing callback for node 69c91eae6a4c026e16349114: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55364 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:45:19,627 - lava_callback - INFO - Received callback for node 69c91ed16a4c026e16349120 INFO: 172.19.0.1:55372 - "POST /node/69c91ed16a4c026e16349120 HTTP/1.0" 202 Accepted 2026-03-31 12:45:19,632 - lava_callback - INFO - Starting async job submit for node 69c91ed16a4c026e16349120 2026-03-31 12:45:19,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:45:19,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91ed16a4c026e16349120 HTTP/1.1" 200 None 2026-03-31 12:45:19,649 - lava_callback - DEBUG - Processing node 69c91ed16a4c026e16349120: baseline-arm-baylibre 2026-03-31 12:45:19,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91ed16a4c026e16349120 2026-03-31 12:45:19,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:45:19,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91ed16a4c026e16349120 HTTP/1.1" 200 None 2026-03-31 12:45:19,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:45:19,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:45:19,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:45:19,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91ed16a4c026e16349120 HTTP/1.1" 200 None 2026-03-31 12:45:19,725 - lava_callback - ERROR - Error processing callback for node 69c91ed16a4c026e16349120: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52112 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:47:16,937 - lava_callback - INFO - Received callback for node 69c91f426a4c026e16349149 INFO: 172.19.0.1:52128 - "POST /node/69c91f426a4c026e16349149 HTTP/1.0" 202 Accepted 2026-03-31 12:47:16,942 - lava_callback - INFO - Starting async job submit for node 69c91f426a4c026e16349149 2026-03-31 12:47:16,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:16,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91f426a4c026e16349149 HTTP/1.1" 200 None 2026-03-31 12:47:16,959 - lava_callback - DEBUG - Processing node 69c91f426a4c026e16349149: baseline-arm-clabbe 2026-03-31 12:47:16,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91f426a4c026e16349149 2026-03-31 12:47:16,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:16,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91f426a4c026e16349149 HTTP/1.1" 200 None 2026-03-31 12:47:16,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:16,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:47:16,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:17,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91f426a4c026e16349149 HTTP/1.1" 200 None 2026-03-31 12:47:17,013 - lava_callback - ERROR - Error processing callback for node 69c91f426a4c026e16349149: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46438 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:47:39,955 - lava_callback - INFO - Received callback for node 69c91f646a4c026e16349159 INFO: 172.19.0.1:50714 - "POST /node/69c91f646a4c026e16349159 HTTP/1.0" 202 Accepted 2026-03-31 12:47:39,960 - lava_callback - INFO - Starting async job submit for node 69c91f646a4c026e16349159 2026-03-31 12:47:39,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:39,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91f646a4c026e16349159 HTTP/1.1" 200 None 2026-03-31 12:47:39,978 - lava_callback - DEBUG - Processing node 69c91f646a4c026e16349159: baseline-arm-baylibre 2026-03-31 12:47:39,978 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91f646a4c026e16349159 2026-03-31 12:47:39,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:39,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91f646a4c026e16349159 HTTP/1.1" 200 None 2026-03-31 12:47:39,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:40,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:47:40,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:47:40,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91f646a4c026e16349159 HTTP/1.1" 200 None 2026-03-31 12:47:40,052 - lava_callback - ERROR - Error processing callback for node 69c91f646a4c026e16349159: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60886 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:48:59,843 - lava_callback - INFO - Received callback for node 69c91fac6a4c026e16349171 INFO: 172.19.0.1:34344 - "POST /node/69c91fac6a4c026e16349171 HTTP/1.0" 202 Accepted 2026-03-31 12:48:59,848 - lava_callback - INFO - Starting async job submit for node 69c91fac6a4c026e16349171 2026-03-31 12:48:59,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:48:59,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91fac6a4c026e16349171 HTTP/1.1" 200 None 2026-03-31 12:48:59,864 - lava_callback - DEBUG - Processing node 69c91fac6a4c026e16349171: baseline-arm-baylibre 2026-03-31 12:48:59,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91fac6a4c026e16349171 2026-03-31 12:48:59,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:48:59,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91fac6a4c026e16349171 HTTP/1.1" 200 None 2026-03-31 12:48:59,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:48:59,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:48:59,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:48:59,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91fac6a4c026e16349171 HTTP/1.1" 200 None 2026-03-31 12:48:59,924 - lava_callback - ERROR - Error processing callback for node 69c91fac6a4c026e16349171: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41204 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:49:36,968 - lava_callback - INFO - Received callback for node 69c91fd26a4c026e1634917c INFO: 172.19.0.1:41214 - "POST /node/69c91fd26a4c026e1634917c HTTP/1.0" 202 Accepted 2026-03-31 12:49:36,974 - lava_callback - INFO - Starting async job submit for node 69c91fd26a4c026e1634917c 2026-03-31 12:49:36,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:49:36,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91fd26a4c026e1634917c HTTP/1.1" 200 None 2026-03-31 12:49:36,991 - lava_callback - DEBUG - Processing node 69c91fd26a4c026e1634917c: baseline-arm-clabbe 2026-03-31 12:49:36,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91fd26a4c026e1634917c 2026-03-31 12:49:36,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:49:37,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91fd26a4c026e1634917c HTTP/1.1" 200 None 2026-03-31 12:49:37,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:49:37,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:49:37,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:49:37,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91fd26a4c026e1634917c HTTP/1.1" 200 None 2026-03-31 12:49:37,070 - lava_callback - ERROR - Error processing callback for node 69c91fd26a4c026e1634917c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:32838 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:50:16,915 - lava_callback - INFO - Received callback for node 69c91ff66a4c026e1634918e INFO: 172.19.0.1:32848 - "POST /node/69c91ff66a4c026e1634918e HTTP/1.0" 202 Accepted 2026-03-31 12:50:16,920 - lava_callback - INFO - Starting async job submit for node 69c91ff66a4c026e1634918e 2026-03-31 12:50:16,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:50:16,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91ff66a4c026e1634918e HTTP/1.1" 200 None 2026-03-31 12:50:16,936 - lava_callback - DEBUG - Processing node 69c91ff66a4c026e1634918e: baseline-arm-clabbe 2026-03-31 12:50:16,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c91ff66a4c026e1634918e 2026-03-31 12:50:16,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:50:16,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c91ff66a4c026e1634918e HTTP/1.1" 200 None 2026-03-31 12:50:16,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:50:16,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:50:16,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:50:16,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c91ff66a4c026e1634918e HTTP/1.1" 200 None 2026-03-31 12:50:16,989 - lava_callback - ERROR - Error processing callback for node 69c91ff66a4c026e1634918e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37464 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:51:57,264 - lava_callback - INFO - Received callback for node 69c920606a4c026e163491bc INFO: 172.19.0.1:37480 - "POST /node/69c920606a4c026e163491bc HTTP/1.0" 202 Accepted 2026-03-31 12:51:57,269 - lava_callback - INFO - Starting async job submit for node 69c920606a4c026e163491bc 2026-03-31 12:51:57,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:51:57,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920606a4c026e163491bc HTTP/1.1" 200 None 2026-03-31 12:51:57,285 - lava_callback - DEBUG - Processing node 69c920606a4c026e163491bc: baseline-arm-clabbe 2026-03-31 12:51:57,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c920606a4c026e163491bc 2026-03-31 12:51:57,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:51:57,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920606a4c026e163491bc HTTP/1.1" 200 None 2026-03-31 12:51:57,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:51:57,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:51:57,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:51:57,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c920606a4c026e163491bc HTTP/1.1" 200 None 2026-03-31 12:51:57,366 - lava_callback - ERROR - Error processing callback for node 69c920606a4c026e163491bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59328 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:52:37,132 - lava_callback - INFO - Received callback for node 69c920866a4c026e163491c7 INFO: 172.19.0.1:59336 - "POST /node/69c920866a4c026e163491c7 HTTP/1.0" 202 Accepted 2026-03-31 12:52:37,138 - lava_callback - INFO - Starting async job submit for node 69c920866a4c026e163491c7 2026-03-31 12:52:37,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:52:37,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920866a4c026e163491c7 HTTP/1.1" 200 None 2026-03-31 12:52:37,175 - lava_callback - DEBUG - Processing node 69c920866a4c026e163491c7: baseline-arm-clabbe 2026-03-31 12:52:37,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c920866a4c026e163491c7 2026-03-31 12:52:37,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:52:37,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920866a4c026e163491c7 HTTP/1.1" 200 None 2026-03-31 12:52:37,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:52:37,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:52:37,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:52:37,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c920866a4c026e163491c7 HTTP/1.1" 200 None 2026-03-31 12:52:37,226 - lava_callback - ERROR - Error processing callback for node 69c920866a4c026e163491c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34714 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:53:17,182 - lava_callback - INFO - Received callback for node 69c920ab6a4c026e163491da INFO: 172.19.0.1:34726 - "POST /node/69c920ab6a4c026e163491da HTTP/1.0" 202 Accepted 2026-03-31 12:53:17,188 - lava_callback - INFO - Starting async job submit for node 69c920ab6a4c026e163491da 2026-03-31 12:53:17,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:17,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920ab6a4c026e163491da HTTP/1.1" 200 None 2026-03-31 12:53:17,204 - lava_callback - DEBUG - Processing node 69c920ab6a4c026e163491da: baseline-arm-clabbe 2026-03-31 12:53:17,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c920ab6a4c026e163491da 2026-03-31 12:53:17,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:17,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920ab6a4c026e163491da HTTP/1.1" 200 None 2026-03-31 12:53:17,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:17,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:53:17,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:17,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c920ab6a4c026e163491da HTTP/1.1" 200 None 2026-03-31 12:53:17,266 - lava_callback - ERROR - Error processing callback for node 69c920ab6a4c026e163491da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35672 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:53:57,164 - lava_callback - INFO - Received callback for node 69c920d06a4c026e163491eb INFO: 172.19.0.1:35682 - "POST /node/69c920d06a4c026e163491eb HTTP/1.0" 202 Accepted 2026-03-31 12:53:57,170 - lava_callback - INFO - Starting async job submit for node 69c920d06a4c026e163491eb 2026-03-31 12:53:57,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:57,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920d06a4c026e163491eb HTTP/1.1" 200 None 2026-03-31 12:53:57,185 - lava_callback - DEBUG - Processing node 69c920d06a4c026e163491eb: baseline-arm-clabbe 2026-03-31 12:53:57,186 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c920d06a4c026e163491eb 2026-03-31 12:53:57,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:57,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920d06a4c026e163491eb HTTP/1.1" 200 None 2026-03-31 12:53:57,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:57,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:53:57,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:53:57,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c920d06a4c026e163491eb HTTP/1.1" 200 None 2026-03-31 12:53:57,244 - lava_callback - ERROR - Error processing callback for node 69c920d06a4c026e163491eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54074 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:54:17,220 - lava_callback - INFO - Received callback for node 69c920f56a4c026e163491f7 INFO: 172.19.0.1:54084 - "POST /node/69c920f56a4c026e163491f7 HTTP/1.0" 202 Accepted 2026-03-31 12:54:17,226 - lava_callback - INFO - Starting async job submit for node 69c920f56a4c026e163491f7 2026-03-31 12:54:17,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:17,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920f56a4c026e163491f7 HTTP/1.1" 200 None 2026-03-31 12:54:17,241 - lava_callback - DEBUG - Processing node 69c920f56a4c026e163491f7: baseline-arm-clabbe 2026-03-31 12:54:17,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c920f56a4c026e163491f7 2026-03-31 12:54:17,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:17,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c920f56a4c026e163491f7 HTTP/1.1" 200 None 2026-03-31 12:54:17,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:17,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:54:17,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:17,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c920f56a4c026e163491f7 HTTP/1.1" 200 None 2026-03-31 12:54:17,304 - lava_callback - ERROR - Error processing callback for node 69c920f56a4c026e163491f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35650 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:54:57,223 - lava_callback - INFO - Received callback for node 69c921196a4c026e16349203 INFO: 172.19.0.1:35654 - "POST /node/69c921196a4c026e16349203 HTTP/1.0" 202 Accepted 2026-03-31 12:54:57,228 - lava_callback - INFO - Starting async job submit for node 69c921196a4c026e16349203 2026-03-31 12:54:57,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:57,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921196a4c026e16349203 HTTP/1.1" 200 None 2026-03-31 12:54:57,245 - lava_callback - DEBUG - Processing node 69c921196a4c026e16349203: baseline-arm-clabbe 2026-03-31 12:54:57,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921196a4c026e16349203 2026-03-31 12:54:57,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:57,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921196a4c026e16349203 HTTP/1.1" 200 None 2026-03-31 12:54:57,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:57,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:54:57,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:54:57,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921196a4c026e16349203 HTTP/1.1" 200 None 2026-03-31 12:54:57,306 - lava_callback - ERROR - Error processing callback for node 69c921196a4c026e16349203: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59846 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:55:37,281 - lava_callback - INFO - Received callback for node 69c9213f6a4c026e1634920f INFO: 172.19.0.1:59862 - "POST /node/69c9213f6a4c026e1634920f HTTP/1.0" 202 Accepted 2026-03-31 12:55:37,287 - lava_callback - INFO - Starting async job submit for node 69c9213f6a4c026e1634920f 2026-03-31 12:55:37,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:55:37,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9213f6a4c026e1634920f HTTP/1.1" 200 None 2026-03-31 12:55:37,306 - lava_callback - DEBUG - Processing node 69c9213f6a4c026e1634920f: baseline-arm-clabbe 2026-03-31 12:55:37,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9213f6a4c026e1634920f 2026-03-31 12:55:37,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:55:37,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9213f6a4c026e1634920f HTTP/1.1" 200 None 2026-03-31 12:55:37,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:55:37,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 12:55:37,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:55:37,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9213f6a4c026e1634920f HTTP/1.1" 200 None 2026-03-31 12:55:37,376 - lava_callback - ERROR - Error processing callback for node 69c9213f6a4c026e1634920f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47300 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:56:17,279 - lava_callback - INFO - Received callback for node 69c921636a4c026e1634921e INFO: 172.19.0.1:47306 - "POST /node/69c921636a4c026e1634921e HTTP/1.0" 202 Accepted 2026-03-31 12:56:17,284 - lava_callback - INFO - Starting async job submit for node 69c921636a4c026e1634921e 2026-03-31 12:56:17,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:17,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921636a4c026e1634921e HTTP/1.1" 200 None 2026-03-31 12:56:17,300 - lava_callback - DEBUG - Processing node 69c921636a4c026e1634921e: baseline-arm-clabbe 2026-03-31 12:56:17,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921636a4c026e1634921e 2026-03-31 12:56:17,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:17,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921636a4c026e1634921e HTTP/1.1" 200 None 2026-03-31 12:56:17,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:17,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:56:17,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:17,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921636a4c026e1634921e HTTP/1.1" 200 None 2026-03-31 12:56:17,374 - lava_callback - ERROR - Error processing callback for node 69c921636a4c026e1634921e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56448 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:56:57,333 - lava_callback - INFO - Received callback for node 69c921886a4c026e1634922f INFO: 172.19.0.1:56458 - "POST /node/69c921886a4c026e1634922f HTTP/1.0" 202 Accepted 2026-03-31 12:56:57,339 - lava_callback - INFO - Starting async job submit for node 69c921886a4c026e1634922f 2026-03-31 12:56:57,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:57,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921886a4c026e1634922f HTTP/1.1" 200 None 2026-03-31 12:56:57,355 - lava_callback - DEBUG - Processing node 69c921886a4c026e1634922f: baseline-arm-clabbe 2026-03-31 12:56:57,355 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921886a4c026e1634922f 2026-03-31 12:56:57,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:57,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921886a4c026e1634922f HTTP/1.1" 200 None 2026-03-31 12:56:57,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:57,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:56:57,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:56:57,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921886a4c026e1634922f HTTP/1.1" 200 None 2026-03-31 12:56:57,423 - lava_callback - ERROR - Error processing callback for node 69c921886a4c026e1634922f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53510 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:57:37,325 - lava_callback - INFO - Received callback for node 69c921ad6a4c026e16349238 INFO: 172.19.0.1:53522 - "POST /node/69c921ad6a4c026e16349238 HTTP/1.0" 202 Accepted 2026-03-31 12:57:37,330 - lava_callback - INFO - Starting async job submit for node 69c921ad6a4c026e16349238 2026-03-31 12:57:37,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:37,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921ad6a4c026e16349238 HTTP/1.1" 200 None 2026-03-31 12:57:37,349 - lava_callback - DEBUG - Processing node 69c921ad6a4c026e16349238: baseline-arm-clabbe 2026-03-31 12:57:37,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921ad6a4c026e16349238 2026-03-31 12:57:37,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:37,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921ad6a4c026e16349238 HTTP/1.1" 200 None 2026-03-31 12:57:37,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:37,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:57:37,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:37,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921ad6a4c026e16349238 HTTP/1.1" 200 None 2026-03-31 12:57:37,406 - lava_callback - ERROR - Error processing callback for node 69c921ad6a4c026e16349238: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59390 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:57:57,363 - lava_callback - INFO - Received callback for node 69c921d26a4c026e16349245 INFO: 172.19.0.1:59394 - "POST /node/69c921d26a4c026e16349245 HTTP/1.0" 202 Accepted 2026-03-31 12:57:57,368 - lava_callback - INFO - Starting async job submit for node 69c921d26a4c026e16349245 2026-03-31 12:57:57,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:57,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921d26a4c026e16349245 HTTP/1.1" 200 None 2026-03-31 12:57:57,385 - lava_callback - DEBUG - Processing node 69c921d26a4c026e16349245: baseline-arm-clabbe 2026-03-31 12:57:57,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921d26a4c026e16349245 2026-03-31 12:57:57,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:57,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921d26a4c026e16349245 HTTP/1.1" 200 None 2026-03-31 12:57:57,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:57,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:57:57,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:57:57,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921d26a4c026e16349245 HTTP/1.1" 200 None 2026-03-31 12:57:57,453 - lava_callback - ERROR - Error processing callback for node 69c921d26a4c026e16349245: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38724 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:58:37,342 - lava_callback - INFO - Received callback for node 69c921f86a4c026e16349253 INFO: 172.19.0.1:38736 - "POST /node/69c921f86a4c026e16349253 HTTP/1.0" 202 Accepted 2026-03-31 12:58:37,348 - lava_callback - INFO - Starting async job submit for node 69c921f86a4c026e16349253 2026-03-31 12:58:37,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:58:37,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921f86a4c026e16349253 HTTP/1.1" 200 None 2026-03-31 12:58:37,364 - lava_callback - DEBUG - Processing node 69c921f86a4c026e16349253: baseline-arm-clabbe 2026-03-31 12:58:37,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c921f86a4c026e16349253 2026-03-31 12:58:37,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:58:37,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c921f86a4c026e16349253 HTTP/1.1" 200 None 2026-03-31 12:58:37,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:58:37,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:58:37,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:58:37,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c921f86a4c026e16349253 HTTP/1.1" 200 None 2026-03-31 12:58:37,421 - lava_callback - ERROR - Error processing callback for node 69c921f86a4c026e16349253: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49972 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:59:17,461 - lava_callback - INFO - Received callback for node 69c9221c6a4c026e16349261 INFO: 172.19.0.1:49980 - "POST /node/69c9221c6a4c026e16349261 HTTP/1.0" 202 Accepted 2026-03-31 12:59:17,466 - lava_callback - INFO - Starting async job submit for node 69c9221c6a4c026e16349261 2026-03-31 12:59:17,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:17,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9221c6a4c026e16349261 HTTP/1.1" 200 None 2026-03-31 12:59:17,482 - lava_callback - DEBUG - Processing node 69c9221c6a4c026e16349261: baseline-arm-clabbe 2026-03-31 12:59:17,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9221c6a4c026e16349261 2026-03-31 12:59:17,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:17,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9221c6a4c026e16349261 HTTP/1.1" 200 None 2026-03-31 12:59:17,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:17,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:59:17,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:17,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9221c6a4c026e16349261 HTTP/1.1" 200 None 2026-03-31 12:59:17,538 - lava_callback - ERROR - Error processing callback for node 69c9221c6a4c026e16349261: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34810 - "GET / HTTP/1.0" 200 OK 2026-03-31 12:59:57,442 - lava_callback - INFO - Received callback for node 69c922416a4c026e1634926c INFO: 172.19.0.1:34820 - "POST /node/69c922416a4c026e1634926c HTTP/1.0" 202 Accepted 2026-03-31 12:59:57,448 - lava_callback - INFO - Starting async job submit for node 69c922416a4c026e1634926c 2026-03-31 12:59:57,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:57,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c922416a4c026e1634926c HTTP/1.1" 200 None 2026-03-31 12:59:57,466 - lava_callback - DEBUG - Processing node 69c922416a4c026e1634926c: baseline-arm-clabbe 2026-03-31 12:59:57,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c922416a4c026e1634926c 2026-03-31 12:59:57,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:57,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c922416a4c026e1634926c HTTP/1.1" 200 None 2026-03-31 12:59:57,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:57,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 12:59:57,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 12:59:57,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c922416a4c026e1634926c HTTP/1.1" 200 None 2026-03-31 12:59:57,526 - lava_callback - ERROR - Error processing callback for node 69c922416a4c026e1634926c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53020 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:00:37,535 - lava_callback - INFO - Received callback for node 69c922666a4c026e16349276 INFO: 172.19.0.1:53036 - "POST /node/69c922666a4c026e16349276 HTTP/1.0" 202 Accepted 2026-03-31 13:00:37,540 - lava_callback - INFO - Starting async job submit for node 69c922666a4c026e16349276 2026-03-31 13:00:37,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:00:37,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c922666a4c026e16349276 HTTP/1.1" 200 None 2026-03-31 13:00:37,556 - lava_callback - DEBUG - Processing node 69c922666a4c026e16349276: baseline-arm-clabbe 2026-03-31 13:00:37,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c922666a4c026e16349276 2026-03-31 13:00:37,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:00:37,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c922666a4c026e16349276 HTTP/1.1" 200 None 2026-03-31 13:00:37,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:00:37,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-31 13:00:37,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:00:37,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c922666a4c026e16349276 HTTP/1.1" 200 None 2026-03-31 13:00:37,613 - lava_callback - ERROR - Error processing callback for node 69c922666a4c026e16349276: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55284 - "GET /node/698d49bf820eff0a56c15c2b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57404 - "GET /node/698a9b49d9a1cfa777a24e88%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50982 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:07:41,010 - lava_callback - INFO - Received callback for node 69c9241b6a4c026e1634929f INFO: 172.19.0.1:43146 - "POST /node/69c9241b6a4c026e1634929f HTTP/1.0" 202 Accepted 2026-03-31 13:07:41,019 - lava_callback - INFO - Starting async job submit for node 69c9241b6a4c026e1634929f 2026-03-31 13:07:41,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:07:41,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9241b6a4c026e1634929f HTTP/1.1" 200 None 2026-03-31 13:07:41,058 - lava_callback - DEBUG - Processing node 69c9241b6a4c026e1634929f: baseline-arm64-baylibre 2026-03-31 13:07:41,059 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9241b6a4c026e1634929f 2026-03-31 13:07:41,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:07:41,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9241b6a4c026e1634929f HTTP/1.1" 200 None 2026-03-31 13:07:41,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:07:41,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:07:41,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:07:41,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9241b6a4c026e1634929f HTTP/1.1" 200 None 2026-03-31 13:07:41,129 - lava_callback - ERROR - Error processing callback for node 69c9241b6a4c026e1634929f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38104 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:09:00,902 - lava_callback - INFO - Received callback for node 69c924686a4c026e163492a7 INFO: 172.19.0.1:56912 - "POST /node/69c924686a4c026e163492a7 HTTP/1.0" 202 Accepted 2026-03-31 13:09:00,908 - lava_callback - INFO - Starting async job submit for node 69c924686a4c026e163492a7 2026-03-31 13:09:00,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:09:00,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c924686a4c026e163492a7 HTTP/1.1" 200 None 2026-03-31 13:09:00,925 - lava_callback - DEBUG - Processing node 69c924686a4c026e163492a7: baseline-arm64-baylibre 2026-03-31 13:09:00,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c924686a4c026e163492a7 2026-03-31 13:09:00,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:09:00,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c924686a4c026e163492a7 HTTP/1.1" 200 None 2026-03-31 13:09:00,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:09:00,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:09:00,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:09:00,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c924686a4c026e163492a7 HTTP/1.1" 200 None 2026-03-31 13:09:00,979 - lava_callback - ERROR - Error processing callback for node 69c924686a4c026e163492a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42134 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:10:20,982 - lava_callback - INFO - Received callback for node 69c924b46a4c026e163492b7 INFO: 172.19.0.1:46112 - "POST /node/69c924b46a4c026e163492b7 HTTP/1.0" 202 Accepted 2026-03-31 13:10:20,988 - lava_callback - INFO - Starting async job submit for node 69c924b46a4c026e163492b7 2026-03-31 13:10:20,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:10:21,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c924b46a4c026e163492b7 HTTP/1.1" 200 None 2026-03-31 13:10:21,018 - lava_callback - DEBUG - Processing node 69c924b46a4c026e163492b7: baseline-arm64-baylibre 2026-03-31 13:10:21,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c924b46a4c026e163492b7 2026-03-31 13:10:21,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:10:21,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c924b46a4c026e163492b7 HTTP/1.1" 200 None 2026-03-31 13:10:21,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:10:21,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:10:21,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:10:21,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c924b46a4c026e163492b7 HTTP/1.1" 200 None 2026-03-31 13:10:21,070 - lava_callback - ERROR - Error processing callback for node 69c924b46a4c026e163492b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32858 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:11:41,131 - lava_callback - INFO - Received callback for node 69c925026a4c026e163492bd INFO: 172.19.0.1:59342 - "POST /node/69c925026a4c026e163492bd HTTP/1.0" 202 Accepted 2026-03-31 13:11:41,136 - lava_callback - INFO - Starting async job submit for node 69c925026a4c026e163492bd 2026-03-31 13:11:41,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:11:41,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c925026a4c026e163492bd HTTP/1.1" 200 None 2026-03-31 13:11:41,152 - lava_callback - DEBUG - Processing node 69c925026a4c026e163492bd: baseline-arm64-baylibre 2026-03-31 13:11:41,152 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c925026a4c026e163492bd 2026-03-31 13:11:41,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:11:41,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c925026a4c026e163492bd HTTP/1.1" 200 None 2026-03-31 13:11:41,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:11:41,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:11:41,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:11:41,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c925026a4c026e163492bd HTTP/1.1" 200 None 2026-03-31 13:11:41,236 - lava_callback - ERROR - Error processing callback for node 69c925026a4c026e163492bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47132 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:13:01,135 - lava_callback - INFO - Received callback for node 69c9254f6a4c026e163492cf INFO: 172.19.0.1:38540 - "POST /node/69c9254f6a4c026e163492cf HTTP/1.0" 202 Accepted 2026-03-31 13:13:01,141 - lava_callback - INFO - Starting async job submit for node 69c9254f6a4c026e163492cf 2026-03-31 13:13:01,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:13:01,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9254f6a4c026e163492cf HTTP/1.1" 200 None 2026-03-31 13:13:01,156 - lava_callback - DEBUG - Processing node 69c9254f6a4c026e163492cf: baseline-arm64-baylibre 2026-03-31 13:13:01,157 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9254f6a4c026e163492cf 2026-03-31 13:13:01,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:13:01,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9254f6a4c026e163492cf HTTP/1.1" 200 None 2026-03-31 13:13:01,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:13:01,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:13:01,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:13:01,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9254f6a4c026e163492cf HTTP/1.1" 200 None 2026-03-31 13:13:01,217 - lava_callback - ERROR - Error processing callback for node 69c9254f6a4c026e163492cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34010 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:14:21,194 - lava_callback - INFO - Received callback for node 69c9259c6a4c026e163492d9 INFO: 172.19.0.1:33140 - "POST /node/69c9259c6a4c026e163492d9 HTTP/1.0" 202 Accepted 2026-03-31 13:14:21,199 - lava_callback - INFO - Starting async job submit for node 69c9259c6a4c026e163492d9 2026-03-31 13:14:21,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:14:21,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9259c6a4c026e163492d9 HTTP/1.1" 200 None 2026-03-31 13:14:21,216 - lava_callback - DEBUG - Processing node 69c9259c6a4c026e163492d9: baseline-arm64-baylibre 2026-03-31 13:14:21,217 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9259c6a4c026e163492d9 2026-03-31 13:14:21,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:14:21,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9259c6a4c026e163492d9 HTTP/1.1" 200 None 2026-03-31 13:14:21,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:14:21,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:14:21,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:14:21,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9259c6a4c026e163492d9 HTTP/1.1" 200 None 2026-03-31 13:14:21,268 - lava_callback - ERROR - Error processing callback for node 69c9259c6a4c026e163492d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52776 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:16:41,298 - lava_callback - INFO - Received callback for node 69c926336a4c026e163492f0 INFO: 172.19.0.1:50650 - "POST /node/69c926336a4c026e163492f0 HTTP/1.0" 202 Accepted 2026-03-31 13:16:41,303 - lava_callback - INFO - Starting async job submit for node 69c926336a4c026e163492f0 2026-03-31 13:16:41,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:16:41,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c926336a4c026e163492f0 HTTP/1.1" 200 None 2026-03-31 13:16:41,319 - lava_callback - DEBUG - Processing node 69c926336a4c026e163492f0: baseline-arm64-baylibre 2026-03-31 13:16:41,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c926336a4c026e163492f0 2026-03-31 13:16:41,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:16:41,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c926336a4c026e163492f0 HTTP/1.1" 200 None 2026-03-31 13:16:41,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:16:41,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:16:41,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:16:41,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c926336a4c026e163492f0 HTTP/1.1" 200 None 2026-03-31 13:16:41,381 - lava_callback - ERROR - Error processing callback for node 69c926336a4c026e163492f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41608 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:18:01,365 - lava_callback - INFO - Received callback for node 69c9267c6a4c026e16349301 INFO: 172.19.0.1:59300 - "POST /node/69c9267c6a4c026e16349301 HTTP/1.0" 202 Accepted 2026-03-31 13:18:01,371 - lava_callback - INFO - Starting async job submit for node 69c9267c6a4c026e16349301 2026-03-31 13:18:01,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:18:01,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9267c6a4c026e16349301 HTTP/1.1" 200 None 2026-03-31 13:18:01,388 - lava_callback - DEBUG - Processing node 69c9267c6a4c026e16349301: baseline-arm64-baylibre 2026-03-31 13:18:01,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9267c6a4c026e16349301 2026-03-31 13:18:01,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:18:01,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9267c6a4c026e16349301 HTTP/1.1" 200 None 2026-03-31 13:18:01,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:18:01,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:18:01,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:18:01,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9267c6a4c026e16349301 HTTP/1.1" 200 None 2026-03-31 13:18:01,468 - lava_callback - ERROR - Error processing callback for node 69c9267c6a4c026e16349301: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53748 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:19:18,345 - lava_callback - INFO - Received callback for node 69c926cf6a4c026e16349313 INFO: 172.19.0.1:53752 - "POST /node/69c926cf6a4c026e16349313 HTTP/1.0" 202 Accepted 2026-03-31 13:19:18,351 - lava_callback - INFO - Starting async job submit for node 69c926cf6a4c026e16349313 2026-03-31 13:19:18,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:19:18,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c926cf6a4c026e16349313 HTTP/1.1" 200 None 2026-03-31 13:19:18,377 - lava_callback - DEBUG - Processing node 69c926cf6a4c026e16349313: baseline-arm64-clabbe 2026-03-31 13:19:18,377 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c926cf6a4c026e16349313 2026-03-31 13:19:18,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:19:18,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c926cf6a4c026e16349313 HTTP/1.1" 200 None 2026-03-31 13:19:18,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:19:18,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:19:18,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:19:18,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c926cf6a4c026e16349313 HTTP/1.1" 200 None 2026-03-31 13:19:18,454 - lava_callback - ERROR - Error processing callback for node 69c926cf6a4c026e16349313: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43494 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:20:38,381 - lava_callback - INFO - Received callback for node 69c9271c6a4c026e1634931e INFO: 172.19.0.1:43502 - "POST /node/69c9271c6a4c026e1634931e HTTP/1.0" 202 Accepted 2026-03-31 13:20:38,386 - lava_callback - INFO - Starting async job submit for node 69c9271c6a4c026e1634931e 2026-03-31 13:20:38,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:20:38,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9271c6a4c026e1634931e HTTP/1.1" 200 None 2026-03-31 13:20:38,414 - lava_callback - DEBUG - Processing node 69c9271c6a4c026e1634931e: baseline-arm64-clabbe 2026-03-31 13:20:38,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9271c6a4c026e1634931e 2026-03-31 13:20:38,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:20:38,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9271c6a4c026e1634931e HTTP/1.1" 200 None 2026-03-31 13:20:38,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:20:38,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:20:38,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:20:38,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9271c6a4c026e1634931e HTTP/1.1" 200 None 2026-03-31 13:20:38,466 - lava_callback - ERROR - Error processing callback for node 69c9271c6a4c026e1634931e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56458 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:21:58,479 - lava_callback - INFO - Received callback for node 69c927696a4c026e16349330 INFO: 172.19.0.1:56460 - "POST /node/69c927696a4c026e16349330 HTTP/1.0" 202 Accepted 2026-03-31 13:21:58,486 - lava_callback - INFO - Starting async job submit for node 69c927696a4c026e16349330 2026-03-31 13:21:58,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:21:58,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c927696a4c026e16349330 HTTP/1.1" 200 None 2026-03-31 13:21:58,505 - lava_callback - DEBUG - Processing node 69c927696a4c026e16349330: baseline-arm64-clabbe 2026-03-31 13:21:58,506 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c927696a4c026e16349330 2026-03-31 13:21:58,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:21:58,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c927696a4c026e16349330 HTTP/1.1" 200 None 2026-03-31 13:21:58,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:21:58,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:21:58,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:21:58,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c927696a4c026e16349330 HTTP/1.1" 200 None 2026-03-31 13:21:58,565 - lava_callback - ERROR - Error processing callback for node 69c927696a4c026e16349330: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38654 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:23:18,485 - lava_callback - INFO - Received callback for node 69c927b66a4c026e1634933b INFO: 172.19.0.1:38668 - "POST /node/69c927b66a4c026e1634933b HTTP/1.0" 202 Accepted 2026-03-31 13:23:18,491 - lava_callback - INFO - Starting async job submit for node 69c927b66a4c026e1634933b 2026-03-31 13:23:18,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:23:18,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c927b66a4c026e1634933b HTTP/1.1" 200 None 2026-03-31 13:23:18,509 - lava_callback - DEBUG - Processing node 69c927b66a4c026e1634933b: baseline-arm64-clabbe 2026-03-31 13:23:18,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c927b66a4c026e1634933b 2026-03-31 13:23:18,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:23:18,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c927b66a4c026e1634933b HTTP/1.1" 200 None 2026-03-31 13:23:18,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:23:18,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:23:18,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:23:18,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c927b66a4c026e1634933b HTTP/1.1" 200 None 2026-03-31 13:23:18,589 - lava_callback - ERROR - Error processing callback for node 69c927b66a4c026e1634933b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56910 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59778 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:24:38,505 - lava_callback - INFO - Received callback for node 69c928036a4c026e1634934d INFO: 172.19.0.1:59784 - "POST /node/69c928036a4c026e1634934d HTTP/1.0" 202 Accepted 2026-03-31 13:24:38,510 - lava_callback - INFO - Starting async job submit for node 69c928036a4c026e1634934d 2026-03-31 13:24:38,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:24:38,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c928036a4c026e1634934d HTTP/1.1" 200 None 2026-03-31 13:24:38,528 - lava_callback - DEBUG - Processing node 69c928036a4c026e1634934d: baseline-arm64-clabbe 2026-03-31 13:24:38,528 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c928036a4c026e1634934d 2026-03-31 13:24:38,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:24:38,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c928036a4c026e1634934d HTTP/1.1" 200 None 2026-03-31 13:24:38,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:24:38,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:24:38,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:24:38,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c928036a4c026e1634934d HTTP/1.1" 200 None 2026-03-31 13:24:38,589 - lava_callback - ERROR - Error processing callback for node 69c928036a4c026e1634934d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53410 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:25:41,898 - lava_callback - INFO - Received callback for node 69c9284d6a4c026e16349357 INFO: 172.19.0.1:57192 - "POST /node/69c9284d6a4c026e16349357 HTTP/1.0" 202 Accepted 2026-03-31 13:25:41,904 - lava_callback - INFO - Starting async job submit for node 69c9284d6a4c026e16349357 2026-03-31 13:25:41,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:25:41,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9284d6a4c026e16349357 HTTP/1.1" 200 None 2026-03-31 13:25:41,920 - lava_callback - DEBUG - Processing node 69c9284d6a4c026e16349357: baseline-arm64-baylibre 2026-03-31 13:25:41,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9284d6a4c026e16349357 2026-03-31 13:25:41,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:25:41,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9284d6a4c026e16349357 HTTP/1.1" 200 None 2026-03-31 13:25:41,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:25:41,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:25:41,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:25:42,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9284d6a4c026e16349357 HTTP/1.1" 200 None 2026-03-31 13:25:42,008 - lava_callback - ERROR - Error processing callback for node 69c9284d6a4c026e16349357: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33866 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:26:58,630 - lava_callback - INFO - Received callback for node 69c9289e6a4c026e16349365 INFO: 172.19.0.1:33880 - "POST /node/69c9289e6a4c026e16349365 HTTP/1.0" 202 Accepted 2026-03-31 13:26:58,635 - lava_callback - INFO - Starting async job submit for node 69c9289e6a4c026e16349365 2026-03-31 13:26:58,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:26:58,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9289e6a4c026e16349365 HTTP/1.1" 200 None 2026-03-31 13:26:58,653 - lava_callback - DEBUG - Processing node 69c9289e6a4c026e16349365: baseline-arm64-clabbe 2026-03-31 13:26:58,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9289e6a4c026e16349365 2026-03-31 13:26:58,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:26:58,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9289e6a4c026e16349365 HTTP/1.1" 200 None 2026-03-31 13:26:58,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:26:58,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:26:58,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:26:58,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9289e6a4c026e16349365 HTTP/1.1" 200 None 2026-03-31 13:26:58,716 - lava_callback - ERROR - Error processing callback for node 69c9289e6a4c026e16349365: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43492 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:28:18,671 - lava_callback - INFO - Received callback for node 69c928ea6a4c026e16349376 INFO: 172.19.0.1:43494 - "POST /node/69c928ea6a4c026e16349376 HTTP/1.0" 202 Accepted 2026-03-31 13:28:18,677 - lava_callback - INFO - Starting async job submit for node 69c928ea6a4c026e16349376 2026-03-31 13:28:18,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:28:18,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c928ea6a4c026e16349376 HTTP/1.1" 200 None 2026-03-31 13:28:18,698 - lava_callback - DEBUG - Processing node 69c928ea6a4c026e16349376: baseline-arm64-clabbe 2026-03-31 13:28:18,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c928ea6a4c026e16349376 2026-03-31 13:28:18,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:28:18,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c928ea6a4c026e16349376 HTTP/1.1" 200 None 2026-03-31 13:28:18,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:28:18,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:28:18,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:28:18,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c928ea6a4c026e16349376 HTTP/1.1" 200 None 2026-03-31 13:28:18,767 - lava_callback - ERROR - Error processing callback for node 69c928ea6a4c026e16349376: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47674 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:29:38,707 - lava_callback - INFO - Received callback for node 69c929366a4c026e16349380 INFO: 172.19.0.1:47682 - "POST /node/69c929366a4c026e16349380 HTTP/1.0" 202 Accepted 2026-03-31 13:29:38,712 - lava_callback - INFO - Starting async job submit for node 69c929366a4c026e16349380 2026-03-31 13:29:38,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:29:38,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929366a4c026e16349380 HTTP/1.1" 200 None 2026-03-31 13:29:38,728 - lava_callback - DEBUG - Processing node 69c929366a4c026e16349380: baseline-arm64-clabbe 2026-03-31 13:29:38,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c929366a4c026e16349380 2026-03-31 13:29:38,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:29:38,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929366a4c026e16349380 HTTP/1.1" 200 None 2026-03-31 13:29:38,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:29:38,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:29:38,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:29:38,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c929366a4c026e16349380 HTTP/1.1" 200 None 2026-03-31 13:29:38,790 - lava_callback - ERROR - Error processing callback for node 69c929366a4c026e16349380: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58860 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:30:58,719 - lava_callback - INFO - Received callback for node 69c929836a4c026e1634938f INFO: 172.19.0.1:58874 - "POST /node/69c929836a4c026e1634938f HTTP/1.0" 202 Accepted 2026-03-31 13:30:58,725 - lava_callback - INFO - Starting async job submit for node 69c929836a4c026e1634938f 2026-03-31 13:30:58,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:30:58,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929836a4c026e1634938f HTTP/1.1" 200 None 2026-03-31 13:30:58,747 - lava_callback - DEBUG - Processing node 69c929836a4c026e1634938f: baseline-arm64-clabbe 2026-03-31 13:30:58,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c929836a4c026e1634938f 2026-03-31 13:30:58,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:30:58,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929836a4c026e1634938f HTTP/1.1" 200 None 2026-03-31 13:30:58,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:30:58,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:30:58,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:30:58,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c929836a4c026e1634938f HTTP/1.1" 200 None 2026-03-31 13:30:58,808 - lava_callback - ERROR - Error processing callback for node 69c929836a4c026e1634938f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45646 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:32:18,820 - lava_callback - INFO - Received callback for node 69c929d16a4c026e16349397 INFO: 172.19.0.1:45660 - "POST /node/69c929d16a4c026e16349397 HTTP/1.0" 202 Accepted 2026-03-31 13:32:18,825 - lava_callback - INFO - Starting async job submit for node 69c929d16a4c026e16349397 2026-03-31 13:32:18,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:32:18,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929d16a4c026e16349397 HTTP/1.1" 200 None 2026-03-31 13:32:18,843 - lava_callback - DEBUG - Processing node 69c929d16a4c026e16349397: baseline-arm64-clabbe 2026-03-31 13:32:18,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c929d16a4c026e16349397 2026-03-31 13:32:18,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:32:18,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c929d16a4c026e16349397 HTTP/1.1" 200 None 2026-03-31 13:32:18,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:32:18,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:32:18,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:32:18,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c929d16a4c026e16349397 HTTP/1.1" 200 None 2026-03-31 13:32:18,917 - lava_callback - ERROR - Error processing callback for node 69c929d16a4c026e16349397: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33088 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:33:38,857 - lava_callback - INFO - Received callback for node 69c92a1f6a4c026e163493a6 INFO: 172.19.0.1:33090 - "POST /node/69c92a1f6a4c026e163493a6 HTTP/1.0" 202 Accepted 2026-03-31 13:33:38,862 - lava_callback - INFO - Starting async job submit for node 69c92a1f6a4c026e163493a6 2026-03-31 13:33:38,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:33:38,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92a1f6a4c026e163493a6 HTTP/1.1" 200 None 2026-03-31 13:33:38,879 - lava_callback - DEBUG - Processing node 69c92a1f6a4c026e163493a6: baseline-arm64-clabbe 2026-03-31 13:33:38,879 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92a1f6a4c026e163493a6 2026-03-31 13:33:38,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:33:38,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92a1f6a4c026e163493a6 HTTP/1.1" 200 None 2026-03-31 13:33:38,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:33:38,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:33:38,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:33:38,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92a1f6a4c026e163493a6 HTTP/1.1" 200 None 2026-03-31 13:33:38,930 - lava_callback - ERROR - Error processing callback for node 69c92a1f6a4c026e163493a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39334 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:34:58,865 - lava_callback - INFO - Received callback for node 69c92a6d6a4c026e163493ae INFO: 172.19.0.1:39338 - "POST /node/69c92a6d6a4c026e163493ae HTTP/1.0" 202 Accepted 2026-03-31 13:34:58,871 - lava_callback - INFO - Starting async job submit for node 69c92a6d6a4c026e163493ae 2026-03-31 13:34:58,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:34:58,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92a6d6a4c026e163493ae HTTP/1.1" 200 None 2026-03-31 13:34:58,898 - lava_callback - DEBUG - Processing node 69c92a6d6a4c026e163493ae: baseline-arm64-clabbe 2026-03-31 13:34:58,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92a6d6a4c026e163493ae 2026-03-31 13:34:58,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:34:58,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92a6d6a4c026e163493ae HTTP/1.1" 200 None 2026-03-31 13:34:58,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:34:58,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:34:58,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:34:58,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92a6d6a4c026e163493ae HTTP/1.1" 200 None 2026-03-31 13:34:58,956 - lava_callback - ERROR - Error processing callback for node 69c92a6d6a4c026e163493ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44828 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:35:58,935 - lava_callback - INFO - Received callback for node 69c92aba6a4c026e163493b8 INFO: 172.19.0.1:44844 - "POST /node/69c92aba6a4c026e163493b8 HTTP/1.0" 202 Accepted 2026-03-31 13:35:58,941 - lava_callback - INFO - Starting async job submit for node 69c92aba6a4c026e163493b8 2026-03-31 13:35:58,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:35:58,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92aba6a4c026e163493b8 HTTP/1.1" 200 None 2026-03-31 13:35:58,967 - lava_callback - DEBUG - Processing node 69c92aba6a4c026e163493b8: baseline-arm64-clabbe 2026-03-31 13:35:58,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92aba6a4c026e163493b8 2026-03-31 13:35:58,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:35:58,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92aba6a4c026e163493b8 HTTP/1.1" 200 None 2026-03-31 13:35:58,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:35:58,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:35:58,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:35:59,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92aba6a4c026e163493b8 HTTP/1.1" 200 None 2026-03-31 13:35:59,026 - lava_callback - ERROR - Error processing callback for node 69c92aba6a4c026e163493b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50812 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:37:18,921 - lava_callback - INFO - Received callback for node 69c92b076a4c026e163493c4 INFO: 172.19.0.1:50828 - "POST /node/69c92b076a4c026e163493c4 HTTP/1.0" 202 Accepted 2026-03-31 13:37:18,931 - lava_callback - INFO - Starting async job submit for node 69c92b076a4c026e163493c4 2026-03-31 13:37:18,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:37:18,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92b076a4c026e163493c4 HTTP/1.1" 200 None 2026-03-31 13:37:18,973 - lava_callback - DEBUG - Processing node 69c92b076a4c026e163493c4: baseline-arm64-clabbe 2026-03-31 13:37:18,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92b076a4c026e163493c4 2026-03-31 13:37:18,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:37:18,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92b076a4c026e163493c4 HTTP/1.1" 200 None 2026-03-31 13:37:18,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:37:19,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:37:19,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:37:19,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92b076a4c026e163493c4 HTTP/1.1" 200 None 2026-03-31 13:37:19,026 - lava_callback - ERROR - Error processing callback for node 69c92b076a4c026e163493c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48876 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:38:39,080 - lava_callback - INFO - Received callback for node 69c92b546a4c026e163493cd INFO: 172.19.0.1:48878 - "POST /node/69c92b546a4c026e163493cd HTTP/1.0" 202 Accepted 2026-03-31 13:38:39,085 - lava_callback - INFO - Starting async job submit for node 69c92b546a4c026e163493cd 2026-03-31 13:38:39,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:38:39,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92b546a4c026e163493cd HTTP/1.1" 200 None 2026-03-31 13:38:39,101 - lava_callback - DEBUG - Processing node 69c92b546a4c026e163493cd: baseline-arm64-clabbe 2026-03-31 13:38:39,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92b546a4c026e163493cd 2026-03-31 13:38:39,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:38:39,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92b546a4c026e163493cd HTTP/1.1" 200 None 2026-03-31 13:38:39,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:38:39,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:38:39,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:38:39,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92b546a4c026e163493cd HTTP/1.1" 200 None 2026-03-31 13:38:39,199 - lava_callback - ERROR - Error processing callback for node 69c92b546a4c026e163493cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59906 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:39:59,122 - lava_callback - INFO - Received callback for node 69c92ba16a4c026e163493da INFO: 172.19.0.1:59908 - "POST /node/69c92ba16a4c026e163493da HTTP/1.0" 202 Accepted 2026-03-31 13:39:59,130 - lava_callback - INFO - Starting async job submit for node 69c92ba16a4c026e163493da 2026-03-31 13:39:59,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:39:59,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ba16a4c026e163493da HTTP/1.1" 200 None 2026-03-31 13:39:59,174 - lava_callback - DEBUG - Processing node 69c92ba16a4c026e163493da: baseline-arm64-clabbe 2026-03-31 13:39:59,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92ba16a4c026e163493da 2026-03-31 13:39:59,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:39:59,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ba16a4c026e163493da HTTP/1.1" 200 None 2026-03-31 13:39:59,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:39:59,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:39:59,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:39:59,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92ba16a4c026e163493da HTTP/1.1" 200 None 2026-03-31 13:39:59,273 - lava_callback - ERROR - Error processing callback for node 69c92ba16a4c026e163493da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35732 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:41:19,191 - lava_callback - INFO - Received callback for node 69c92bee6a4c026e163493e1 INFO: 172.19.0.1:35746 - "POST /node/69c92bee6a4c026e163493e1 HTTP/1.0" 202 Accepted 2026-03-31 13:41:19,197 - lava_callback - INFO - Starting async job submit for node 69c92bee6a4c026e163493e1 2026-03-31 13:41:19,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:41:19,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92bee6a4c026e163493e1 HTTP/1.1" 200 None 2026-03-31 13:41:19,213 - lava_callback - DEBUG - Processing node 69c92bee6a4c026e163493e1: baseline-arm64-clabbe 2026-03-31 13:41:19,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92bee6a4c026e163493e1 2026-03-31 13:41:19,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:41:19,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92bee6a4c026e163493e1 HTTP/1.1" 200 None 2026-03-31 13:41:19,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:41:19,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:41:19,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:41:19,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92bee6a4c026e163493e1 HTTP/1.1" 200 None 2026-03-31 13:41:19,285 - lava_callback - ERROR - Error processing callback for node 69c92bee6a4c026e163493e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53046 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:42:39,257 - lava_callback - INFO - Received callback for node 69c92c3d6a4c026e163493ef INFO: 172.19.0.1:53052 - "POST /node/69c92c3d6a4c026e163493ef HTTP/1.0" 202 Accepted 2026-03-31 13:42:39,263 - lava_callback - INFO - Starting async job submit for node 69c92c3d6a4c026e163493ef 2026-03-31 13:42:39,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:42:39,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92c3d6a4c026e163493ef HTTP/1.1" 200 None 2026-03-31 13:42:39,279 - lava_callback - DEBUG - Processing node 69c92c3d6a4c026e163493ef: baseline-arm64-clabbe 2026-03-31 13:42:39,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92c3d6a4c026e163493ef 2026-03-31 13:42:39,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:42:39,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92c3d6a4c026e163493ef HTTP/1.1" 200 None 2026-03-31 13:42:39,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:42:39,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-31 13:42:39,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:42:39,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92c3d6a4c026e163493ef HTTP/1.1" 200 None 2026-03-31 13:42:39,353 - lava_callback - ERROR - Error processing callback for node 69c92c3d6a4c026e163493ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39064 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:48:43,074 - lava_callback - INFO - Received callback for node 69c92db76a4c026e1634942b INFO: 172.19.0.1:43514 - "POST /node/69c92db76a4c026e1634942b HTTP/1.0" 202 Accepted 2026-03-31 13:48:43,079 - lava_callback - INFO - Starting async job submit for node 69c92db76a4c026e1634942b 2026-03-31 13:48:43,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:48:43,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92db76a4c026e1634942b HTTP/1.1" 200 None 2026-03-31 13:48:43,096 - lava_callback - DEBUG - Processing node 69c92db76a4c026e1634942b: baseline-arm64-baylibre 2026-03-31 13:48:43,096 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92db76a4c026e1634942b 2026-03-31 13:48:43,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:48:43,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92db76a4c026e1634942b HTTP/1.1" 200 None 2026-03-31 13:48:43,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:48:43,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:48:43,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:48:43,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92db76a4c026e1634942b HTTP/1.1" 200 None 2026-03-31 13:48:43,155 - lava_callback - ERROR - Error processing callback for node 69c92db76a4c026e1634942b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47576 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:50:19,592 - lava_callback - INFO - Received callback for node 69c92e0a6a4c026e16349435 INFO: 172.19.0.1:47578 - "POST /node/69c92e0a6a4c026e16349435 HTTP/1.0" 202 Accepted 2026-03-31 13:50:19,598 - lava_callback - INFO - Starting async job submit for node 69c92e0a6a4c026e16349435 2026-03-31 13:50:19,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:50:19,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92e0a6a4c026e16349435 HTTP/1.1" 200 None 2026-03-31 13:50:19,614 - lava_callback - DEBUG - Processing node 69c92e0a6a4c026e16349435: baseline-arm64-clabbe 2026-03-31 13:50:19,615 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92e0a6a4c026e16349435 2026-03-31 13:50:19,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:50:19,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92e0a6a4c026e16349435 HTTP/1.1" 200 None 2026-03-31 13:50:19,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:50:19,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:50:19,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:50:19,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92e0a6a4c026e16349435 HTTP/1.1" 200 None 2026-03-31 13:50:19,698 - lava_callback - ERROR - Error processing callback for node 69c92e0a6a4c026e16349435: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51820 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:51:59,640 - lava_callback - INFO - Received callback for node 69c92e7d6a4c026e16349441 INFO: 172.19.0.1:51824 - "POST /node/69c92e7d6a4c026e16349441 HTTP/1.0" 202 Accepted 2026-03-31 13:51:59,646 - lava_callback - INFO - Starting async job submit for node 69c92e7d6a4c026e16349441 2026-03-31 13:51:59,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:51:59,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92e7d6a4c026e16349441 HTTP/1.1" 200 None 2026-03-31 13:51:59,662 - lava_callback - DEBUG - Processing node 69c92e7d6a4c026e16349441: baseline-arm64-clabbe 2026-03-31 13:51:59,662 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92e7d6a4c026e16349441 2026-03-31 13:51:59,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:51:59,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92e7d6a4c026e16349441 HTTP/1.1" 200 None 2026-03-31 13:51:59,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:51:59,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:51:59,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:51:59,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92e7d6a4c026e16349441 HTTP/1.1" 200 None 2026-03-31 13:51:59,719 - lava_callback - ERROR - Error processing callback for node 69c92e7d6a4c026e16349441: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36506 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:53:19,666 - lava_callback - INFO - Received callback for node 69c92ec96a4c026e1634944f INFO: 172.19.0.1:36510 - "POST /node/69c92ec96a4c026e1634944f HTTP/1.0" 202 Accepted 2026-03-31 13:53:19,672 - lava_callback - INFO - Starting async job submit for node 69c92ec96a4c026e1634944f 2026-03-31 13:53:19,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:53:19,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ec96a4c026e1634944f HTTP/1.1" 200 None 2026-03-31 13:53:19,689 - lava_callback - DEBUG - Processing node 69c92ec96a4c026e1634944f: baseline-arm64-clabbe 2026-03-31 13:53:19,689 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92ec96a4c026e1634944f 2026-03-31 13:53:19,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:53:19,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ec96a4c026e1634944f HTTP/1.1" 200 None 2026-03-31 13:53:19,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:53:19,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:53:19,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:53:19,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92ec96a4c026e1634944f HTTP/1.1" 200 None 2026-03-31 13:53:19,749 - lava_callback - ERROR - Error processing callback for node 69c92ec96a4c026e1634944f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59128 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.19.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40010 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:54:39,614 - lava_callback - INFO - Received callback for node 69c92f156a4c026e1634945a INFO: 172.19.0.1:40020 - "POST /node/69c92f156a4c026e1634945a HTTP/1.0" 202 Accepted 2026-03-31 13:54:39,620 - lava_callback - INFO - Starting async job submit for node 69c92f156a4c026e1634945a 2026-03-31 13:54:39,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:54:39,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92f156a4c026e1634945a HTTP/1.1" 200 None 2026-03-31 13:54:39,637 - lava_callback - DEBUG - Processing node 69c92f156a4c026e1634945a: baseline-arm64-clabbe 2026-03-31 13:54:39,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92f156a4c026e1634945a 2026-03-31 13:54:39,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:54:39,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92f156a4c026e1634945a HTTP/1.1" 200 None 2026-03-31 13:54:39,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:54:39,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:54:39,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:54:39,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92f156a4c026e1634945a HTTP/1.1" 200 None 2026-03-31 13:54:39,708 - lava_callback - ERROR - Error processing callback for node 69c92f156a4c026e1634945a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37268 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:56:59,910 - lava_callback - INFO - Received callback for node 69c92fa96a4c026e16349468 INFO: 172.19.0.1:40336 - "POST /node/69c92fa96a4c026e16349468 HTTP/1.0" 202 Accepted 2026-03-31 13:56:59,916 - lava_callback - INFO - Starting async job submit for node 69c92fa96a4c026e16349468 2026-03-31 13:56:59,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:56:59,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92fa96a4c026e16349468 HTTP/1.1" 200 None 2026-03-31 13:56:59,945 - lava_callback - DEBUG - Processing node 69c92fa96a4c026e16349468: baseline-arm64-clabbe 2026-03-31 13:56:59,945 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92fa96a4c026e16349468 2026-03-31 13:56:59,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:56:59,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92fa96a4c026e16349468 HTTP/1.1" 200 None 2026-03-31 13:56:59,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:56:59,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:56:59,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:57:00,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92fa96a4c026e16349468 HTTP/1.1" 200 None 2026-03-31 13:57:00,026 - lava_callback - ERROR - Error processing callback for node 69c92fa96a4c026e16349468: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46002 - "GET / HTTP/1.0" 200 OK 2026-03-31 13:58:19,899 - lava_callback - INFO - Received callback for node 69c92ff66a4c026e16349477 INFO: 172.19.0.1:51232 - "POST /node/69c92ff66a4c026e16349477 HTTP/1.0" 202 Accepted 2026-03-31 13:58:19,905 - lava_callback - INFO - Starting async job submit for node 69c92ff66a4c026e16349477 2026-03-31 13:58:19,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:58:19,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ff66a4c026e16349477 HTTP/1.1" 200 None 2026-03-31 13:58:19,921 - lava_callback - DEBUG - Processing node 69c92ff66a4c026e16349477: baseline-arm64-clabbe 2026-03-31 13:58:19,921 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c92ff66a4c026e16349477 2026-03-31 13:58:19,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:58:19,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c92ff66a4c026e16349477 HTTP/1.1" 200 None 2026-03-31 13:58:19,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:58:19,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 13:58:19,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 13:58:19,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c92ff66a4c026e16349477 HTTP/1.1" 200 None 2026-03-31 13:58:19,989 - lava_callback - ERROR - Error processing callback for node 69c92ff66a4c026e16349477: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56726 - "GET /node/698c587d755d7811a4141957%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:56732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38824 - "GET /node/698f59243013abef95cf7221%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:38840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33126 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46754 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:38:05,801 - lava_callback - INFO - Received callback for node 69c9394b6a4c026e163494f7 INFO: 172.19.0.1:35808 - "POST /node/69c9394b6a4c026e163494f7 HTTP/1.0" 202 Accepted 2026-03-31 14:38:05,811 - lava_callback - INFO - Starting async job submit for node 69c9394b6a4c026e163494f7 2026-03-31 14:38:05,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:05,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9394b6a4c026e163494f7 HTTP/1.1" 200 None 2026-03-31 14:38:05,830 - lava_callback - DEBUG - Processing node 69c9394b6a4c026e163494f7: baseline-arm-baylibre 2026-03-31 14:38:05,830 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9394b6a4c026e163494f7 2026-03-31 14:38:05,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:05,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9394b6a4c026e163494f7 HTTP/1.1" 200 None 2026-03-31 14:38:05,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:05,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:38:05,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:05,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9394b6a4c026e163494f7 HTTP/1.1" 200 None 2026-03-31 14:38:05,905 - lava_callback - ERROR - Error processing callback for node 69c9394b6a4c026e163494f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46840 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:38:45,744 - lava_callback - INFO - Received callback for node 69c939706a4c026e16349503 INFO: 172.19.0.1:41514 - "POST /node/69c939706a4c026e16349503 HTTP/1.0" 202 Accepted 2026-03-31 14:38:45,750 - lava_callback - INFO - Starting async job submit for node 69c939706a4c026e16349503 2026-03-31 14:38:45,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:45,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939706a4c026e16349503 HTTP/1.1" 200 None 2026-03-31 14:38:45,768 - lava_callback - DEBUG - Processing node 69c939706a4c026e16349503: baseline-arm-baylibre 2026-03-31 14:38:45,768 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c939706a4c026e16349503 2026-03-31 14:38:45,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:45,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939706a4c026e16349503 HTTP/1.1" 200 None 2026-03-31 14:38:45,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:45,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:38:45,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:38:45,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c939706a4c026e16349503 HTTP/1.1" 200 None 2026-03-31 14:38:45,834 - lava_callback - ERROR - Error processing callback for node 69c939706a4c026e16349503: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55172 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:39:25,858 - lava_callback - INFO - Received callback for node 69c939956a4c026e1634950d INFO: 172.19.0.1:37472 - "POST /node/69c939956a4c026e1634950d HTTP/1.0" 202 Accepted 2026-03-31 14:39:25,864 - lava_callback - INFO - Starting async job submit for node 69c939956a4c026e1634950d 2026-03-31 14:39:25,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:39:25,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939956a4c026e1634950d HTTP/1.1" 200 None 2026-03-31 14:39:25,880 - lava_callback - DEBUG - Processing node 69c939956a4c026e1634950d: baseline-arm-baylibre 2026-03-31 14:39:25,880 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c939956a4c026e1634950d 2026-03-31 14:39:25,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:39:25,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939956a4c026e1634950d HTTP/1.1" 200 None 2026-03-31 14:39:25,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:39:25,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:39:25,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:39:25,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c939956a4c026e1634950d HTTP/1.1" 200 None 2026-03-31 14:39:25,953 - lava_callback - ERROR - Error processing callback for node 69c939956a4c026e1634950d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59474 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:40:05,815 - lava_callback - INFO - Received callback for node 69c939b96a4c026e1634951b INFO: 172.19.0.1:53256 - "POST /node/69c939b96a4c026e1634951b HTTP/1.0" 202 Accepted 2026-03-31 14:40:05,820 - lava_callback - INFO - Starting async job submit for node 69c939b96a4c026e1634951b 2026-03-31 14:40:05,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:05,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939b96a4c026e1634951b HTTP/1.1" 200 None 2026-03-31 14:40:05,840 - lava_callback - DEBUG - Processing node 69c939b96a4c026e1634951b: baseline-arm-baylibre 2026-03-31 14:40:05,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c939b96a4c026e1634951b 2026-03-31 14:40:05,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:05,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939b96a4c026e1634951b HTTP/1.1" 200 None 2026-03-31 14:40:05,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:05,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:40:05,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:05,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c939b96a4c026e1634951b HTTP/1.1" 200 None 2026-03-31 14:40:05,895 - lava_callback - ERROR - Error processing callback for node 69c939b96a4c026e1634951b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54078 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:40:45,965 - lava_callback - INFO - Received callback for node 69c939df6a4c026e16349527 INFO: 172.19.0.1:49078 - "POST /node/69c939df6a4c026e16349527 HTTP/1.0" 202 Accepted 2026-03-31 14:40:45,970 - lava_callback - INFO - Starting async job submit for node 69c939df6a4c026e16349527 2026-03-31 14:40:45,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:45,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939df6a4c026e16349527 HTTP/1.1" 200 None 2026-03-31 14:40:45,991 - lava_callback - DEBUG - Processing node 69c939df6a4c026e16349527: baseline-arm-baylibre 2026-03-31 14:40:45,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c939df6a4c026e16349527 2026-03-31 14:40:45,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:46,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c939df6a4c026e16349527 HTTP/1.1" 200 None 2026-03-31 14:40:46,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:46,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:40:46,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:40:46,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c939df6a4c026e16349527 HTTP/1.1" 200 None 2026-03-31 14:40:46,052 - lava_callback - ERROR - Error processing callback for node 69c939df6a4c026e16349527: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52314 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:42:41,710 - lava_callback - INFO - Received callback for node 69c93a4d6a4c026e1634954a INFO: 172.19.0.1:34304 - "POST /node/69c93a4d6a4c026e1634954a HTTP/1.0" 202 Accepted 2026-03-31 14:42:41,717 - lava_callback - INFO - Starting async job submit for node 69c93a4d6a4c026e1634954a 2026-03-31 14:42:41,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:42:41,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a4d6a4c026e1634954a HTTP/1.1" 200 None 2026-03-31 14:42:41,740 - lava_callback - DEBUG - Processing node 69c93a4d6a4c026e1634954a: baseline-arm-clabbe 2026-03-31 14:42:41,740 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93a4d6a4c026e1634954a 2026-03-31 14:42:41,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:42:41,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a4d6a4c026e1634954a HTTP/1.1" 200 None 2026-03-31 14:42:41,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:42:41,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:42:41,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:42:41,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93a4d6a4c026e1634954a HTTP/1.1" 200 None 2026-03-31 14:42:41,799 - lava_callback - ERROR - Error processing callback for node 69c93a4d6a4c026e1634954a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33582 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:43:01,690 - lava_callback - INFO - Received callback for node 69c93a726a4c026e16349555 INFO: 172.19.0.1:60988 - "POST /node/69c93a726a4c026e16349555 HTTP/1.0" 202 Accepted 2026-03-31 14:43:01,695 - lava_callback - INFO - Starting async job submit for node 69c93a726a4c026e16349555 2026-03-31 14:43:01,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:01,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a726a4c026e16349555 HTTP/1.1" 200 None 2026-03-31 14:43:01,726 - lava_callback - DEBUG - Processing node 69c93a726a4c026e16349555: baseline-arm-clabbe 2026-03-31 14:43:01,727 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93a726a4c026e16349555 2026-03-31 14:43:01,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:01,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a726a4c026e16349555 HTTP/1.1" 200 None 2026-03-31 14:43:01,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:01,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:43:01,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:01,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93a726a4c026e16349555 HTTP/1.1" 200 None 2026-03-31 14:43:01,792 - lava_callback - ERROR - Error processing callback for node 69c93a726a4c026e16349555: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:32772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46798 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:43:41,749 - lava_callback - INFO - Received callback for node 69c93a986a4c026e1634955f INFO: 172.19.0.1:58568 - "POST /node/69c93a986a4c026e1634955f HTTP/1.0" 202 Accepted 2026-03-31 14:43:41,754 - lava_callback - INFO - Starting async job submit for node 69c93a986a4c026e1634955f 2026-03-31 14:43:41,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:41,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a986a4c026e1634955f HTTP/1.1" 200 None 2026-03-31 14:43:41,776 - lava_callback - DEBUG - Processing node 69c93a986a4c026e1634955f: baseline-arm-clabbe 2026-03-31 14:43:41,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93a986a4c026e1634955f 2026-03-31 14:43:41,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:41,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93a986a4c026e1634955f HTTP/1.1" 200 None 2026-03-31 14:43:41,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:41,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:43:41,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:43:41,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93a986a4c026e1634955f HTTP/1.1" 200 None 2026-03-31 14:43:41,833 - lava_callback - ERROR - Error processing callback for node 69c93a986a4c026e1634955f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58974 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:44:21,729 - lava_callback - INFO - Received callback for node 69c93abe6a4c026e1634956f INFO: 172.19.0.1:52716 - "POST /node/69c93abe6a4c026e1634956f HTTP/1.0" 202 Accepted 2026-03-31 14:44:21,734 - lava_callback - INFO - Starting async job submit for node 69c93abe6a4c026e1634956f 2026-03-31 14:44:21,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:44:21,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93abe6a4c026e1634956f HTTP/1.1" 200 None 2026-03-31 14:44:21,751 - lava_callback - DEBUG - Processing node 69c93abe6a4c026e1634956f: baseline-arm-clabbe 2026-03-31 14:44:21,751 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93abe6a4c026e1634956f 2026-03-31 14:44:21,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:44:21,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93abe6a4c026e1634956f HTTP/1.1" 200 None 2026-03-31 14:44:21,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:44:21,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:44:21,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:44:21,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93abe6a4c026e1634956f HTTP/1.1" 200 None 2026-03-31 14:44:21,839 - lava_callback - ERROR - Error processing callback for node 69c93abe6a4c026e1634956f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48902 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:45:01,782 - lava_callback - INFO - Received callback for node 69c93ae36a4c026e1634957c INFO: 172.19.0.1:41992 - "POST /node/69c93ae36a4c026e1634957c HTTP/1.0" 202 Accepted 2026-03-31 14:45:01,788 - lava_callback - INFO - Starting async job submit for node 69c93ae36a4c026e1634957c 2026-03-31 14:45:01,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:01,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ae36a4c026e1634957c HTTP/1.1" 200 None 2026-03-31 14:45:01,805 - lava_callback - DEBUG - Processing node 69c93ae36a4c026e1634957c: baseline-arm-clabbe 2026-03-31 14:45:01,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93ae36a4c026e1634957c 2026-03-31 14:45:01,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:01,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ae36a4c026e1634957c HTTP/1.1" 200 None 2026-03-31 14:45:01,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:01,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 14:45:01,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:01,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93ae36a4c026e1634957c HTTP/1.1" 200 None 2026-03-31 14:45:01,874 - lava_callback - ERROR - Error processing callback for node 69c93ae36a4c026e1634957c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46826 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:45:26,174 - lava_callback - INFO - Received callback for node 69c93b026a4c026e1634958b INFO: 172.19.0.1:48614 - "POST /node/69c93b026a4c026e1634958b HTTP/1.0" 202 Accepted 2026-03-31 14:45:26,180 - lava_callback - INFO - Starting async job submit for node 69c93b026a4c026e1634958b 2026-03-31 14:45:26,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:26,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b026a4c026e1634958b HTTP/1.1" 200 None 2026-03-31 14:45:26,197 - lava_callback - DEBUG - Processing node 69c93b026a4c026e1634958b: baseline-arm-baylibre 2026-03-31 14:45:26,197 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93b026a4c026e1634958b 2026-03-31 14:45:26,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:26,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b026a4c026e1634958b HTTP/1.1" 200 None 2026-03-31 14:45:26,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:26,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:45:26,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:45:26,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93b026a4c026e1634958b HTTP/1.1" 200 None 2026-03-31 14:45:26,303 - lava_callback - ERROR - Error processing callback for node 69c93b026a4c026e1634958b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54614 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:46:06,141 - lava_callback - INFO - Received callback for node 69c93b276a4c026e16349597 INFO: 172.19.0.1:55766 - "POST /node/69c93b276a4c026e16349597 HTTP/1.0" 202 Accepted 2026-03-31 14:46:06,147 - lava_callback - INFO - Starting async job submit for node 69c93b276a4c026e16349597 2026-03-31 14:46:06,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:06,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b276a4c026e16349597 HTTP/1.1" 200 None 2026-03-31 14:46:06,175 - lava_callback - DEBUG - Processing node 69c93b276a4c026e16349597: baseline-arm-baylibre 2026-03-31 14:46:06,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93b276a4c026e16349597 2026-03-31 14:46:06,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:06,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b276a4c026e16349597 HTTP/1.1" 200 None 2026-03-31 14:46:06,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:06,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:46:06,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:06,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93b276a4c026e16349597 HTTP/1.1" 200 None 2026-03-31 14:46:06,236 - lava_callback - ERROR - Error processing callback for node 69c93b276a4c026e16349597: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34440 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:46:46,212 - lava_callback - INFO - Received callback for node 69c93b4b6a4c026e163495a4 INFO: 172.19.0.1:38506 - "POST /node/69c93b4b6a4c026e163495a4 HTTP/1.0" 202 Accepted 2026-03-31 14:46:46,218 - lava_callback - INFO - Starting async job submit for node 69c93b4b6a4c026e163495a4 2026-03-31 14:46:46,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:46,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b4b6a4c026e163495a4 HTTP/1.1" 200 None 2026-03-31 14:46:46,235 - lava_callback - DEBUG - Processing node 69c93b4b6a4c026e163495a4: baseline-arm-baylibre 2026-03-31 14:46:46,235 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93b4b6a4c026e163495a4 2026-03-31 14:46:46,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:46,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b4b6a4c026e163495a4 HTTP/1.1" 200 None 2026-03-31 14:46:46,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:46,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:46:46,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:46:46,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93b4b6a4c026e163495a4 HTTP/1.1" 200 None 2026-03-31 14:46:46,308 - lava_callback - ERROR - Error processing callback for node 69c93b4b6a4c026e163495a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58786 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:47:26,216 - lava_callback - INFO - Received callback for node 69c93b706a4c026e163495b7 INFO: 172.19.0.1:39854 - "POST /node/69c93b706a4c026e163495b7 HTTP/1.0" 202 Accepted 2026-03-31 14:47:26,222 - lava_callback - INFO - Starting async job submit for node 69c93b706a4c026e163495b7 2026-03-31 14:47:26,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:47:26,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b706a4c026e163495b7 HTTP/1.1" 200 None 2026-03-31 14:47:26,238 - lava_callback - DEBUG - Processing node 69c93b706a4c026e163495b7: baseline-arm-baylibre 2026-03-31 14:47:26,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93b706a4c026e163495b7 2026-03-31 14:47:26,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:47:26,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b706a4c026e163495b7 HTTP/1.1" 200 None 2026-03-31 14:47:26,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:47:26,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:47:26,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:47:26,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93b706a4c026e163495b7 HTTP/1.1" 200 None 2026-03-31 14:47:26,291 - lava_callback - ERROR - Error processing callback for node 69c93b706a4c026e163495b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60590 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:48:06,376 - lava_callback - INFO - Received callback for node 69c93b966a4c026e163495c0 INFO: 172.19.0.1:35488 - "POST /node/69c93b966a4c026e163495c0 HTTP/1.0" 202 Accepted 2026-03-31 14:48:06,382 - lava_callback - INFO - Starting async job submit for node 69c93b966a4c026e163495c0 2026-03-31 14:48:06,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:06,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b966a4c026e163495c0 HTTP/1.1" 200 None 2026-03-31 14:48:06,398 - lava_callback - DEBUG - Processing node 69c93b966a4c026e163495c0: baseline-arm-baylibre 2026-03-31 14:48:06,398 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93b966a4c026e163495c0 2026-03-31 14:48:06,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:06,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93b966a4c026e163495c0 HTTP/1.1" 200 None 2026-03-31 14:48:06,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:06,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:48:06,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:06,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93b966a4c026e163495c0 HTTP/1.1" 200 None 2026-03-31 14:48:06,453 - lava_callback - ERROR - Error processing callback for node 69c93b966a4c026e163495c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45006 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:48:46,307 - lava_callback - INFO - Received callback for node 69c93bba6a4c026e163495c9 INFO: 172.19.0.1:41574 - "POST /node/69c93bba6a4c026e163495c9 HTTP/1.0" 202 Accepted 2026-03-31 14:48:46,313 - lava_callback - INFO - Starting async job submit for node 69c93bba6a4c026e163495c9 2026-03-31 14:48:46,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:46,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93bba6a4c026e163495c9 HTTP/1.1" 200 None 2026-03-31 14:48:46,334 - lava_callback - DEBUG - Processing node 69c93bba6a4c026e163495c9: baseline-arm-baylibre 2026-03-31 14:48:46,335 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93bba6a4c026e163495c9 2026-03-31 14:48:46,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:46,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93bba6a4c026e163495c9 HTTP/1.1" 200 None 2026-03-31 14:48:46,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:46,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:48:46,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:48:46,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93bba6a4c026e163495c9 HTTP/1.1" 200 None 2026-03-31 14:48:46,408 - lava_callback - ERROR - Error processing callback for node 69c93bba6a4c026e163495c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49792 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:49:21,990 - lava_callback - INFO - Received callback for node 69c93be16a4c026e163495de INFO: 172.19.0.1:47044 - "POST /node/69c93be16a4c026e163495de HTTP/1.0" 202 Accepted 2026-03-31 14:49:21,996 - lava_callback - INFO - Starting async job submit for node 69c93be16a4c026e163495de 2026-03-31 14:49:21,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:22,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93be16a4c026e163495de HTTP/1.1" 200 None 2026-03-31 14:49:22,014 - lava_callback - DEBUG - Processing node 69c93be16a4c026e163495de: baseline-arm-clabbe 2026-03-31 14:49:22,014 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93be16a4c026e163495de 2026-03-31 14:49:22,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:22,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93be16a4c026e163495de HTTP/1.1" 200 None 2026-03-31 14:49:22,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:22,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:49:22,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:22,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93be16a4c026e163495de HTTP/1.1" 200 None 2026-03-31 14:49:22,089 - lava_callback - ERROR - Error processing callback for node 69c93be16a4c026e163495de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56322 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:49:46,360 - lava_callback - INFO - Received callback for node 69c93c016a4c026e163495ed INFO: 172.19.0.1:35300 - "POST /node/69c93c016a4c026e163495ed HTTP/1.0" 202 Accepted 2026-03-31 14:49:46,366 - lava_callback - INFO - Starting async job submit for node 69c93c016a4c026e163495ed 2026-03-31 14:49:46,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:46,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c016a4c026e163495ed HTTP/1.1" 200 None 2026-03-31 14:49:46,384 - lava_callback - DEBUG - Processing node 69c93c016a4c026e163495ed: baseline-arm-baylibre 2026-03-31 14:49:46,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93c016a4c026e163495ed 2026-03-31 14:49:46,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:46,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c016a4c026e163495ed HTTP/1.1" 200 None 2026-03-31 14:49:46,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:46,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:49:46,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:49:46,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93c016a4c026e163495ed HTTP/1.1" 200 None 2026-03-31 14:49:46,442 - lava_callback - ERROR - Error processing callback for node 69c93c016a4c026e163495ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39758 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:50:22,108 - lava_callback - INFO - Received callback for node 69c93c2b6a4c026e163495ff INFO: 172.19.0.1:47050 - "POST /node/69c93c2b6a4c026e163495ff HTTP/1.0" 202 Accepted 2026-03-31 14:50:22,113 - lava_callback - INFO - Starting async job submit for node 69c93c2b6a4c026e163495ff 2026-03-31 14:50:22,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:50:22,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c2b6a4c026e163495ff HTTP/1.1" 200 None 2026-03-31 14:50:22,137 - lava_callback - DEBUG - Processing node 69c93c2b6a4c026e163495ff: baseline-arm-clabbe 2026-03-31 14:50:22,137 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93c2b6a4c026e163495ff 2026-03-31 14:50:22,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:50:22,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c2b6a4c026e163495ff HTTP/1.1" 200 None 2026-03-31 14:50:22,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:50:22,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:50:22,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:50:22,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93c2b6a4c026e163495ff HTTP/1.1" 200 None 2026-03-31 14:50:22,196 - lava_callback - ERROR - Error processing callback for node 69c93c2b6a4c026e163495ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58916 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:51:06,683 - lava_callback - INFO - Received callback for node 69c93c4b6a4c026e1634960a INFO: 172.19.0.1:58930 - "POST /node/69c93c4b6a4c026e1634960a HTTP/1.0" 202 Accepted 2026-03-31 14:51:06,689 - lava_callback - INFO - Starting async job submit for node 69c93c4b6a4c026e1634960a 2026-03-31 14:51:06,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:06,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c4b6a4c026e1634960a HTTP/1.1" 200 None 2026-03-31 14:51:06,704 - lava_callback - DEBUG - Processing node 69c93c4b6a4c026e1634960a: baseline-arm-baylibre 2026-03-31 14:51:06,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93c4b6a4c026e1634960a 2026-03-31 14:51:06,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:06,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c4b6a4c026e1634960a HTTP/1.1" 200 None 2026-03-31 14:51:06,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:06,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:51:06,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:06,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93c4b6a4c026e1634960a HTTP/1.1" 200 None 2026-03-31 14:51:06,769 - lava_callback - ERROR - Error processing callback for node 69c93c4b6a4c026e1634960a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42008 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:51:46,445 - lava_callback - INFO - Received callback for node 69c93c706a4c026e16349616 INFO: 172.19.0.1:36320 - "POST /node/69c93c706a4c026e16349616 HTTP/1.0" 202 Accepted 2026-03-31 14:51:46,450 - lava_callback - INFO - Starting async job submit for node 69c93c706a4c026e16349616 2026-03-31 14:51:46,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:46,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c706a4c026e16349616 HTTP/1.1" 200 None 2026-03-31 14:51:46,466 - lava_callback - DEBUG - Processing node 69c93c706a4c026e16349616: baseline-arm-baylibre 2026-03-31 14:51:46,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93c706a4c026e16349616 2026-03-31 14:51:46,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:46,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c706a4c026e16349616 HTTP/1.1" 200 None 2026-03-31 14:51:46,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:51:46,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:51:46,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:36330 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:51:46,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93c706a4c026e16349616 HTTP/1.1" 200 None 2026-03-31 14:51:46,533 - lava_callback - ERROR - Error processing callback for node 69c93c706a4c026e16349616: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33474 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:52:22,164 - lava_callback - INFO - Received callback for node 69c93c986a4c026e1634962f INFO: 172.19.0.1:55930 - "POST /node/69c93c986a4c026e1634962f HTTP/1.0" 202 Accepted 2026-03-31 14:52:22,170 - lava_callback - INFO - Starting async job submit for node 69c93c986a4c026e1634962f 2026-03-31 14:52:22,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:52:22,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c986a4c026e1634962f HTTP/1.1" 200 None 2026-03-31 14:52:22,191 - lava_callback - DEBUG - Processing node 69c93c986a4c026e1634962f: baseline-arm-clabbe 2026-03-31 14:52:22,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93c986a4c026e1634962f 2026-03-31 14:52:22,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:52:22,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93c986a4c026e1634962f HTTP/1.1" 200 None 2026-03-31 14:52:22,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:52:22,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:52:22,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:52:22,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93c986a4c026e1634962f HTTP/1.1" 200 None 2026-03-31 14:52:22,267 - lava_callback - ERROR - Error processing callback for node 69c93c986a4c026e1634962f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55770 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:53:02,132 - lava_callback - INFO - Received callback for node 69c93cbe6a4c026e16349654 INFO: 172.19.0.1:47212 - "POST /node/69c93cbe6a4c026e16349654 HTTP/1.0" 202 Accepted 2026-03-31 14:53:02,137 - lava_callback - INFO - Starting async job submit for node 69c93cbe6a4c026e16349654 2026-03-31 14:53:02,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:02,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93cbe6a4c026e16349654 HTTP/1.1" 200 None 2026-03-31 14:53:02,160 - lava_callback - DEBUG - Processing node 69c93cbe6a4c026e16349654: baseline-arm-clabbe 2026-03-31 14:53:02,160 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93cbe6a4c026e16349654 2026-03-31 14:53:02,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:02,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93cbe6a4c026e16349654 HTTP/1.1" 200 None 2026-03-31 14:53:02,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:02,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:53:02,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:02,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93cbe6a4c026e16349654 HTTP/1.1" 200 None 2026-03-31 14:53:02,260 - lava_callback - ERROR - Error processing callback for node 69c93cbe6a4c026e16349654: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52614 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:53:26,583 - lava_callback - INFO - Received callback for node 69c93cdf6a4c026e16349667 INFO: 172.19.0.1:52618 - "POST /node/69c93cdf6a4c026e16349667 HTTP/1.0" 202 Accepted 2026-03-31 14:53:26,589 - lava_callback - INFO - Starting async job submit for node 69c93cdf6a4c026e16349667 2026-03-31 14:53:26,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:26,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93cdf6a4c026e16349667 HTTP/1.1" 200 None 2026-03-31 14:53:26,605 - lava_callback - DEBUG - Processing node 69c93cdf6a4c026e16349667: baseline-arm-baylibre 2026-03-31 14:53:26,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93cdf6a4c026e16349667 2026-03-31 14:53:26,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:26,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93cdf6a4c026e16349667 HTTP/1.1" 200 None 2026-03-31 14:53:26,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:26,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:53:26,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:53:26,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93cdf6a4c026e16349667 HTTP/1.1" 200 None 2026-03-31 14:53:26,668 - lava_callback - ERROR - Error processing callback for node 69c93cdf6a4c026e16349667: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55650 - "GET /robots.txt HTTP/1.0" 404 Not Found INFO: 172.19.0.1:55654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51194 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:54:02,151 - lava_callback - INFO - Received callback for node 69c93d076a4c026e16349672 INFO: 172.19.0.1:33338 - "POST /node/69c93d076a4c026e16349672 HTTP/1.0" 202 Accepted 2026-03-31 14:54:02,157 - lava_callback - INFO - Starting async job submit for node 69c93d076a4c026e16349672 2026-03-31 14:54:02,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:02,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d076a4c026e16349672 HTTP/1.1" 200 None 2026-03-31 14:54:02,173 - lava_callback - DEBUG - Processing node 69c93d076a4c026e16349672: baseline-arm-clabbe 2026-03-31 14:54:02,173 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93d076a4c026e16349672 2026-03-31 14:54:02,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:02,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d076a4c026e16349672 HTTP/1.1" 200 None 2026-03-31 14:54:02,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:02,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:54:02,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:02,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93d076a4c026e16349672 HTTP/1.1" 200 None 2026-03-31 14:54:02,233 - lava_callback - ERROR - Error processing callback for node 69c93d076a4c026e16349672: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39410 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:54:46,827 - lava_callback - INFO - Received callback for node 69c93d266a4c026e1634967f INFO: 172.19.0.1:39412 - "POST /node/69c93d266a4c026e1634967f HTTP/1.0" 202 Accepted 2026-03-31 14:54:46,833 - lava_callback - INFO - Starting async job submit for node 69c93d266a4c026e1634967f 2026-03-31 14:54:46,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:46,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d266a4c026e1634967f HTTP/1.1" 200 None 2026-03-31 14:54:46,849 - lava_callback - DEBUG - Processing node 69c93d266a4c026e1634967f: baseline-arm-baylibre 2026-03-31 14:54:46,849 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93d266a4c026e1634967f 2026-03-31 14:54:46,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:46,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d266a4c026e1634967f HTTP/1.1" 200 None 2026-03-31 14:54:46,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:46,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 14:54:46,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:54:46,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93d266a4c026e1634967f HTTP/1.1" 200 None 2026-03-31 14:54:46,916 - lava_callback - ERROR - Error processing callback for node 69c93d266a4c026e1634967f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54558 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:55:26,684 - lava_callback - INFO - Received callback for node 69c93d4d6a4c026e16349689 INFO: 172.19.0.1:54566 - "POST /node/69c93d4d6a4c026e16349689 HTTP/1.0" 202 Accepted 2026-03-31 14:55:26,689 - lava_callback - INFO - Starting async job submit for node 69c93d4d6a4c026e16349689 2026-03-31 14:55:26,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:55:26,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d4d6a4c026e16349689 HTTP/1.1" 200 None 2026-03-31 14:55:26,706 - lava_callback - DEBUG - Processing node 69c93d4d6a4c026e16349689: baseline-arm-baylibre 2026-03-31 14:55:26,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93d4d6a4c026e16349689 2026-03-31 14:55:26,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:55:26,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d4d6a4c026e16349689 HTTP/1.1" 200 None 2026-03-31 14:55:26,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:55:26,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:55:26,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:55:26,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93d4d6a4c026e16349689 HTTP/1.1" 200 None 2026-03-31 14:55:26,764 - lava_callback - ERROR - Error processing callback for node 69c93d4d6a4c026e16349689: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38342 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:56:06,734 - lava_callback - INFO - Received callback for node 69c93d716a4c026e16349695 INFO: 172.19.0.1:38358 - "POST /node/69c93d716a4c026e16349695 HTTP/1.0" 202 Accepted 2026-03-31 14:56:06,740 - lava_callback - INFO - Starting async job submit for node 69c93d716a4c026e16349695 2026-03-31 14:56:06,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:06,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d716a4c026e16349695 HTTP/1.1" 200 None 2026-03-31 14:56:06,756 - lava_callback - DEBUG - Processing node 69c93d716a4c026e16349695: baseline-arm-baylibre 2026-03-31 14:56:06,756 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93d716a4c026e16349695 2026-03-31 14:56:06,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:06,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d716a4c026e16349695 HTTP/1.1" 200 None 2026-03-31 14:56:06,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:06,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:56:06,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:06,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93d716a4c026e16349695 HTTP/1.1" 200 None 2026-03-31 14:56:06,819 - lava_callback - ERROR - Error processing callback for node 69c93d716a4c026e16349695: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52916 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:56:26,849 - lava_callback - INFO - Received callback for node 69c93d936a4c026e163496a0 INFO: 172.19.0.1:52922 - "POST /node/69c93d936a4c026e163496a0 HTTP/1.0" 202 Accepted 2026-03-31 14:56:26,854 - lava_callback - INFO - Starting async job submit for node 69c93d936a4c026e163496a0 2026-03-31 14:56:26,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:26,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d936a4c026e163496a0 HTTP/1.1" 200 None 2026-03-31 14:56:26,871 - lava_callback - DEBUG - Processing node 69c93d936a4c026e163496a0: baseline-arm-baylibre 2026-03-31 14:56:26,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93d936a4c026e163496a0 2026-03-31 14:56:26,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:26,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93d936a4c026e163496a0 HTTP/1.1" 200 None 2026-03-31 14:56:26,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:26,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 14:56:26,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:56:26,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93d936a4c026e163496a0 HTTP/1.1" 200 None 2026-03-31 14:56:26,945 - lava_callback - ERROR - Error processing callback for node 69c93d936a4c026e163496a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43860 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:57:06,728 - lava_callback - INFO - Received callback for node 69c93db86a4c026e163496ae INFO: 172.19.0.1:43872 - "POST /node/69c93db86a4c026e163496ae HTTP/1.0" 202 Accepted 2026-03-31 14:57:06,734 - lava_callback - INFO - Starting async job submit for node 69c93db86a4c026e163496ae 2026-03-31 14:57:06,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:06,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93db86a4c026e163496ae HTTP/1.1" 200 None 2026-03-31 14:57:06,750 - lava_callback - DEBUG - Processing node 69c93db86a4c026e163496ae: baseline-arm-baylibre 2026-03-31 14:57:06,750 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93db86a4c026e163496ae 2026-03-31 14:57:06,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:06,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93db86a4c026e163496ae HTTP/1.1" 200 None 2026-03-31 14:57:06,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:06,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 14:57:06,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:06,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93db86a4c026e163496ae HTTP/1.1" 200 None 2026-03-31 14:57:06,814 - lava_callback - ERROR - Error processing callback for node 69c93db86a4c026e163496ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45318 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:57:46,786 - lava_callback - INFO - Received callback for node 69c93ddc6a4c026e163496bd INFO: 172.19.0.1:45328 - "POST /node/69c93ddc6a4c026e163496bd HTTP/1.0" 202 Accepted 2026-03-31 14:57:46,792 - lava_callback - INFO - Starting async job submit for node 69c93ddc6a4c026e163496bd 2026-03-31 14:57:46,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:46,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ddc6a4c026e163496bd HTTP/1.1" 200 None 2026-03-31 14:57:46,808 - lava_callback - DEBUG - Processing node 69c93ddc6a4c026e163496bd: baseline-arm-baylibre 2026-03-31 14:57:46,808 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93ddc6a4c026e163496bd 2026-03-31 14:57:46,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:46,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ddc6a4c026e163496bd HTTP/1.1" 200 None 2026-03-31 14:57:46,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:46,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 14:57:46,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:57:46,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93ddc6a4c026e163496bd HTTP/1.1" 200 None 2026-03-31 14:57:46,866 - lava_callback - ERROR - Error processing callback for node 69c93ddc6a4c026e163496bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56012 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:58:22,394 - lava_callback - INFO - Received callback for node 69c93e046a4c026e163496d1 INFO: 172.19.0.1:40706 - "POST /node/69c93e046a4c026e163496d1 HTTP/1.0" 202 Accepted 2026-03-31 14:58:22,400 - lava_callback - INFO - Starting async job submit for node 69c93e046a4c026e163496d1 2026-03-31 14:58:22,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:58:22,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e046a4c026e163496d1 HTTP/1.1" 200 None 2026-03-31 14:58:22,415 - lava_callback - DEBUG - Processing node 69c93e046a4c026e163496d1: baseline-arm-clabbe 2026-03-31 14:58:22,416 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93e046a4c026e163496d1 2026-03-31 14:58:22,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:58:22,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e046a4c026e163496d1 HTTP/1.1" 200 None 2026-03-31 14:58:22,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:58:22,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:58:22,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:58:22,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93e046a4c026e163496d1 HTTP/1.1" 200 None 2026-03-31 14:58:22,475 - lava_callback - ERROR - Error processing callback for node 69c93e046a4c026e163496d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41426 - "GET / HTTP/1.0" 200 OK 2026-03-31 14:59:02,320 - lava_callback - INFO - Received callback for node 69c93e296a4c026e163496dd INFO: 172.19.0.1:55568 - "POST /node/69c93e296a4c026e163496dd HTTP/1.0" 202 Accepted 2026-03-31 14:59:02,325 - lava_callback - INFO - Starting async job submit for node 69c93e296a4c026e163496dd 2026-03-31 14:59:02,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:59:02,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e296a4c026e163496dd HTTP/1.1" 200 None 2026-03-31 14:59:02,342 - lava_callback - DEBUG - Processing node 69c93e296a4c026e163496dd: baseline-arm-clabbe 2026-03-31 14:59:02,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93e296a4c026e163496dd 2026-03-31 14:59:02,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:59:02,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e296a4c026e163496dd HTTP/1.1" 200 None 2026-03-31 14:59:02,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:59:02,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 14:59:02,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 14:59:02,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93e296a4c026e163496dd HTTP/1.1" 200 None 2026-03-31 14:59:02,405 - lava_callback - ERROR - Error processing callback for node 69c93e296a4c026e163496dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43676 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:00:42,472 - lava_callback - INFO - Received callback for node 69c93e946a4c026e163496fe INFO: 172.19.0.1:50726 - "POST /node/69c93e946a4c026e163496fe HTTP/1.0" 202 Accepted 2026-03-31 15:00:42,478 - lava_callback - INFO - Starting async job submit for node 69c93e946a4c026e163496fe 2026-03-31 15:00:42,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:00:42,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e946a4c026e163496fe HTTP/1.1" 200 None 2026-03-31 15:00:42,494 - lava_callback - DEBUG - Processing node 69c93e946a4c026e163496fe: baseline-arm-clabbe 2026-03-31 15:00:42,494 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93e946a4c026e163496fe 2026-03-31 15:00:42,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:00:42,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93e946a4c026e163496fe HTTP/1.1" 200 None 2026-03-31 15:00:42,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:00:42,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 15:00:42,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:00:42,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93e946a4c026e163496fe HTTP/1.1" 200 None 2026-03-31 15:00:42,549 - lava_callback - ERROR - Error processing callback for node 69c93e946a4c026e163496fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38364 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:01:22,430 - lava_callback - INFO - Received callback for node 69c93eb96a4c026e16349710 INFO: 172.19.0.1:39168 - "POST /node/69c93eb96a4c026e16349710 HTTP/1.0" 202 Accepted 2026-03-31 15:01:22,437 - lava_callback - INFO - Starting async job submit for node 69c93eb96a4c026e16349710 2026-03-31 15:01:22,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:22,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93eb96a4c026e16349710 HTTP/1.1" 200 None 2026-03-31 15:01:22,463 - lava_callback - DEBUG - Processing node 69c93eb96a4c026e16349710: baseline-arm-clabbe 2026-03-31 15:01:22,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93eb96a4c026e16349710 2026-03-31 15:01:22,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:22,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93eb96a4c026e16349710 HTTP/1.1" 200 None 2026-03-31 15:01:22,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:22,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 15:01:22,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:22,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93eb96a4c026e16349710 HTTP/1.1" 200 None 2026-03-31 15:01:22,527 - lava_callback - ERROR - Error processing callback for node 69c93eb96a4c026e16349710: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35986 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:01:47,053 - lava_callback - INFO - Received callback for node 69c93ed96a4c026e1634971b INFO: 172.19.0.1:35990 - "POST /node/69c93ed96a4c026e1634971b HTTP/1.0" 202 Accepted 2026-03-31 15:01:47,059 - lava_callback - INFO - Starting async job submit for node 69c93ed96a4c026e1634971b 2026-03-31 15:01:47,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:47,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ed96a4c026e1634971b HTTP/1.1" 200 None 2026-03-31 15:01:47,076 - lava_callback - DEBUG - Processing node 69c93ed96a4c026e1634971b: baseline-arm-baylibre 2026-03-31 15:01:47,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93ed96a4c026e1634971b 2026-03-31 15:01:47,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:47,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93ed96a4c026e1634971b HTTP/1.1" 200 None 2026-03-31 15:01:47,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:47,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:01:47,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:01:47,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93ed96a4c026e1634971b HTTP/1.1" 200 None 2026-03-31 15:01:47,131 - lava_callback - ERROR - Error processing callback for node 69c93ed96a4c026e1634971b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36928 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:02:42,592 - lava_callback - INFO - Received callback for node 69c93f026a4c026e16349728 INFO: 172.19.0.1:52590 - "POST /node/69c93f026a4c026e16349728 HTTP/1.0" 202 Accepted 2026-03-31 15:02:42,598 - lava_callback - INFO - Starting async job submit for node 69c93f026a4c026e16349728 2026-03-31 15:02:42,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:02:42,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f026a4c026e16349728 HTTP/1.1" 200 None 2026-03-31 15:02:42,619 - lava_callback - DEBUG - Processing node 69c93f026a4c026e16349728: baseline-arm-clabbe 2026-03-31 15:02:42,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93f026a4c026e16349728 2026-03-31 15:02:42,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:02:42,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f026a4c026e16349728 HTTP/1.1" 200 None 2026-03-31 15:02:42,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:02:42,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 15:02:42,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:02:42,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93f026a4c026e16349728 HTTP/1.1" 200 None 2026-03-31 15:02:42,676 - lava_callback - ERROR - Error processing callback for node 69c93f026a4c026e16349728: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60042 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:03:22,529 - lava_callback - INFO - Received callback for node 69c93f276a4c026e16349735 INFO: 172.19.0.1:38912 - "POST /node/69c93f276a4c026e16349735 HTTP/1.0" 202 Accepted 2026-03-31 15:03:22,535 - lava_callback - INFO - Starting async job submit for node 69c93f276a4c026e16349735 2026-03-31 15:03:22,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:22,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f276a4c026e16349735 HTTP/1.1" 200 None 2026-03-31 15:03:22,562 - lava_callback - DEBUG - Processing node 69c93f276a4c026e16349735: baseline-arm-clabbe 2026-03-31 15:03:22,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93f276a4c026e16349735 2026-03-31 15:03:22,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:22,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f276a4c026e16349735 HTTP/1.1" 200 None 2026-03-31 15:03:22,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:22,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:03:22,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:22,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93f276a4c026e16349735 HTTP/1.1" 200 None 2026-03-31 15:03:22,620 - lava_callback - ERROR - Error processing callback for node 69c93f276a4c026e16349735: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37912 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:03:47,205 - lava_callback - INFO - Received callback for node 69c93f476a4c026e16349742 INFO: 172.19.0.1:37922 - "POST /node/69c93f476a4c026e16349742 HTTP/1.0" 202 Accepted 2026-03-31 15:03:47,211 - lava_callback - INFO - Starting async job submit for node 69c93f476a4c026e16349742 2026-03-31 15:03:47,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:47,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f476a4c026e16349742 HTTP/1.1" 200 None 2026-03-31 15:03:47,228 - lava_callback - DEBUG - Processing node 69c93f476a4c026e16349742: baseline-arm-baylibre 2026-03-31 15:03:47,228 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c93f476a4c026e16349742 2026-03-31 15:03:47,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:47,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c93f476a4c026e16349742 HTTP/1.1" 200 None 2026-03-31 15:03:47,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:47,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:03:47,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:03:47,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c93f476a4c026e16349742 HTTP/1.1" 200 None 2026-03-31 15:03:47,282 - lava_callback - ERROR - Error processing callback for node 69c93f476a4c026e16349742: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41974 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:09:27,475 - lava_callback - INFO - Received callback for node 69c940a06a4c026e16349772 INFO: 172.19.0.1:41976 - "POST /node/69c940a06a4c026e16349772 HTTP/1.0" 202 Accepted 2026-03-31 15:09:27,480 - lava_callback - INFO - Starting async job submit for node 69c940a06a4c026e16349772 2026-03-31 15:09:27,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:09:27,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940a06a4c026e16349772 HTTP/1.1" 200 None 2026-03-31 15:09:27,496 - lava_callback - DEBUG - Processing node 69c940a06a4c026e16349772: baseline-arm-baylibre 2026-03-31 15:09:27,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c940a06a4c026e16349772 2026-03-31 15:09:27,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:09:27,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940a06a4c026e16349772 HTTP/1.1" 200 None 2026-03-31 15:09:27,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:09:27,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:09:27,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:09:27,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c940a06a4c026e16349772 HTTP/1.1" 200 None 2026-03-31 15:09:27,581 - lava_callback - ERROR - Error processing callback for node 69c940a06a4c026e16349772: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52638 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:10:22,905 - lava_callback - INFO - Received callback for node 69c940c86a4c026e1634977f INFO: 172.19.0.1:34974 - "POST /node/69c940c86a4c026e1634977f HTTP/1.0" 202 Accepted 2026-03-31 15:10:22,911 - lava_callback - INFO - Starting async job submit for node 69c940c86a4c026e1634977f 2026-03-31 15:10:22,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:22,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940c86a4c026e1634977f HTTP/1.1" 200 None 2026-03-31 15:10:22,927 - lava_callback - DEBUG - Processing node 69c940c86a4c026e1634977f: baseline-arm-clabbe 2026-03-31 15:10:22,927 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c940c86a4c026e1634977f 2026-03-31 15:10:22,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:22,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940c86a4c026e1634977f HTTP/1.1" 200 None 2026-03-31 15:10:22,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:22,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:10:22,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:22,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c940c86a4c026e1634977f HTTP/1.1" 200 None 2026-03-31 15:10:22,981 - lava_callback - ERROR - Error processing callback for node 69c940c86a4c026e1634977f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34656 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:10:47,520 - lava_callback - INFO - Received callback for node 69c940f26a4c026e16349792 INFO: 172.19.0.1:34660 - "POST /node/69c940f26a4c026e16349792 HTTP/1.0" 202 Accepted 2026-03-31 15:10:47,525 - lava_callback - INFO - Starting async job submit for node 69c940f26a4c026e16349792 2026-03-31 15:10:47,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:47,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940f26a4c026e16349792 HTTP/1.1" 200 None 2026-03-31 15:10:47,544 - lava_callback - DEBUG - Processing node 69c940f26a4c026e16349792: baseline-arm64-baylibre 2026-03-31 15:10:47,545 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c940f26a4c026e16349792 2026-03-31 15:10:47,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:47,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c940f26a4c026e16349792 HTTP/1.1" 200 None 2026-03-31 15:10:47,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:47,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:10:47,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:10:47,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c940f26a4c026e16349792 HTTP/1.1" 200 None 2026-03-31 15:10:47,604 - lava_callback - ERROR - Error processing callback for node 69c940f26a4c026e16349792: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34216 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:12:07,565 - lava_callback - INFO - Received callback for node 69c9413f6a4c026e16349797 INFO: 172.19.0.1:34230 - "POST /node/69c9413f6a4c026e16349797 HTTP/1.0" 202 Accepted 2026-03-31 15:12:07,571 - lava_callback - INFO - Starting async job submit for node 69c9413f6a4c026e16349797 2026-03-31 15:12:07,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:12:07,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9413f6a4c026e16349797 HTTP/1.1" 200 None 2026-03-31 15:12:07,588 - lava_callback - DEBUG - Processing node 69c9413f6a4c026e16349797: baseline-arm64-baylibre 2026-03-31 15:12:07,588 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9413f6a4c026e16349797 2026-03-31 15:12:07,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:12:07,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9413f6a4c026e16349797 HTTP/1.1" 200 None 2026-03-31 15:12:07,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:12:07,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:12:07,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:12:07,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9413f6a4c026e16349797 HTTP/1.1" 200 None 2026-03-31 15:12:07,678 - lava_callback - ERROR - Error processing callback for node 69c9413f6a4c026e16349797: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59254 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:13:27,692 - lava_callback - INFO - Received callback for node 69c9418a6a4c026e163497a3 INFO: 172.19.0.1:59260 - "POST /node/69c9418a6a4c026e163497a3 HTTP/1.0" 202 Accepted 2026-03-31 15:13:27,698 - lava_callback - INFO - Starting async job submit for node 69c9418a6a4c026e163497a3 2026-03-31 15:13:27,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:13:27,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9418a6a4c026e163497a3 HTTP/1.1" 200 None 2026-03-31 15:13:27,715 - lava_callback - DEBUG - Processing node 69c9418a6a4c026e163497a3: baseline-arm64-baylibre 2026-03-31 15:13:27,715 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9418a6a4c026e163497a3 2026-03-31 15:13:27,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:13:27,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9418a6a4c026e163497a3 HTTP/1.1" 200 None 2026-03-31 15:13:27,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:13:27,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:13:27,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:13:27,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9418a6a4c026e163497a3 HTTP/1.1" 200 None 2026-03-31 15:13:27,777 - lava_callback - ERROR - Error processing callback for node 69c9418a6a4c026e163497a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41698 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:14:47,814 - lava_callback - INFO - Received callback for node 69c941d66a4c026e163497a9 INFO: 172.19.0.1:41710 - "POST /node/69c941d66a4c026e163497a9 HTTP/1.0" 202 Accepted 2026-03-31 15:14:47,820 - lava_callback - INFO - Starting async job submit for node 69c941d66a4c026e163497a9 2026-03-31 15:14:47,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:14:47,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c941d66a4c026e163497a9 HTTP/1.1" 200 None 2026-03-31 15:14:47,839 - lava_callback - DEBUG - Processing node 69c941d66a4c026e163497a9: baseline-arm64-baylibre 2026-03-31 15:14:47,839 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c941d66a4c026e163497a9 2026-03-31 15:14:47,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:14:47,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c941d66a4c026e163497a9 HTTP/1.1" 200 None 2026-03-31 15:14:47,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:14:47,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:14:47,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:14:47,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c941d66a4c026e163497a9 HTTP/1.1" 200 None 2026-03-31 15:14:47,898 - lava_callback - ERROR - Error processing callback for node 69c941d66a4c026e163497a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36818 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:16:07,788 - lava_callback - INFO - Received callback for node 69c942266a4c026e163497b4 INFO: 172.19.0.1:36826 - "POST /node/69c942266a4c026e163497b4 HTTP/1.0" 202 Accepted 2026-03-31 15:16:07,793 - lava_callback - INFO - Starting async job submit for node 69c942266a4c026e163497b4 2026-03-31 15:16:07,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:16:07,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942266a4c026e163497b4 HTTP/1.1" 200 None 2026-03-31 15:16:07,809 - lava_callback - DEBUG - Processing node 69c942266a4c026e163497b4: baseline-arm64-baylibre 2026-03-31 15:16:07,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c942266a4c026e163497b4 2026-03-31 15:16:07,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:16:07,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942266a4c026e163497b4 HTTP/1.1" 200 None 2026-03-31 15:16:07,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:16:07,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:16:07,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:16:07,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c942266a4c026e163497b4 HTTP/1.1" 200 None 2026-03-31 15:16:07,868 - lava_callback - ERROR - Error processing callback for node 69c942266a4c026e163497b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59164 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:17:27,975 - lava_callback - INFO - Received callback for node 69c942736a4c026e163497bc INFO: 172.19.0.1:59174 - "POST /node/69c942736a4c026e163497bc HTTP/1.0" 202 Accepted 2026-03-31 15:17:27,981 - lava_callback - INFO - Starting async job submit for node 69c942736a4c026e163497bc 2026-03-31 15:17:27,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:17:27,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942736a4c026e163497bc HTTP/1.1" 200 None 2026-03-31 15:17:27,998 - lava_callback - DEBUG - Processing node 69c942736a4c026e163497bc: baseline-arm64-baylibre 2026-03-31 15:17:27,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c942736a4c026e163497bc 2026-03-31 15:17:27,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:17:28,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942736a4c026e163497bc HTTP/1.1" 200 None 2026-03-31 15:17:28,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:17:28,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:17:28,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:17:28,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c942736a4c026e163497bc HTTP/1.1" 200 None 2026-03-31 15:17:28,077 - lava_callback - ERROR - Error processing callback for node 69c942736a4c026e163497bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59100 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:18:43,244 - lava_callback - INFO - Received callback for node 69c942c46a4c026e163497c7 INFO: 172.19.0.1:45396 - "POST /node/69c942c46a4c026e163497c7 HTTP/1.0" 202 Accepted 2026-03-31 15:18:43,250 - lava_callback - INFO - Starting async job submit for node 69c942c46a4c026e163497c7 2026-03-31 15:18:43,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:18:43,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942c46a4c026e163497c7 HTTP/1.1" 200 None 2026-03-31 15:18:43,267 - lava_callback - DEBUG - Processing node 69c942c46a4c026e163497c7: baseline-arm64-clabbe 2026-03-31 15:18:43,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c942c46a4c026e163497c7 2026-03-31 15:18:43,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:18:43,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c942c46a4c026e163497c7 HTTP/1.1" 200 None 2026-03-31 15:18:43,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:18:43,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:18:43,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:18:43,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c942c46a4c026e163497c7 HTTP/1.1" 200 None 2026-03-31 15:18:43,342 - lava_callback - ERROR - Error processing callback for node 69c942c46a4c026e163497c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38566 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:20:03,467 - lava_callback - INFO - Received callback for node 69c943126a4c026e163497d2 INFO: 172.19.0.1:54952 - "POST /node/69c943126a4c026e163497d2 HTTP/1.0" 202 Accepted 2026-03-31 15:20:03,472 - lava_callback - INFO - Starting async job submit for node 69c943126a4c026e163497d2 2026-03-31 15:20:03,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:20:03,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943126a4c026e163497d2 HTTP/1.1" 200 None 2026-03-31 15:20:03,490 - lava_callback - DEBUG - Processing node 69c943126a4c026e163497d2: baseline-arm64-clabbe 2026-03-31 15:20:03,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c943126a4c026e163497d2 2026-03-31 15:20:03,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:20:03,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943126a4c026e163497d2 HTTP/1.1" 200 None 2026-03-31 15:20:03,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:20:03,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:20:03,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:20:03,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c943126a4c026e163497d2 HTTP/1.1" 200 None 2026-03-31 15:20:03,556 - lava_callback - ERROR - Error processing callback for node 69c943126a4c026e163497d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55438 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:21:23,288 - lava_callback - INFO - Received callback for node 69c943606a4c026e163497d9 INFO: 172.19.0.1:44712 - "POST /node/69c943606a4c026e163497d9 HTTP/1.0" 202 Accepted 2026-03-31 15:21:23,293 - lava_callback - INFO - Starting async job submit for node 69c943606a4c026e163497d9 2026-03-31 15:21:23,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:21:23,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943606a4c026e163497d9 HTTP/1.1" 200 None 2026-03-31 15:21:23,310 - lava_callback - DEBUG - Processing node 69c943606a4c026e163497d9: baseline-arm64-clabbe 2026-03-31 15:21:23,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c943606a4c026e163497d9 2026-03-31 15:21:23,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:21:23,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943606a4c026e163497d9 HTTP/1.1" 200 None 2026-03-31 15:21:23,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:21:23,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:21:23,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:21:23,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c943606a4c026e163497d9 HTTP/1.1" 200 None 2026-03-31 15:21:23,396 - lava_callback - ERROR - Error processing callback for node 69c943606a4c026e163497d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46256 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:22:43,336 - lava_callback - INFO - Received callback for node 69c943ae6a4c026e163497e6 INFO: 172.19.0.1:60568 - "POST /node/69c943ae6a4c026e163497e6 HTTP/1.0" 202 Accepted 2026-03-31 15:22:43,342 - lava_callback - INFO - Starting async job submit for node 69c943ae6a4c026e163497e6 2026-03-31 15:22:43,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:22:43,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943ae6a4c026e163497e6 HTTP/1.1" 200 None 2026-03-31 15:22:43,366 - lava_callback - DEBUG - Processing node 69c943ae6a4c026e163497e6: baseline-arm64-clabbe 2026-03-31 15:22:43,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c943ae6a4c026e163497e6 2026-03-31 15:22:43,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:22:43,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943ae6a4c026e163497e6 HTTP/1.1" 200 None 2026-03-31 15:22:43,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:22:43,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:22:43,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:22:43,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c943ae6a4c026e163497e6 HTTP/1.1" 200 None 2026-03-31 15:22:43,443 - lava_callback - ERROR - Error processing callback for node 69c943ae6a4c026e163497e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49260 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:23:43,420 - lava_callback - INFO - Received callback for node 69c943fc6a4c026e163497ed INFO: 172.19.0.1:33834 - "POST /node/69c943fc6a4c026e163497ed HTTP/1.0" 202 Accepted 2026-03-31 15:23:43,426 - lava_callback - INFO - Starting async job submit for node 69c943fc6a4c026e163497ed 2026-03-31 15:23:43,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:23:43,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943fc6a4c026e163497ed HTTP/1.1" 200 None 2026-03-31 15:23:43,446 - lava_callback - DEBUG - Processing node 69c943fc6a4c026e163497ed: baseline-arm64-clabbe 2026-03-31 15:23:43,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c943fc6a4c026e163497ed 2026-03-31 15:23:43,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:23:43,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c943fc6a4c026e163497ed HTTP/1.1" 200 None 2026-03-31 15:23:43,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:23:43,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:23:43,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:23:43,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c943fc6a4c026e163497ed HTTP/1.1" 200 None 2026-03-31 15:23:43,511 - lava_callback - ERROR - Error processing callback for node 69c943fc6a4c026e163497ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50370 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:25:03,503 - lava_callback - INFO - Received callback for node 69c9444a6a4c026e163497fb INFO: 172.19.0.1:32818 - "POST /node/69c9444a6a4c026e163497fb HTTP/1.0" 202 Accepted 2026-03-31 15:25:03,510 - lava_callback - INFO - Starting async job submit for node 69c9444a6a4c026e163497fb 2026-03-31 15:25:03,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:25:03,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9444a6a4c026e163497fb HTTP/1.1" 200 None 2026-03-31 15:25:03,527 - lava_callback - DEBUG - Processing node 69c9444a6a4c026e163497fb: baseline-arm64-clabbe 2026-03-31 15:25:03,527 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9444a6a4c026e163497fb 2026-03-31 15:25:03,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:25:03,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9444a6a4c026e163497fb HTTP/1.1" 200 None 2026-03-31 15:25:03,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:25:03,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 15:25:03,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:25:03,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9444a6a4c026e163497fb HTTP/1.1" 200 None 2026-03-31 15:25:03,613 - lava_callback - ERROR - Error processing callback for node 69c9444a6a4c026e163497fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:32820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44938 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:26:23,515 - lava_callback - INFO - Received callback for node 69c944896a4c026e16349805 INFO: 172.19.0.1:36760 - "POST /node/69c944896a4c026e16349805 HTTP/1.0" 202 Accepted 2026-03-31 15:26:23,521 - lava_callback - INFO - Starting async job submit for node 69c944896a4c026e16349805 2026-03-31 15:26:23,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:23,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944896a4c026e16349805 HTTP/1.1" 200 None 2026-03-31 15:26:23,537 - lava_callback - DEBUG - Processing node 69c944896a4c026e16349805: baseline-arm-clabbe 2026-03-31 15:26:23,537 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c944896a4c026e16349805 2026-03-31 15:26:23,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:23,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944896a4c026e16349805 HTTP/1.1" 200 None 2026-03-31 15:26:23,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:23,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:26:23,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:23,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c944896a4c026e16349805 HTTP/1.1" 200 None 2026-03-31 15:26:23,598 - lava_callback - ERROR - Error processing callback for node 69c944896a4c026e16349805: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36772 - "GET /node/69c72ca929015d0f959922b8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:46210 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:26:43,445 - lava_callback - INFO - Received callback for node 69c944ae6a4c026e16349814 INFO: 172.19.0.1:35304 - "POST /node/69c944ae6a4c026e16349814 HTTP/1.0" 202 Accepted 2026-03-31 15:26:43,450 - lava_callback - INFO - Starting async job submit for node 69c944ae6a4c026e16349814 2026-03-31 15:26:43,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:43,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944ae6a4c026e16349814 HTTP/1.1" 200 None 2026-03-31 15:26:43,468 - lava_callback - DEBUG - Processing node 69c944ae6a4c026e16349814: baseline-arm-clabbe 2026-03-31 15:26:43,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c944ae6a4c026e16349814 2026-03-31 15:26:43,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:43,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944ae6a4c026e16349814 HTTP/1.1" 200 None 2026-03-31 15:26:43,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:43,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:26:43,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:26:43,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c944ae6a4c026e16349814 HTTP/1.1" 200 None 2026-03-31 15:26:43,527 - lava_callback - ERROR - Error processing callback for node 69c944ae6a4c026e16349814: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46236 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:27:23,600 - lava_callback - INFO - Received callback for node 69c944d36a4c026e16349824 INFO: 172.19.0.1:60510 - "POST /node/69c944d36a4c026e16349824 HTTP/1.0" 202 Accepted 2026-03-31 15:27:23,606 - lava_callback - INFO - Starting async job submit for node 69c944d36a4c026e16349824 2026-03-31 15:27:23,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:27:23,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944d36a4c026e16349824 HTTP/1.1" 200 None 2026-03-31 15:27:23,622 - lava_callback - DEBUG - Processing node 69c944d36a4c026e16349824: baseline-arm-clabbe 2026-03-31 15:27:23,623 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c944d36a4c026e16349824 2026-03-31 15:27:23,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:27:23,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944d36a4c026e16349824 HTTP/1.1" 200 None 2026-03-31 15:27:23,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:27:23,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:27:23,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:27:23,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c944d36a4c026e16349824 HTTP/1.1" 200 None 2026-03-31 15:27:23,686 - lava_callback - ERROR - Error processing callback for node 69c944d36a4c026e16349824: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46062 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:28:03,550 - lava_callback - INFO - Received callback for node 69c944f96a4c026e16349830 INFO: 172.19.0.1:58144 - "POST /node/69c944f96a4c026e16349830 HTTP/1.0" 202 Accepted 2026-03-31 15:28:03,555 - lava_callback - INFO - Starting async job submit for node 69c944f96a4c026e16349830 2026-03-31 15:28:03,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:03,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944f96a4c026e16349830 HTTP/1.1" 200 None 2026-03-31 15:28:03,571 - lava_callback - DEBUG - Processing node 69c944f96a4c026e16349830: baseline-arm-clabbe 2026-03-31 15:28:03,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c944f96a4c026e16349830 2026-03-31 15:28:03,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:03,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c944f96a4c026e16349830 HTTP/1.1" 200 None 2026-03-31 15:28:03,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:03,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:28:03,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:03,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c944f96a4c026e16349830 HTTP/1.1" 200 None 2026-03-31 15:28:03,631 - lava_callback - ERROR - Error processing callback for node 69c944f96a4c026e16349830: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33860 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:28:43,628 - lava_callback - INFO - Received callback for node 69c9451d6a4c026e1634983d INFO: 172.19.0.1:35912 - "POST /node/69c9451d6a4c026e1634983d HTTP/1.0" 202 Accepted 2026-03-31 15:28:43,633 - lava_callback - INFO - Starting async job submit for node 69c9451d6a4c026e1634983d 2026-03-31 15:28:43,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:43,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9451d6a4c026e1634983d HTTP/1.1" 200 None 2026-03-31 15:28:43,670 - lava_callback - DEBUG - Processing node 69c9451d6a4c026e1634983d: baseline-arm-clabbe 2026-03-31 15:28:43,670 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9451d6a4c026e1634983d 2026-03-31 15:28:43,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:43,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9451d6a4c026e1634983d HTTP/1.1" 200 None 2026-03-31 15:28:43,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:43,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:28:43,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:28:43,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9451d6a4c026e1634983d HTTP/1.1" 200 None 2026-03-31 15:28:43,732 - lava_callback - ERROR - Error processing callback for node 69c9451d6a4c026e1634983d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52996 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:29:23,609 - lava_callback - INFO - Received callback for node 69c945436a4c026e1634984f INFO: 172.19.0.1:58418 - "POST /node/69c945436a4c026e1634984f HTTP/1.0" 202 Accepted 2026-03-31 15:29:23,614 - lava_callback - INFO - Starting async job submit for node 69c945436a4c026e1634984f 2026-03-31 15:29:23,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:29:23,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945436a4c026e1634984f HTTP/1.1" 200 None 2026-03-31 15:29:23,630 - lava_callback - DEBUG - Processing node 69c945436a4c026e1634984f: baseline-arm-clabbe 2026-03-31 15:29:23,631 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c945436a4c026e1634984f 2026-03-31 15:29:23,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:29:23,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945436a4c026e1634984f HTTP/1.1" 200 None 2026-03-31 15:29:23,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:29:23,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:29:23,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:29:23,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c945436a4c026e1634984f HTTP/1.1" 200 None 2026-03-31 15:29:23,703 - lava_callback - ERROR - Error processing callback for node 69c945436a4c026e1634984f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55368 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:30:23,684 - lava_callback - INFO - Received callback for node 69c9458a6a4c026e1634986b INFO: 172.19.0.1:43708 - "POST /node/69c9458a6a4c026e1634986b HTTP/1.0" 202 Accepted 2026-03-31 15:30:23,689 - lava_callback - INFO - Starting async job submit for node 69c9458a6a4c026e1634986b 2026-03-31 15:30:23,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:30:23,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9458a6a4c026e1634986b HTTP/1.1" 200 None 2026-03-31 15:30:23,706 - lava_callback - DEBUG - Processing node 69c9458a6a4c026e1634986b: baseline-arm-clabbe 2026-03-31 15:30:23,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9458a6a4c026e1634986b 2026-03-31 15:30:23,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:30:23,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9458a6a4c026e1634986b HTTP/1.1" 200 None 2026-03-31 15:30:23,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:30:23,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:30:23,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:30:23,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9458a6a4c026e1634986b HTTP/1.1" 200 None 2026-03-31 15:30:23,762 - lava_callback - ERROR - Error processing callback for node 69c9458a6a4c026e1634986b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34118 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:31:03,676 - lava_callback - INFO - Received callback for node 69c945af6a4c026e1634987a INFO: 172.19.0.1:51004 - "POST /node/69c945af6a4c026e1634987a HTTP/1.0" 202 Accepted 2026-03-31 15:31:03,681 - lava_callback - INFO - Starting async job submit for node 69c945af6a4c026e1634987a 2026-03-31 15:31:03,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:03,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945af6a4c026e1634987a HTTP/1.1" 200 None 2026-03-31 15:31:03,698 - lava_callback - DEBUG - Processing node 69c945af6a4c026e1634987a: baseline-arm-clabbe 2026-03-31 15:31:03,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c945af6a4c026e1634987a 2026-03-31 15:31:03,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:03,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945af6a4c026e1634987a HTTP/1.1" 200 None 2026-03-31 15:31:03,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:03,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:31:03,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:03,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c945af6a4c026e1634987a HTTP/1.1" 200 None 2026-03-31 15:31:03,762 - lava_callback - ERROR - Error processing callback for node 69c945af6a4c026e1634987a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48798 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:31:48,755 - lava_callback - INFO - Received callback for node 69c945cf6a4c026e1634988a INFO: 172.19.0.1:48802 - "POST /node/69c945cf6a4c026e1634988a HTTP/1.0" 202 Accepted 2026-03-31 15:31:48,761 - lava_callback - INFO - Starting async job submit for node 69c945cf6a4c026e1634988a 2026-03-31 15:31:48,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:48,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945cf6a4c026e1634988a HTTP/1.1" 200 None 2026-03-31 15:31:48,777 - lava_callback - DEBUG - Processing node 69c945cf6a4c026e1634988a: baseline-arm-baylibre 2026-03-31 15:31:48,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c945cf6a4c026e1634988a 2026-03-31 15:31:48,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:48,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945cf6a4c026e1634988a HTTP/1.1" 200 None 2026-03-31 15:31:48,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:48,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:31:48,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:31:48,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c945cf6a4c026e1634988a HTTP/1.1" 200 None 2026-03-31 15:31:48,842 - lava_callback - ERROR - Error processing callback for node 69c945cf6a4c026e1634988a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45336 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:32:08,599 - lava_callback - INFO - Received callback for node 69c945f36a4c026e1634989a INFO: 172.19.0.1:45344 - "POST /node/69c945f36a4c026e1634989a HTTP/1.0" 202 Accepted 2026-03-31 15:32:08,604 - lava_callback - INFO - Starting async job submit for node 69c945f36a4c026e1634989a 2026-03-31 15:32:08,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:32:08,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945f36a4c026e1634989a HTTP/1.1" 200 None 2026-03-31 15:32:08,620 - lava_callback - DEBUG - Processing node 69c945f36a4c026e1634989a: baseline-arm-baylibre 2026-03-31 15:32:08,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c945f36a4c026e1634989a 2026-03-31 15:32:08,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:32:08,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c945f36a4c026e1634989a HTTP/1.1" 200 None 2026-03-31 15:32:08,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:32:08,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:32:08,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:32:08,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c945f36a4c026e1634989a HTTP/1.1" 200 None 2026-03-31 15:32:08,677 - lava_callback - ERROR - Error processing callback for node 69c945f36a4c026e1634989a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53626 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:33:03,807 - lava_callback - INFO - Received callback for node 69c9461c6a4c026e163498ae INFO: 172.19.0.1:42678 - "POST /node/69c9461c6a4c026e163498ae HTTP/1.0" 202 Accepted 2026-03-31 15:33:03,812 - lava_callback - INFO - Starting async job submit for node 69c9461c6a4c026e163498ae 2026-03-31 15:33:03,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:03,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9461c6a4c026e163498ae HTTP/1.1" 200 None 2026-03-31 15:33:03,828 - lava_callback - DEBUG - Processing node 69c9461c6a4c026e163498ae: baseline-arm-clabbe 2026-03-31 15:33:03,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9461c6a4c026e163498ae 2026-03-31 15:33:03,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:03,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9461c6a4c026e163498ae HTTP/1.1" 200 None 2026-03-31 15:33:03,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:03,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:33:03,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:03,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9461c6a4c026e163498ae HTTP/1.1" 200 None 2026-03-31 15:33:03,879 - lava_callback - ERROR - Error processing callback for node 69c9461c6a4c026e163498ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39038 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:33:43,790 - lava_callback - INFO - Received callback for node 69c946426a4c026e163498b7 INFO: 172.19.0.1:43568 - "POST /node/69c946426a4c026e163498b7 HTTP/1.0" 202 Accepted 2026-03-31 15:33:43,795 - lava_callback - INFO - Starting async job submit for node 69c946426a4c026e163498b7 2026-03-31 15:33:43,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:43,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946426a4c026e163498b7 HTTP/1.1" 200 None 2026-03-31 15:33:43,811 - lava_callback - DEBUG - Processing node 69c946426a4c026e163498b7: baseline-arm-clabbe 2026-03-31 15:33:43,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c946426a4c026e163498b7 2026-03-31 15:33:43,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:43,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946426a4c026e163498b7 HTTP/1.1" 200 None 2026-03-31 15:33:43,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:43,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:33:43,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:33:43,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c946426a4c026e163498b7 HTTP/1.1" 200 None 2026-03-31 15:33:43,868 - lava_callback - ERROR - Error processing callback for node 69c946426a4c026e163498b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47544 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:34:03,863 - lava_callback - INFO - Received callback for node 69c946676a4c026e163498c6 INFO: 172.19.0.1:55654 - "POST /node/69c946676a4c026e163498c6 HTTP/1.0" 202 Accepted 2026-03-31 15:34:03,869 - lava_callback - INFO - Starting async job submit for node 69c946676a4c026e163498c6 2026-03-31 15:34:03,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:03,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946676a4c026e163498c6 HTTP/1.1" 200 None 2026-03-31 15:34:03,886 - lava_callback - DEBUG - Processing node 69c946676a4c026e163498c6: baseline-arm-clabbe 2026-03-31 15:34:03,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c946676a4c026e163498c6 2026-03-31 15:34:03,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:03,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946676a4c026e163498c6 HTTP/1.1" 200 None 2026-03-31 15:34:03,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:03,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:34:03,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:03,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c946676a4c026e163498c6 HTTP/1.1" 200 None 2026-03-31 15:34:03,943 - lava_callback - ERROR - Error processing callback for node 69c946676a4c026e163498c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42782 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:34:43,862 - lava_callback - INFO - Received callback for node 69c9468c6a4c026e163498d6 INFO: 172.19.0.1:43182 - "POST /node/69c9468c6a4c026e163498d6 HTTP/1.0" 202 Accepted 2026-03-31 15:34:43,867 - lava_callback - INFO - Starting async job submit for node 69c9468c6a4c026e163498d6 2026-03-31 15:34:43,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:43,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9468c6a4c026e163498d6 HTTP/1.1" 200 None 2026-03-31 15:34:43,904 - lava_callback - DEBUG - Processing node 69c9468c6a4c026e163498d6: baseline-arm-clabbe 2026-03-31 15:34:43,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9468c6a4c026e163498d6 2026-03-31 15:34:43,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:43,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9468c6a4c026e163498d6 HTTP/1.1" 200 None 2026-03-31 15:34:43,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:43,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:34:43,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:34:43,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9468c6a4c026e163498d6 HTTP/1.1" 200 None 2026-03-31 15:34:43,966 - lava_callback - ERROR - Error processing callback for node 69c9468c6a4c026e163498d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47724 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:35:23,918 - lava_callback - INFO - Received callback for node 69c946b16a4c026e163498ec INFO: 172.19.0.1:38596 - "POST /node/69c946b16a4c026e163498ec HTTP/1.0" 202 Accepted 2026-03-31 15:35:23,924 - lava_callback - INFO - Starting async job submit for node 69c946b16a4c026e163498ec 2026-03-31 15:35:23,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:35:23,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946b16a4c026e163498ec HTTP/1.1" 200 None 2026-03-31 15:35:23,941 - lava_callback - DEBUG - Processing node 69c946b16a4c026e163498ec: baseline-arm-clabbe 2026-03-31 15:35:23,941 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c946b16a4c026e163498ec 2026-03-31 15:35:23,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:35:23,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946b16a4c026e163498ec HTTP/1.1" 200 None 2026-03-31 15:35:23,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:35:23,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-31 15:35:23,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:35:24,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c946b16a4c026e163498ec HTTP/1.1" 200 None 2026-03-31 15:35:24,014 - lava_callback - ERROR - Error processing callback for node 69c946b16a4c026e163498ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35296 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:36:08,800 - lava_callback - INFO - Received callback for node 69c946dd6a4c026e163498f5 INFO: 172.19.0.1:35302 - "POST /node/69c946dd6a4c026e163498f5 HTTP/1.0" 202 Accepted 2026-03-31 15:36:08,809 - lava_callback - INFO - Starting async job submit for node 69c946dd6a4c026e163498f5 2026-03-31 15:36:08,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:36:08,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946dd6a4c026e163498f5 HTTP/1.1" 200 None 2026-03-31 15:36:08,827 - lava_callback - DEBUG - Processing node 69c946dd6a4c026e163498f5: baseline-arm64-baylibre 2026-03-31 15:36:08,827 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c946dd6a4c026e163498f5 2026-03-31 15:36:08,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:36:08,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c946dd6a4c026e163498f5 HTTP/1.1" 200 None 2026-03-31 15:36:08,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:36:08,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 15:36:08,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:36:08,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c946dd6a4c026e163498f5 HTTP/1.1" 200 None 2026-03-31 15:36:08,901 - lava_callback - ERROR - Error processing callback for node 69c946dd6a4c026e163498f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41392 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:37:28,889 - lava_callback - INFO - Received callback for node 69c9472a6a4c026e16349906 INFO: 172.19.0.1:41402 - "POST /node/69c9472a6a4c026e16349906 HTTP/1.0" 202 Accepted 2026-03-31 15:37:28,894 - lava_callback - INFO - Starting async job submit for node 69c9472a6a4c026e16349906 2026-03-31 15:37:28,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:37:28,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9472a6a4c026e16349906 HTTP/1.1" 200 None 2026-03-31 15:37:28,915 - lava_callback - DEBUG - Processing node 69c9472a6a4c026e16349906: baseline-arm64-baylibre 2026-03-31 15:37:28,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9472a6a4c026e16349906 2026-03-31 15:37:28,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:37:28,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9472a6a4c026e16349906 HTTP/1.1" 200 None 2026-03-31 15:37:28,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:37:28,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 15:37:28,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:37:28,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9472a6a4c026e16349906 HTTP/1.1" 200 None 2026-03-31 15:37:28,975 - lava_callback - ERROR - Error processing callback for node 69c9472a6a4c026e16349906: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37260 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:38:48,974 - lava_callback - INFO - Received callback for node 69c947776a4c026e1634990b INFO: 172.19.0.1:37276 - "POST /node/69c947776a4c026e1634990b HTTP/1.0" 202 Accepted 2026-03-31 15:38:48,980 - lava_callback - INFO - Starting async job submit for node 69c947776a4c026e1634990b 2026-03-31 15:38:48,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:38:49,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c947776a4c026e1634990b HTTP/1.1" 200 None 2026-03-31 15:38:49,001 - lava_callback - DEBUG - Processing node 69c947776a4c026e1634990b: baseline-arm64-baylibre 2026-03-31 15:38:49,001 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c947776a4c026e1634990b 2026-03-31 15:38:49,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:38:49,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c947776a4c026e1634990b HTTP/1.1" 200 None 2026-03-31 15:38:49,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:38:49,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:38:49,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:38:49,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c947776a4c026e1634990b HTTP/1.1" 200 None 2026-03-31 15:38:49,061 - lava_callback - ERROR - Error processing callback for node 69c947776a4c026e1634990b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42424 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:39:48,993 - lava_callback - INFO - Received callback for node 69c947c26a4c026e1634991d INFO: 172.19.0.1:42438 - "POST /node/69c947c26a4c026e1634991d HTTP/1.0" 202 Accepted 2026-03-31 15:39:48,999 - lava_callback - INFO - Starting async job submit for node 69c947c26a4c026e1634991d 2026-03-31 15:39:49,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:39:49,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c947c26a4c026e1634991d HTTP/1.1" 200 None 2026-03-31 15:39:49,021 - lava_callback - DEBUG - Processing node 69c947c26a4c026e1634991d: baseline-arm64-baylibre 2026-03-31 15:39:49,022 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c947c26a4c026e1634991d 2026-03-31 15:39:49,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:39:49,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c947c26a4c026e1634991d HTTP/1.1" 200 None 2026-03-31 15:39:49,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:39:49,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:39:49,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:39:49,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c947c26a4c026e1634991d HTTP/1.1" 200 None 2026-03-31 15:39:49,098 - lava_callback - ERROR - Error processing callback for node 69c947c26a4c026e1634991d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47380 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:41:09,170 - lava_callback - INFO - Received callback for node 69c948116a4c026e1634992b INFO: 172.19.0.1:47386 - "POST /node/69c948116a4c026e1634992b HTTP/1.0" 202 Accepted 2026-03-31 15:41:09,176 - lava_callback - INFO - Starting async job submit for node 69c948116a4c026e1634992b 2026-03-31 15:41:09,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:41:09,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c948116a4c026e1634992b HTTP/1.1" 200 None 2026-03-31 15:41:09,192 - lava_callback - DEBUG - Processing node 69c948116a4c026e1634992b: baseline-arm64-baylibre 2026-03-31 15:41:09,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c948116a4c026e1634992b 2026-03-31 15:41:09,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:41:09,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c948116a4c026e1634992b HTTP/1.1" 200 None 2026-03-31 15:41:09,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:41:09,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:41:09,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:41:09,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c948116a4c026e1634992b HTTP/1.1" 200 None 2026-03-31 15:41:09,268 - lava_callback - ERROR - Error processing callback for node 69c948116a4c026e1634992b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56318 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:42:29,195 - lava_callback - INFO - Received callback for node 69c9485f6a4c026e16349936 INFO: 172.19.0.1:56334 - "POST /node/69c9485f6a4c026e16349936 HTTP/1.0" 202 Accepted 2026-03-31 15:42:29,203 - lava_callback - INFO - Starting async job submit for node 69c9485f6a4c026e16349936 2026-03-31 15:42:29,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:42:29,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9485f6a4c026e16349936 HTTP/1.1" 200 None 2026-03-31 15:42:29,227 - lava_callback - DEBUG - Processing node 69c9485f6a4c026e16349936: baseline-arm64-baylibre 2026-03-31 15:42:29,227 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9485f6a4c026e16349936 2026-03-31 15:42:29,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:42:29,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9485f6a4c026e16349936 HTTP/1.1" 200 None 2026-03-31 15:42:29,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:42:29,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:42:29,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:42:29,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9485f6a4c026e16349936 HTTP/1.1" 200 None 2026-03-31 15:42:29,298 - lava_callback - ERROR - Error processing callback for node 69c9485f6a4c026e16349936: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40556 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:43:49,589 - lava_callback - INFO - Received callback for node 69c948ad6a4c026e1634993f INFO: 172.19.0.1:40560 - "POST /node/69c948ad6a4c026e1634993f HTTP/1.0" 202 Accepted 2026-03-31 15:43:49,594 - lava_callback - INFO - Starting async job submit for node 69c948ad6a4c026e1634993f 2026-03-31 15:43:49,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:43:49,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c948ad6a4c026e1634993f HTTP/1.1" 200 None 2026-03-31 15:43:49,614 - lava_callback - DEBUG - Processing node 69c948ad6a4c026e1634993f: baseline-arm64-baylibre 2026-03-31 15:43:49,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c948ad6a4c026e1634993f 2026-03-31 15:43:49,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:43:49,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c948ad6a4c026e1634993f HTTP/1.1" 200 None 2026-03-31 15:43:49,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:43:49,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:43:49,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:43:49,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c948ad6a4c026e1634993f HTTP/1.1" 200 None 2026-03-31 15:43:49,686 - lava_callback - ERROR - Error processing callback for node 69c948ad6a4c026e1634993f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45824 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:46:29,465 - lava_callback - INFO - Received callback for node 69c949426a4c026e16349955 INFO: 172.19.0.1:45828 - "POST /node/69c949426a4c026e16349955 HTTP/1.0" 202 Accepted 2026-03-31 15:46:29,471 - lava_callback - INFO - Starting async job submit for node 69c949426a4c026e16349955 2026-03-31 15:46:29,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:46:29,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949426a4c026e16349955 HTTP/1.1" 200 None 2026-03-31 15:46:29,487 - lava_callback - DEBUG - Processing node 69c949426a4c026e16349955: baseline-arm64-baylibre 2026-03-31 15:46:29,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c949426a4c026e16349955 2026-03-31 15:46:29,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:46:29,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949426a4c026e16349955 HTTP/1.1" 200 None 2026-03-31 15:46:29,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:46:29,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 15:46:29,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:46:29,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c949426a4c026e16349955 HTTP/1.1" 200 None 2026-03-31 15:46:29,577 - lava_callback - ERROR - Error processing callback for node 69c949426a4c026e16349955: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43406 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:47:49,428 - lava_callback - INFO - Received callback for node 69c949916a4c026e16349961 INFO: 172.19.0.1:43408 - "POST /node/69c949916a4c026e16349961 HTTP/1.0" 202 Accepted 2026-03-31 15:47:49,433 - lava_callback - INFO - Starting async job submit for node 69c949916a4c026e16349961 2026-03-31 15:47:49,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:47:49,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949916a4c026e16349961 HTTP/1.1" 200 None 2026-03-31 15:47:49,462 - lava_callback - DEBUG - Processing node 69c949916a4c026e16349961: baseline-arm64-baylibre 2026-03-31 15:47:49,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c949916a4c026e16349961 2026-03-31 15:47:49,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:47:49,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949916a4c026e16349961 HTTP/1.1" 200 None 2026-03-31 15:47:49,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:47:49,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 15:47:49,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:47:49,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c949916a4c026e16349961 HTTP/1.1" 200 None 2026-03-31 15:47:49,525 - lava_callback - ERROR - Error processing callback for node 69c949916a4c026e16349961: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39466 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:49:04,574 - lava_callback - INFO - Received callback for node 69c949e36a4c026e16349975 INFO: 172.19.0.1:33492 - "POST /node/69c949e36a4c026e16349975 HTTP/1.0" 202 Accepted 2026-03-31 15:49:04,580 - lava_callback - INFO - Starting async job submit for node 69c949e36a4c026e16349975 2026-03-31 15:49:04,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:49:04,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949e36a4c026e16349975 HTTP/1.1" 200 None 2026-03-31 15:49:04,619 - lava_callback - DEBUG - Processing node 69c949e36a4c026e16349975: baseline-arm64-clabbe 2026-03-31 15:49:04,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c949e36a4c026e16349975 2026-03-31 15:49:04,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:49:04,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c949e36a4c026e16349975 HTTP/1.1" 200 None 2026-03-31 15:49:04,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:49:04,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:49:04,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:49:04,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c949e36a4c026e16349975 HTTP/1.1" 200 None 2026-03-31 15:49:04,695 - lava_callback - ERROR - Error processing callback for node 69c949e36a4c026e16349975: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58800 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.19.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50932 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:50:24,583 - lava_callback - INFO - Received callback for node 69c94a306a4c026e16349981 INFO: 172.19.0.1:40362 - "POST /node/69c94a306a4c026e16349981 HTTP/1.0" 202 Accepted 2026-03-31 15:50:24,589 - lava_callback - INFO - Starting async job submit for node 69c94a306a4c026e16349981 2026-03-31 15:50:24,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:50:24,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94a306a4c026e16349981 HTTP/1.1" 200 None 2026-03-31 15:50:24,611 - lava_callback - DEBUG - Processing node 69c94a306a4c026e16349981: baseline-arm64-clabbe 2026-03-31 15:50:24,611 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94a306a4c026e16349981 2026-03-31 15:50:24,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:50:24,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94a306a4c026e16349981 HTTP/1.1" 200 None 2026-03-31 15:50:24,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:50:24,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:50:24,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:50:24,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94a306a4c026e16349981 HTTP/1.1" 200 None 2026-03-31 15:50:24,675 - lava_callback - ERROR - Error processing callback for node 69c94a306a4c026e16349981: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46224 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:51:29,748 - lava_callback - INFO - Received callback for node 69c94a756a4c026e1634998d INFO: 172.19.0.1:60484 - "POST /node/69c94a756a4c026e1634998d HTTP/1.0" 202 Accepted 2026-03-31 15:51:29,754 - lava_callback - INFO - Starting async job submit for node 69c94a756a4c026e1634998d 2026-03-31 15:51:29,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:51:29,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94a756a4c026e1634998d HTTP/1.1" 200 None 2026-03-31 15:51:29,770 - lava_callback - DEBUG - Processing node 69c94a756a4c026e1634998d: baseline-arm64-baylibre 2026-03-31 15:51:29,770 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94a756a4c026e1634998d 2026-03-31 15:51:29,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:51:29,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94a756a4c026e1634998d HTTP/1.1" 200 None 2026-03-31 15:51:29,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:51:29,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 15:51:29,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:51:29,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94a756a4c026e1634998d HTTP/1.1" 200 None 2026-03-31 15:51:29,995 - lava_callback - ERROR - Error processing callback for node 69c94a756a4c026e1634998d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53912 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:54:04,679 - lava_callback - INFO - Received callback for node 69c94b146a4c026e163499a8 INFO: 172.19.0.1:46794 - "POST /node/69c94b146a4c026e163499a8 HTTP/1.0" 202 Accepted 2026-03-31 15:54:04,685 - lava_callback - INFO - Starting async job submit for node 69c94b146a4c026e163499a8 2026-03-31 15:54:04,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:54:04,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94b146a4c026e163499a8 HTTP/1.1" 200 None 2026-03-31 15:54:04,707 - lava_callback - DEBUG - Processing node 69c94b146a4c026e163499a8: baseline-arm64-clabbe 2026-03-31 15:54:04,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94b146a4c026e163499a8 2026-03-31 15:54:04,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:54:04,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94b146a4c026e163499a8 HTTP/1.1" 200 None 2026-03-31 15:54:04,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:54:04,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:54:04,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:54:04,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94b146a4c026e163499a8 HTTP/1.1" 200 None 2026-03-31 15:54:04,773 - lava_callback - ERROR - Error processing callback for node 69c94b146a4c026e163499a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52046 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:56:44,831 - lava_callback - INFO - Received callback for node 69c94bac6a4c026e163499c2 INFO: 172.19.0.1:34978 - "POST /node/69c94bac6a4c026e163499c2 HTTP/1.0" 202 Accepted 2026-03-31 15:56:44,837 - lava_callback - INFO - Starting async job submit for node 69c94bac6a4c026e163499c2 2026-03-31 15:56:44,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:56:44,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94bac6a4c026e163499c2 HTTP/1.1" 200 None 2026-03-31 15:56:44,853 - lava_callback - DEBUG - Processing node 69c94bac6a4c026e163499c2: baseline-arm64-clabbe 2026-03-31 15:56:44,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94bac6a4c026e163499c2 2026-03-31 15:56:44,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:56:44,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94bac6a4c026e163499c2 HTTP/1.1" 200 None 2026-03-31 15:56:44,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:56:44,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:56:44,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:56:44,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94bac6a4c026e163499c2 HTTP/1.1" 200 None 2026-03-31 15:56:44,918 - lava_callback - ERROR - Error processing callback for node 69c94bac6a4c026e163499c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46456 - "GET / HTTP/1.0" 200 OK 2026-03-31 15:59:04,929 - lava_callback - INFO - Received callback for node 69c94c446a4c026e163499de INFO: 172.19.0.1:51940 - "POST /node/69c94c446a4c026e163499de HTTP/1.0" 202 Accepted 2026-03-31 15:59:04,934 - lava_callback - INFO - Starting async job submit for node 69c94c446a4c026e163499de 2026-03-31 15:59:04,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:59:04,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94c446a4c026e163499de HTTP/1.1" 200 None 2026-03-31 15:59:04,958 - lava_callback - DEBUG - Processing node 69c94c446a4c026e163499de: baseline-arm64-clabbe 2026-03-31 15:59:04,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94c446a4c026e163499de 2026-03-31 15:59:04,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:59:04,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94c446a4c026e163499de HTTP/1.1" 200 None 2026-03-31 15:59:04,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:59:04,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 15:59:04,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 15:59:05,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94c446a4c026e163499de HTTP/1.1" 200 None 2026-03-31 15:59:05,015 - lava_callback - ERROR - Error processing callback for node 69c94c446a4c026e163499de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46106 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:00:30,224 - lava_callback - INFO - Received callback for node 69c94c8d6a4c026e163499f0 INFO: 172.19.0.1:36172 - "POST /node/69c94c8d6a4c026e163499f0 HTTP/1.0" 202 Accepted 2026-03-31 16:00:30,229 - lava_callback - INFO - Starting async job submit for node 69c94c8d6a4c026e163499f0 2026-03-31 16:00:30,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:00:30,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94c8d6a4c026e163499f0 HTTP/1.1" 200 None 2026-03-31 16:00:30,261 - lava_callback - DEBUG - Processing node 69c94c8d6a4c026e163499f0: baseline-arm64-baylibre 2026-03-31 16:00:30,261 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94c8d6a4c026e163499f0 2026-03-31 16:00:30,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:00:30,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94c8d6a4c026e163499f0 HTTP/1.1" 200 None 2026-03-31 16:00:30,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:00:30,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:00:30,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:00:30,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94c8d6a4c026e163499f0 HTTP/1.1" 200 None 2026-03-31 16:00:30,313 - lava_callback - ERROR - Error processing callback for node 69c94c8d6a4c026e163499f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58978 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:01:45,076 - lava_callback - INFO - Received callback for node 69c94cdf6a4c026e163499f7 INFO: 172.19.0.1:60706 - "POST /node/69c94cdf6a4c026e163499f7 HTTP/1.0" 202 Accepted 2026-03-31 16:01:45,081 - lava_callback - INFO - Starting async job submit for node 69c94cdf6a4c026e163499f7 2026-03-31 16:01:45,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:01:45,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94cdf6a4c026e163499f7 HTTP/1.1" 200 None 2026-03-31 16:01:45,098 - lava_callback - DEBUG - Processing node 69c94cdf6a4c026e163499f7: baseline-arm64-clabbe 2026-03-31 16:01:45,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94cdf6a4c026e163499f7 2026-03-31 16:01:45,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:01:45,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94cdf6a4c026e163499f7 HTTP/1.1" 200 None 2026-03-31 16:01:45,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:01:45,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:01:45,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:01:45,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94cdf6a4c026e163499f7 HTTP/1.1" 200 None 2026-03-31 16:01:45,189 - lava_callback - ERROR - Error processing callback for node 69c94cdf6a4c026e163499f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48584 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:02:50,360 - lava_callback - INFO - Received callback for node 69c94d276a4c026e16349a0a INFO: 172.19.0.1:52232 - "POST /node/69c94d276a4c026e16349a0a HTTP/1.0" 202 Accepted 2026-03-31 16:02:50,366 - lava_callback - INFO - Starting async job submit for node 69c94d276a4c026e16349a0a 2026-03-31 16:02:50,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:02:50,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94d276a4c026e16349a0a HTTP/1.1" 200 None 2026-03-31 16:02:50,382 - lava_callback - DEBUG - Processing node 69c94d276a4c026e16349a0a: baseline-arm64-baylibre 2026-03-31 16:02:50,383 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94d276a4c026e16349a0a 2026-03-31 16:02:50,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:02:50,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94d276a4c026e16349a0a HTTP/1.1" 200 None 2026-03-31 16:02:50,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:02:50,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:02:50,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:02:50,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94d276a4c026e16349a0a HTTP/1.1" 200 None 2026-03-31 16:02:50,435 - lava_callback - ERROR - Error processing callback for node 69c94d276a4c026e16349a0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35600 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:04:10,428 - lava_callback - INFO - Received callback for node 69c94d746a4c026e16349a14 INFO: 172.19.0.1:39686 - "POST /node/69c94d746a4c026e16349a14 HTTP/1.0" 202 Accepted 2026-03-31 16:04:10,439 - lava_callback - INFO - Starting async job submit for node 69c94d746a4c026e16349a14 2026-03-31 16:04:10,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:04:10,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94d746a4c026e16349a14 HTTP/1.1" 200 None 2026-03-31 16:04:10,464 - lava_callback - DEBUG - Processing node 69c94d746a4c026e16349a14: baseline-arm64-baylibre 2026-03-31 16:04:10,465 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94d746a4c026e16349a14 2026-03-31 16:04:10,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:04:10,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94d746a4c026e16349a14 HTTP/1.1" 200 None 2026-03-31 16:04:10,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:04:10,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:04:10,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:04:10,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94d746a4c026e16349a14 HTTP/1.1" 200 None 2026-03-31 16:04:10,526 - lava_callback - ERROR - Error processing callback for node 69c94d746a4c026e16349a14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51846 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:05:30,451 - lava_callback - INFO - Received callback for node 69c94dc16a4c026e16349a23 INFO: 172.19.0.1:58734 - "POST /node/69c94dc16a4c026e16349a23 HTTP/1.0" 202 Accepted 2026-03-31 16:05:30,456 - lava_callback - INFO - Starting async job submit for node 69c94dc16a4c026e16349a23 2026-03-31 16:05:30,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:05:30,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94dc16a4c026e16349a23 HTTP/1.1" 200 None 2026-03-31 16:05:30,478 - lava_callback - DEBUG - Processing node 69c94dc16a4c026e16349a23: baseline-arm64-baylibre 2026-03-31 16:05:30,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94dc16a4c026e16349a23 2026-03-31 16:05:30,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:05:30,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94dc16a4c026e16349a23 HTTP/1.1" 200 None 2026-03-31 16:05:30,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:05:30,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 16:05:30,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:05:30,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94dc16a4c026e16349a23 HTTP/1.1" 200 None 2026-03-31 16:05:30,540 - lava_callback - ERROR - Error processing callback for node 69c94dc16a4c026e16349a23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50682 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:06:50,610 - lava_callback - INFO - Received callback for node 69c94e0c6a4c026e16349a2e INFO: 172.19.0.1:52570 - "POST /node/69c94e0c6a4c026e16349a2e HTTP/1.0" 202 Accepted 2026-03-31 16:06:50,616 - lava_callback - INFO - Starting async job submit for node 69c94e0c6a4c026e16349a2e 2026-03-31 16:06:50,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:06:50,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94e0c6a4c026e16349a2e HTTP/1.1" 200 None 2026-03-31 16:06:50,633 - lava_callback - DEBUG - Processing node 69c94e0c6a4c026e16349a2e: baseline-arm64-baylibre 2026-03-31 16:06:50,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94e0c6a4c026e16349a2e 2026-03-31 16:06:50,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:06:50,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94e0c6a4c026e16349a2e HTTP/1.1" 200 None 2026-03-31 16:06:50,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:06:50,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 16:06:50,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:06:50,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94e0c6a4c026e16349a2e HTTP/1.1" 200 None 2026-03-31 16:06:50,691 - lava_callback - ERROR - Error processing callback for node 69c94e0c6a4c026e16349a2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33100 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:08:05,351 - lava_callback - INFO - Received callback for node 69c94e5f6a4c026e16349a3e INFO: 172.19.0.1:52878 - "POST /node/69c94e5f6a4c026e16349a3e HTTP/1.0" 202 Accepted 2026-03-31 16:08:05,356 - lava_callback - INFO - Starting async job submit for node 69c94e5f6a4c026e16349a3e 2026-03-31 16:08:05,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:08:05,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94e5f6a4c026e16349a3e HTTP/1.1" 200 None 2026-03-31 16:08:05,373 - lava_callback - DEBUG - Processing node 69c94e5f6a4c026e16349a3e: baseline-arm64-clabbe 2026-03-31 16:08:05,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94e5f6a4c026e16349a3e 2026-03-31 16:08:05,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:08:05,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94e5f6a4c026e16349a3e HTTP/1.1" 200 None 2026-03-31 16:08:05,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:08:05,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:08:05,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:08:05,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94e5f6a4c026e16349a3e HTTP/1.1" 200 None 2026-03-31 16:08:05,443 - lava_callback - ERROR - Error processing callback for node 69c94e5f6a4c026e16349a3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51098 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:09:30,684 - lava_callback - INFO - Received callback for node 69c94ea86a4c026e16349a4c INFO: 172.19.0.1:48686 - "POST /node/69c94ea86a4c026e16349a4c HTTP/1.0" 202 Accepted 2026-03-31 16:09:30,689 - lava_callback - INFO - Starting async job submit for node 69c94ea86a4c026e16349a4c 2026-03-31 16:09:30,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:09:30,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94ea86a4c026e16349a4c HTTP/1.1" 200 None 2026-03-31 16:09:30,716 - lava_callback - DEBUG - Processing node 69c94ea86a4c026e16349a4c: baseline-arm64-baylibre 2026-03-31 16:09:30,716 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94ea86a4c026e16349a4c 2026-03-31 16:09:30,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:09:30,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94ea86a4c026e16349a4c HTTP/1.1" 200 None 2026-03-31 16:09:30,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:09:30,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 16:09:30,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:09:30,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94ea86a4c026e16349a4c HTTP/1.1" 200 None 2026-03-31 16:09:30,767 - lava_callback - ERROR - Error processing callback for node 69c94ea86a4c026e16349a4c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42996 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:10:45,495 - lava_callback - INFO - Received callback for node 69c94efa6a4c026e16349a5c INFO: 172.19.0.1:34984 - "POST /node/69c94efa6a4c026e16349a5c HTTP/1.0" 202 Accepted 2026-03-31 16:10:45,501 - lava_callback - INFO - Starting async job submit for node 69c94efa6a4c026e16349a5c 2026-03-31 16:10:45,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:10:45,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94efa6a4c026e16349a5c HTTP/1.1" 200 None 2026-03-31 16:10:45,542 - lava_callback - DEBUG - Processing node 69c94efa6a4c026e16349a5c: baseline-arm64-clabbe 2026-03-31 16:10:45,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94efa6a4c026e16349a5c 2026-03-31 16:10:45,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:10:45,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94efa6a4c026e16349a5c HTTP/1.1" 200 None 2026-03-31 16:10:45,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:10:45,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:10:45,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:10:45,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94efa6a4c026e16349a5c HTTP/1.1" 200 None 2026-03-31 16:10:45,595 - lava_callback - ERROR - Error processing callback for node 69c94efa6a4c026e16349a5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40668 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40684 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:12:05,567 - lava_callback - INFO - Received callback for node 69c94f486a4c026e16349a70 INFO: 172.19.0.1:55280 - "POST /node/69c94f486a4c026e16349a70 HTTP/1.0" 202 Accepted 2026-03-31 16:12:05,572 - lava_callback - INFO - Starting async job submit for node 69c94f486a4c026e16349a70 2026-03-31 16:12:05,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:12:05,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94f486a4c026e16349a70 HTTP/1.1" 200 None 2026-03-31 16:12:05,596 - lava_callback - DEBUG - Processing node 69c94f486a4c026e16349a70: baseline-arm64-clabbe 2026-03-31 16:12:05,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94f486a4c026e16349a70 2026-03-31 16:12:05,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:12:05,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94f486a4c026e16349a70 HTTP/1.1" 200 None 2026-03-31 16:12:05,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:12:05,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 16:12:05,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:12:05,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94f486a4c026e16349a70 HTTP/1.1" 200 None 2026-03-31 16:12:05,651 - lava_callback - ERROR - Error processing callback for node 69c94f486a4c026e16349a70: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41334 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:13:25,618 - lava_callback - INFO - Received callback for node 69c94f966a4c026e16349a7e INFO: 172.19.0.1:38542 - "POST /node/69c94f966a4c026e16349a7e HTTP/1.0" 202 Accepted 2026-03-31 16:13:25,626 - lava_callback - INFO - Starting async job submit for node 69c94f966a4c026e16349a7e 2026-03-31 16:13:25,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:13:25,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94f966a4c026e16349a7e HTTP/1.1" 200 None 2026-03-31 16:13:25,648 - lava_callback - DEBUG - Processing node 69c94f966a4c026e16349a7e: baseline-arm64-clabbe 2026-03-31 16:13:25,648 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c94f966a4c026e16349a7e 2026-03-31 16:13:25,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:13:25,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c94f966a4c026e16349a7e HTTP/1.1" 200 None 2026-03-31 16:13:25,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:13:25,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-31 16:13:25,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:13:25,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c94f966a4c026e16349a7e HTTP/1.1" 200 None 2026-03-31 16:13:25,724 - lava_callback - ERROR - Error processing callback for node 69c94f966a4c026e16349a7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]