INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.19.0.1:46484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48818 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60824 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:32:57,209 - lava_callback - INFO - Received callback for node 69cbf6c08df9359ff4897d72 2026-03-31 16:32:57,216 - lava_callback - INFO - Starting async job submit for node 69cbf6c08df9359ff4897d72 INFO: 172.19.0.1:60830 - "POST /node/69cbf6c08df9359ff4897d72 HTTP/1.0" 202 Accepted 2026-03-31 16:32:57,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:32:57,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf6c08df9359ff4897d72 HTTP/1.1" 200 None 2026-03-31 16:32:57,272 - lava_callback - DEBUG - Processing node 69cbf6c08df9359ff4897d72: baseline-arm 2026-03-31 16:32:57,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:32:57,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:32:57,818 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cbf6c08df9359ff4897d72/log.txt.gz 2026-03-31 16:32:57,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:32:58,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:32:58,127 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cbf6c08df9359ff4897d72/lava-logs.yaml 2026-03-31 16:32:58,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:32:58,293 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:32:58,295 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cbf6c08df9359ff4897d72/lava_callback.json.gz 2026-03-31 16:32:58,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:32:58,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf6c08df9359ff4897d72 HTTP/1.1" 200 None 2026-03-31 16:32:58,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:32:58,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4198df9359ff4897d2e HTTP/1.1" 200 None 2026-03-31 16:32:58,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:32:58,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbf6c08df9359ff4897d72 HTTP/1.1" 200 None 2026-03-31 16:32:58,404 - lava_callback - INFO - Completed processing callback for node 69cbf6c08df9359ff4897d72 INFO: 172.19.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52794 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:33:28,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:33:28,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:34056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37126 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40390 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:40:32,535 - lava_callback - INFO - Received callback for node 69c955f3f2866272ea51e007 INFO: 172.19.0.1:41896 - "POST /node/69c955f3f2866272ea51e007 HTTP/1.0" 202 Accepted 2026-03-31 16:40:32,540 - lava_callback - INFO - Starting async job submit for node 69c955f3f2866272ea51e007 2026-03-31 16:40:32,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:40:32,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c955f3f2866272ea51e007 HTTP/1.1" 200 None 2026-03-31 16:40:32,557 - lava_callback - DEBUG - Processing node 69c955f3f2866272ea51e007: baseline-arm-baylibre 2026-03-31 16:40:32,558 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c955f3f2866272ea51e007 2026-03-31 16:40:32,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:40:32,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c955f3f2866272ea51e007 HTTP/1.1" 200 None 2026-03-31 16:40:32,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:40:32,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 16:40:32,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:40:32,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c955f3f2866272ea51e007 HTTP/1.1" 200 None 2026-03-31 16:40:32,620 - lava_callback - ERROR - Error processing callback for node 69c955f3f2866272ea51e007: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57244 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:41:12,376 - lava_callback - INFO - Received callback for node 69c9561bf2866272ea51e013 INFO: 172.19.0.1:48892 - "POST /node/69c9561bf2866272ea51e013 HTTP/1.0" 202 Accepted 2026-03-31 16:41:12,385 - lava_callback - INFO - Starting async job submit for node 69c9561bf2866272ea51e013 2026-03-31 16:41:12,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:41:12,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9561bf2866272ea51e013 HTTP/1.1" 200 None 2026-03-31 16:41:12,412 - lava_callback - DEBUG - Processing node 69c9561bf2866272ea51e013: baseline-arm-baylibre 2026-03-31 16:41:12,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9561bf2866272ea51e013 2026-03-31 16:41:12,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:41:12,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9561bf2866272ea51e013 HTTP/1.1" 200 None 2026-03-31 16:41:12,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:41:12,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 16:41:12,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:41:12,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9561bf2866272ea51e013 HTTP/1.1" 200 None 2026-03-31 16:41:12,504 - lava_callback - ERROR - Error processing callback for node 69c9561bf2866272ea51e013: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48904 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48584 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:42:12,429 - lava_callback - INFO - Received callback for node 69c95662f2866272ea51e038 INFO: 172.19.0.1:44456 - "POST /node/69c95662f2866272ea51e038 HTTP/1.0" 202 Accepted 2026-03-31 16:42:12,435 - lava_callback - INFO - Starting async job submit for node 69c95662f2866272ea51e038 2026-03-31 16:42:12,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:12,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95662f2866272ea51e038 HTTP/1.1" 200 None 2026-03-31 16:42:12,451 - lava_callback - DEBUG - Processing node 69c95662f2866272ea51e038: baseline-arm-baylibre 2026-03-31 16:42:12,452 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95662f2866272ea51e038 2026-03-31 16:42:12,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:12,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95662f2866272ea51e038 HTTP/1.1" 200 None 2026-03-31 16:42:12,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:12,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:12,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:12,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95662f2866272ea51e038 HTTP/1.1" 200 None 2026-03-31 16:42:12,515 - lava_callback - ERROR - Error processing callback for node 69c95662f2866272ea51e038: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53328 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:42:32,427 - lava_callback - INFO - Received callback for node 69c95665f2866272ea51e03f INFO: 172.19.0.1:51142 - "POST /node/69c95665f2866272ea51e03f HTTP/1.0" 202 Accepted 2026-03-31 16:42:32,437 - lava_callback - INFO - Starting async job submit for node 69c95665f2866272ea51e03f 2026-03-31 16:42:32,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:32,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95665f2866272ea51e03f HTTP/1.1" 200 None 2026-03-31 16:42:32,484 - lava_callback - DEBUG - Processing node 69c95665f2866272ea51e03f: baseline-arm-baylibre 2026-03-31 16:42:32,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95665f2866272ea51e03f 2026-03-31 16:42:32,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:32,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95665f2866272ea51e03f HTTP/1.1" 200 None 2026-03-31 16:42:32,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:32,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:32,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:32,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95665f2866272ea51e03f HTTP/1.1" 200 None 2026-03-31 16:42:32,578 - lava_callback - ERROR - Error processing callback for node 69c95665f2866272ea51e03f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:42:33,118 - lava_callback - INFO - Received callback for node 69c95666f2866272ea51e042 INFO: 172.19.0.1:51144 - "POST /node/69c95666f2866272ea51e042 HTTP/1.0" 202 Accepted 2026-03-31 16:42:33,128 - lava_callback - INFO - Starting async job submit for node 69c95666f2866272ea51e042 2026-03-31 16:42:33,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:33,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95666f2866272ea51e042 HTTP/1.1" 200 None 2026-03-31 16:42:33,153 - lava_callback - DEBUG - Processing node 69c95666f2866272ea51e042: baseline-arm-baylibre 2026-03-31 16:42:33,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95666f2866272ea51e042 2026-03-31 16:42:33,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:33,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95666f2866272ea51e042 HTTP/1.1" 200 None 2026-03-31 16:42:33,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:33,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:33,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:33,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95666f2866272ea51e042 HTTP/1.1" 200 None 2026-03-31 16:42:33,292 - lava_callback - ERROR - Error processing callback for node 69c95666f2866272ea51e042: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:42:33,806 - lava_callback - INFO - Received callback for node 69c95669f2866272ea51e04d INFO: 172.19.0.1:51146 - "POST /node/69c95669f2866272ea51e04d HTTP/1.0" 202 Accepted 2026-03-31 16:42:33,816 - lava_callback - INFO - Starting async job submit for node 69c95669f2866272ea51e04d 2026-03-31 16:42:33,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:33,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95669f2866272ea51e04d HTTP/1.1" 200 None 2026-03-31 16:42:33,842 - lava_callback - DEBUG - Processing node 69c95669f2866272ea51e04d: baseline-arm-baylibre 2026-03-31 16:42:33,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95669f2866272ea51e04d 2026-03-31 16:42:33,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:33,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95669f2866272ea51e04d HTTP/1.1" 200 None 2026-03-31 16:42:33,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:33,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:33,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:33,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95669f2866272ea51e04d HTTP/1.1" 200 None 2026-03-31 16:42:33,940 - lava_callback - ERROR - Error processing callback for node 69c95669f2866272ea51e04d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:42:34,478 - lava_callback - INFO - Received callback for node 69c9566bf2866272ea51e050 INFO: 172.19.0.1:51160 - "POST /node/69c9566bf2866272ea51e050 HTTP/1.0" 202 Accepted 2026-03-31 16:42:34,484 - lava_callback - INFO - Starting async job submit for node 69c9566bf2866272ea51e050 2026-03-31 16:42:34,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:34,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9566bf2866272ea51e050 HTTP/1.1" 200 None 2026-03-31 16:42:34,503 - lava_callback - DEBUG - Processing node 69c9566bf2866272ea51e050: baseline-arm-baylibre 2026-03-31 16:42:34,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9566bf2866272ea51e050 2026-03-31 16:42:34,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:34,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9566bf2866272ea51e050 HTTP/1.1" 200 None 2026-03-31 16:42:34,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:34,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:34,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:34,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9566bf2866272ea51e050 HTTP/1.1" 200 None 2026-03-31 16:42:34,566 - lava_callback - ERROR - Error processing callback for node 69c9566bf2866272ea51e050: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:42:35,121 - lava_callback - INFO - Received callback for node 69c95664f2866272ea51e03b INFO: 172.19.0.1:51172 - "POST /node/69c95664f2866272ea51e03b HTTP/1.0" 202 Accepted 2026-03-31 16:42:35,127 - lava_callback - INFO - Starting async job submit for node 69c95664f2866272ea51e03b 2026-03-31 16:42:35,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:35,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95664f2866272ea51e03b HTTP/1.1" 200 None 2026-03-31 16:42:35,148 - lava_callback - DEBUG - Processing node 69c95664f2866272ea51e03b: baseline-arm-baylibre 2026-03-31 16:42:35,148 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95664f2866272ea51e03b 2026-03-31 16:42:35,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:35,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95664f2866272ea51e03b HTTP/1.1" 200 None 2026-03-31 16:42:35,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:35,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:35,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:35,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95664f2866272ea51e03b HTTP/1.1" 200 None 2026-03-31 16:42:35,217 - lava_callback - ERROR - Error processing callback for node 69c95664f2866272ea51e03b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57218 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:42:46,833 - lava_callback - INFO - Received callback for node 69c95672f2866272ea51e081 INFO: 172.19.0.1:57226 - "POST /node/69c95672f2866272ea51e081 HTTP/1.0" 202 Accepted 2026-03-31 16:42:46,838 - lava_callback - INFO - Starting async job submit for node 69c95672f2866272ea51e081 2026-03-31 16:42:46,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:46,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95672f2866272ea51e081 HTTP/1.1" 200 None 2026-03-31 16:42:46,855 - lava_callback - DEBUG - Processing node 69c95672f2866272ea51e081: baseline-arm-clabbe 2026-03-31 16:42:46,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95672f2866272ea51e081 2026-03-31 16:42:46,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:46,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95672f2866272ea51e081 HTTP/1.1" 200 None 2026-03-31 16:42:46,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:46,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:46,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:46,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95672f2866272ea51e081 HTTP/1.1" 200 None 2026-03-31 16:42:46,908 - lava_callback - ERROR - Error processing callback for node 69c95672f2866272ea51e081: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:42:47,503 - lava_callback - INFO - Received callback for node 69c95677f2866272ea51e08e INFO: 172.19.0.1:57228 - "POST /node/69c95677f2866272ea51e08e HTTP/1.0" 202 Accepted 2026-03-31 16:42:47,508 - lava_callback - INFO - Starting async job submit for node 69c95677f2866272ea51e08e 2026-03-31 16:42:47,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:47,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95677f2866272ea51e08e HTTP/1.1" 200 None 2026-03-31 16:42:47,523 - lava_callback - DEBUG - Processing node 69c95677f2866272ea51e08e: baseline-arm-clabbe 2026-03-31 16:42:47,524 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95677f2866272ea51e08e 2026-03-31 16:42:47,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:47,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95677f2866272ea51e08e HTTP/1.1" 200 None 2026-03-31 16:42:47,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:47,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:47,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:47,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95677f2866272ea51e08e HTTP/1.1" 200 None 2026-03-31 16:42:47,586 - lava_callback - ERROR - Error processing callback for node 69c95677f2866272ea51e08e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:42:48,238 - lava_callback - INFO - Received callback for node 69c95678f2866272ea51e093 INFO: 172.19.0.1:57234 - "POST /node/69c95678f2866272ea51e093 HTTP/1.0" 202 Accepted 2026-03-31 16:42:48,243 - lava_callback - INFO - Starting async job submit for node 69c95678f2866272ea51e093 2026-03-31 16:42:48,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:48,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95678f2866272ea51e093 HTTP/1.1" 200 None 2026-03-31 16:42:48,260 - lava_callback - DEBUG - Processing node 69c95678f2866272ea51e093: baseline-arm-clabbe 2026-03-31 16:42:48,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95678f2866272ea51e093 2026-03-31 16:42:48,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:48,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95678f2866272ea51e093 HTTP/1.1" 200 None 2026-03-31 16:42:48,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:48,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:48,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:48,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95678f2866272ea51e093 HTTP/1.1" 200 None 2026-03-31 16:42:48,330 - lava_callback - ERROR - Error processing callback for node 69c95678f2866272ea51e093: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:42:48,904 - lava_callback - INFO - Received callback for node 69c95678f2866272ea51e096 INFO: 172.19.0.1:57250 - "POST /node/69c95678f2866272ea51e096 HTTP/1.0" 202 Accepted 2026-03-31 16:42:48,909 - lava_callback - INFO - Starting async job submit for node 69c95678f2866272ea51e096 2026-03-31 16:42:48,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:48,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95678f2866272ea51e096 HTTP/1.1" 200 None 2026-03-31 16:42:48,929 - lava_callback - DEBUG - Processing node 69c95678f2866272ea51e096: baseline-arm-clabbe 2026-03-31 16:42:48,929 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95678f2866272ea51e096 2026-03-31 16:42:48,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:48,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95678f2866272ea51e096 HTTP/1.1" 200 None 2026-03-31 16:42:48,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:48,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:48,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:48,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95678f2866272ea51e096 HTTP/1.1" 200 None 2026-03-31 16:42:48,992 - lava_callback - ERROR - Error processing callback for node 69c95678f2866272ea51e096: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:42:49,577 - lava_callback - INFO - Received callback for node 69c9567bf2866272ea51e09f INFO: 172.19.0.1:57262 - "POST /node/69c9567bf2866272ea51e09f HTTP/1.0" 202 Accepted 2026-03-31 16:42:49,583 - lava_callback - INFO - Starting async job submit for node 69c9567bf2866272ea51e09f 2026-03-31 16:42:49,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:49,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9567bf2866272ea51e09f HTTP/1.1" 200 None 2026-03-31 16:42:49,598 - lava_callback - DEBUG - Processing node 69c9567bf2866272ea51e09f: baseline-arm-clabbe 2026-03-31 16:42:49,599 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9567bf2866272ea51e09f 2026-03-31 16:42:49,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:49,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9567bf2866272ea51e09f HTTP/1.1" 200 None 2026-03-31 16:42:49,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:49,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 16:42:49,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:42:49,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9567bf2866272ea51e09f HTTP/1.1" 200 None 2026-03-31 16:42:49,656 - lava_callback - ERROR - Error processing callback for node 69c9567bf2866272ea51e09f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50238 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:47:32,737 - lava_callback - INFO - Received callback for node 69c95797f2866272ea51e1df INFO: 172.19.0.1:48748 - "POST /node/69c95797f2866272ea51e1df HTTP/1.0" 202 Accepted 2026-03-31 16:47:32,743 - lava_callback - INFO - Starting async job submit for node 69c95797f2866272ea51e1df 2026-03-31 16:47:32,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:47:32,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95797f2866272ea51e1df HTTP/1.1" 200 None 2026-03-31 16:47:32,769 - lava_callback - DEBUG - Processing node 69c95797f2866272ea51e1df: baseline-arm-baylibre 2026-03-31 16:47:32,770 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95797f2866272ea51e1df 2026-03-31 16:47:32,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:47:32,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95797f2866272ea51e1df HTTP/1.1" 200 None 2026-03-31 16:47:32,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:47:32,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 16:47:32,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:47:32,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95797f2866272ea51e1df HTTP/1.1" 200 None 2026-03-31 16:47:32,830 - lava_callback - ERROR - Error processing callback for node 69c95797f2866272ea51e1df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58960 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43146 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:49:32,787 - lava_callback - INFO - Received callback for node 69c95809f2866272ea51e239 INFO: 172.19.0.1:45852 - "POST /node/69c95809f2866272ea51e239 HTTP/1.0" 202 Accepted 2026-03-31 16:49:32,793 - lava_callback - INFO - Starting async job submit for node 69c95809f2866272ea51e239 2026-03-31 16:49:32,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:49:32,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95809f2866272ea51e239 HTTP/1.1" 200 None 2026-03-31 16:49:32,810 - lava_callback - DEBUG - Processing node 69c95809f2866272ea51e239: baseline-arm-baylibre 2026-03-31 16:49:32,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95809f2866272ea51e239 2026-03-31 16:49:32,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:49:32,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95809f2866272ea51e239 HTTP/1.1" 200 None 2026-03-31 16:49:32,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:49:32,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 16:49:32,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:49:32,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95809f2866272ea51e239 HTTP/1.1" 200 None 2026-03-31 16:49:32,873 - lava_callback - ERROR - Error processing callback for node 69c95809f2866272ea51e239: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49808 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:50:32,842 - lava_callback - INFO - Received callback for node 69c95853f2866272ea51e254 INFO: 172.19.0.1:40878 - "POST /node/69c95853f2866272ea51e254 HTTP/1.0" 202 Accepted 2026-03-31 16:50:32,849 - lava_callback - INFO - Starting async job submit for node 69c95853f2866272ea51e254 2026-03-31 16:50:32,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:50:32,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95853f2866272ea51e254 HTTP/1.1" 200 None 2026-03-31 16:50:32,868 - lava_callback - DEBUG - Processing node 69c95853f2866272ea51e254: baseline-arm-baylibre 2026-03-31 16:50:32,868 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95853f2866272ea51e254 2026-03-31 16:50:32,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:50:32,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95853f2866272ea51e254 HTTP/1.1" 200 None 2026-03-31 16:50:32,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:50:32,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 16:50:32,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:50:32,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95853f2866272ea51e254 HTTP/1.1" 200 None 2026-03-31 16:50:32,929 - lava_callback - ERROR - Error processing callback for node 69c95853f2866272ea51e254: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48278 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:50:58,475 - lava_callback - INFO - Received callback for node 69cbf6c18df9359ff4897d76 INFO: 172.19.0.1:48292 - "POST /node/69cbf6c18df9359ff4897d76 HTTP/1.0" 202 Accepted 2026-03-31 16:50:58,482 - lava_callback - INFO - Starting async job submit for node 69cbf6c18df9359ff4897d76 2026-03-31 16:50:58,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:50:58,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf6c18df9359ff4897d76 HTTP/1.1" 200 None 2026-03-31 16:50:58,574 - lava_callback - DEBUG - Processing node 69cbf6c18df9359ff4897d76: ltp-smoke 2026-03-31 16:50:59,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:50:59,637 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:50:59,639 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cbf6c18df9359ff4897d76/log.txt.gz 2026-03-31 16:50:59,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:51:00,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:51:00,450 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cbf6c18df9359ff4897d76/lava-logs.yaml 2026-03-31 16:51:00,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:51:00,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:51:00,689 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cbf6c18df9359ff4897d76/lava_callback.json.gz 2026-03-31 16:51:00,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:00,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf6c18df9359ff4897d76 HTTP/1.1" 200 None 2026-03-31 16:51:00,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:00,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4198df9359ff4897d2e HTTP/1.1" 200 None 2026-03-31 16:51:00,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:01,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbf6c18df9359ff4897d76 HTTP/1.1" 200 None 2026-03-31 16:51:01,184 - lava_callback - INFO - Completed processing callback for node 69cbf6c18df9359ff4897d76 INFO: 172.19.0.1:47566 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:51:12,889 - lava_callback - INFO - Received callback for node 69c9587cf2866272ea51e25f INFO: 172.19.0.1:41500 - "POST /node/69c9587cf2866272ea51e25f HTTP/1.0" 202 Accepted 2026-03-31 16:51:12,898 - lava_callback - INFO - Starting async job submit for node 69c9587cf2866272ea51e25f 2026-03-31 16:51:12,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:12,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9587cf2866272ea51e25f HTTP/1.1" 200 None 2026-03-31 16:51:12,928 - lava_callback - DEBUG - Processing node 69c9587cf2866272ea51e25f: baseline-arm-baylibre 2026-03-31 16:51:12,928 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9587cf2866272ea51e25f 2026-03-31 16:51:12,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:12,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9587cf2866272ea51e25f HTTP/1.1" 200 None 2026-03-31 16:51:12,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:12,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 16:51:12,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:13,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9587cf2866272ea51e25f HTTP/1.1" 200 None 2026-03-31 16:51:13,034 - lava_callback - ERROR - Error processing callback for node 69c9587cf2866272ea51e25f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47140 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:51:28,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:28,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:60978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48214 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:51:54,233 - lava_callback - INFO - Received callback for node 69cbfb988df9359ff48981e1 INFO: 172.19.0.1:48226 - "POST /node/69cbfb988df9359ff48981e1 HTTP/1.0" 202 Accepted 2026-03-31 16:51:54,243 - lava_callback - INFO - Starting async job submit for node 69cbfb988df9359ff48981e1 2026-03-31 16:51:54,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:54,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb988df9359ff48981e1 HTTP/1.1" 200 None 2026-03-31 16:51:54,282 - lava_callback - DEBUG - Processing node 69cbfb988df9359ff48981e1: baseline-arm 2026-03-31 16:51:54,292 - lava_callback - WARNING - Failed to upload log 2026-03-31 16:51:54,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:51:54,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:51:54,941 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb988df9359ff48981e1/lava-logs.yaml 2026-03-31 16:51:54,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:51:55,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:51:55,408 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb988df9359ff48981e1/lava_callback.json.gz 2026-03-31 16:51:55,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:55,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb988df9359ff48981e1 HTTP/1.1" 200 None 2026-03-31 16:51:55,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:55,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:51:55,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:55,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfb988df9359ff48981e1 HTTP/1.1" 200 None 2026-03-31 16:51:55,529 - lava_callback - INFO - Completed processing callback for node 69cbfb988df9359ff48981e1 INFO: 172.19.0.1:48240 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:51:58,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:51:58,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:54534 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:52:12,997 - lava_callback - INFO - Received callback for node 69cbfb988df9359ff48981e4 INFO: 172.19.0.1:38468 - "POST /node/69cbfb988df9359ff48981e4 HTTP/1.0" 202 Accepted 2026-03-31 16:52:13,050 - lava_callback - INFO - Starting async job submit for node 69cbfb988df9359ff48981e4 2026-03-31 16:52:13,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:13,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb988df9359ff48981e4 HTTP/1.1" 200 None 2026-03-31 16:52:13,108 - lava_callback - DEBUG - Processing node 69cbfb988df9359ff48981e4: baseline-arm 2026-03-31 16:52:13,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:52:13,883 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:52:13,885 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb988df9359ff48981e4/log.txt.gz 2026-03-31 16:52:14,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:52:15,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:52:15,008 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb988df9359ff48981e4/lava-logs.yaml 2026-03-31 16:52:15,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:38482 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 16:52:15,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:52:15,842 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb988df9359ff48981e4/lava_callback.json.gz 2026-03-31 16:52:15,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:15,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb988df9359ff48981e4 HTTP/1.1" 200 None 2026-03-31 16:52:15,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:15,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:52:15,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:16,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfb988df9359ff48981e4 HTTP/1.1" 200 None 2026-03-31 16:52:16,006 - lava_callback - INFO - Completed processing callback for node 69cbfb988df9359ff48981e4 INFO: 172.19.0.1:38492 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:52:16,637 - lava_callback - INFO - Received callback for node 69cbfb9b8df9359ff48981ed INFO: 172.19.0.1:38500 - "POST /node/69cbfb9b8df9359ff48981ed HTTP/1.0" 202 Accepted 2026-03-31 16:52:16,690 - lava_callback - INFO - Starting async job submit for node 69cbfb9b8df9359ff48981ed 2026-03-31 16:52:16,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:16,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb9b8df9359ff48981ed HTTP/1.1" 200 None 2026-03-31 16:52:16,756 - lava_callback - DEBUG - Processing node 69cbfb9b8df9359ff48981ed: baseline-arm 2026-03-31 16:52:17,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:52:17,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:52:17,790 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb9b8df9359ff48981ed/log.txt.gz 2026-03-31 16:52:18,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:52:18,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:52:18,916 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb9b8df9359ff48981ed/lava-logs.yaml 2026-03-31 16:52:18,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:52:19,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:52:19,668 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb9b8df9359ff48981ed/lava_callback.json.gz 2026-03-31 16:52:19,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:19,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb9b8df9359ff48981ed HTTP/1.1" 200 None 2026-03-31 16:52:19,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:19,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:52:19,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:19,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfb9b8df9359ff48981ed HTTP/1.1" 200 None 2026-03-31 16:52:19,805 - lava_callback - INFO - Completed processing callback for node 69cbfb9b8df9359ff48981ed INFO: 172.19.0.1:44644 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:52:28,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:28,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 16:52:32,980 - lava_callback - INFO - Received callback for node 69c958c8f2866272ea51e2c2 INFO: 172.19.0.1:39588 - "POST /node/69c958c8f2866272ea51e2c2 HTTP/1.0" 202 Accepted 2026-03-31 16:52:32,989 - lava_callback - INFO - Starting async job submit for node 69c958c8f2866272ea51e2c2 2026-03-31 16:52:32,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:33,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c8f2866272ea51e2c2 HTTP/1.1" 200 None 2026-03-31 16:52:33,021 - lava_callback - DEBUG - Processing node 69c958c8f2866272ea51e2c2: baseline-arm-baylibre 2026-03-31 16:52:33,022 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c958c8f2866272ea51e2c2 2026-03-31 16:52:33,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:33,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958c8f2866272ea51e2c2 HTTP/1.1" 200 None 2026-03-31 16:52:33,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:33,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 16:52:33,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:33,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958c8f2866272ea51e2c2 HTTP/1.1" 200 None 2026-03-31 16:52:33,126 - lava_callback - ERROR - Error processing callback for node 69c958c8f2866272ea51e2c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36984 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:52:51,486 - lava_callback - INFO - Received callback for node 69cbfb9a8df9359ff48981ea INFO: 172.19.0.1:58078 - "POST /node/69cbfb9a8df9359ff48981ea HTTP/1.0" 202 Accepted 2026-03-31 16:52:51,495 - lava_callback - INFO - Starting async job submit for node 69cbfb9a8df9359ff48981ea 2026-03-31 16:52:51,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:51,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb9a8df9359ff48981ea HTTP/1.1" 200 None 2026-03-31 16:52:51,567 - lava_callback - DEBUG - Processing node 69cbfb9a8df9359ff48981ea: baseline-arm 2026-03-31 16:52:51,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:52:52,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:52:52,812 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb9a8df9359ff48981ea/log.txt.gz 2026-03-31 16:52:53,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:58090 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:52:54,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:52:54,527 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb9a8df9359ff48981ea/lava-logs.yaml 2026-03-31 16:52:54,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:52:55,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:52:55,763 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb9a8df9359ff48981ea/lava_callback.json.gz 2026-03-31 16:52:55,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:55,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb9a8df9359ff48981ea HTTP/1.1" 200 None 2026-03-31 16:52:55,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:55,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:52:55,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:55,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfb9a8df9359ff48981ea HTTP/1.1" 200 None 2026-03-31 16:52:55,920 - lava_callback - INFO - Completed processing callback for node 69cbfb9a8df9359ff48981ea INFO: 172.19.0.1:58102 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:52:58,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:52:58,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:47216 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:53:13,052 - lava_callback - INFO - Received callback for node 69c958e7f2866272ea51e33c INFO: 172.19.0.1:44056 - "POST /node/69c958e7f2866272ea51e33c HTTP/1.0" 202 Accepted 2026-03-31 16:53:13,061 - lava_callback - INFO - Starting async job submit for node 69c958e7f2866272ea51e33c 2026-03-31 16:53:13,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:13,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e7f2866272ea51e33c HTTP/1.1" 200 None 2026-03-31 16:53:13,101 - lava_callback - DEBUG - Processing node 69c958e7f2866272ea51e33c: baseline-arm64-baylibre 2026-03-31 16:53:13,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c958e7f2866272ea51e33c 2026-03-31 16:53:13,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:13,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e7f2866272ea51e33c HTTP/1.1" 200 None 2026-03-31 16:53:13,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:13,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 16:53:13,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:13,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958e7f2866272ea51e33c HTTP/1.1" 200 None 2026-03-31 16:53:13,212 - lava_callback - ERROR - Error processing callback for node 69c958e7f2866272ea51e33c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:53:13,749 - lava_callback - INFO - Received callback for node 69c958e9f2866272ea51e340 INFO: 172.19.0.1:44058 - "POST /node/69c958e9f2866272ea51e340 HTTP/1.0" 202 Accepted 2026-03-31 16:53:13,759 - lava_callback - INFO - Starting async job submit for node 69c958e9f2866272ea51e340 2026-03-31 16:53:13,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:13,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e9f2866272ea51e340 HTTP/1.1" 200 None 2026-03-31 16:53:13,787 - lava_callback - DEBUG - Processing node 69c958e9f2866272ea51e340: baseline-arm64-baylibre 2026-03-31 16:53:13,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c958e9f2866272ea51e340 2026-03-31 16:53:13,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:13,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958e9f2866272ea51e340 HTTP/1.1" 200 None 2026-03-31 16:53:13,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:13,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 16:53:13,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:13,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958e9f2866272ea51e340 HTTP/1.1" 200 None 2026-03-31 16:53:13,898 - lava_callback - ERROR - Error processing callback for node 69c958e9f2866272ea51e340: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:53:14,438 - lava_callback - INFO - Received callback for node 69c958eef2866272ea51e354 INFO: 172.19.0.1:44064 - "POST /node/69c958eef2866272ea51e354 HTTP/1.0" 202 Accepted 2026-03-31 16:53:14,447 - lava_callback - INFO - Starting async job submit for node 69c958eef2866272ea51e354 2026-03-31 16:53:14,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:14,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958eef2866272ea51e354 HTTP/1.1" 200 None 2026-03-31 16:53:14,487 - lava_callback - DEBUG - Processing node 69c958eef2866272ea51e354: baseline-arm64-baylibre 2026-03-31 16:53:14,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c958eef2866272ea51e354 2026-03-31 16:53:14,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:14,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958eef2866272ea51e354 HTTP/1.1" 200 None 2026-03-31 16:53:14,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:14,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 16:53:14,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:14,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958eef2866272ea51e354 HTTP/1.1" 200 None 2026-03-31 16:53:14,596 - lava_callback - ERROR - Error processing callback for node 69c958eef2866272ea51e354: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:53:15,127 - lava_callback - INFO - Received callback for node 69c958eff2866272ea51e357 INFO: 172.19.0.1:44078 - "POST /node/69c958eff2866272ea51e357 HTTP/1.0" 202 Accepted 2026-03-31 16:53:15,137 - lava_callback - INFO - Starting async job submit for node 69c958eff2866272ea51e357 2026-03-31 16:53:15,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:44082 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 16:53:15,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958eff2866272ea51e357 HTTP/1.1" 200 None 2026-03-31 16:53:15,170 - lava_callback - DEBUG - Processing node 69c958eff2866272ea51e357: baseline-arm64-baylibre 2026-03-31 16:53:15,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c958eff2866272ea51e357 2026-03-31 16:53:15,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:15,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c958eff2866272ea51e357 HTTP/1.1" 200 None 2026-03-31 16:53:15,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:15,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 16:53:15,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:15,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c958eff2866272ea51e357 HTTP/1.1" 200 None 2026-03-31 16:53:15,271 - lava_callback - ERROR - Error processing callback for node 69c958eff2866272ea51e357: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44090 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:53:16,657 - lava_callback - INFO - Received callback for node 69cbfb978df9359ff48981de INFO: 172.19.0.1:44092 - "POST /node/69cbfb978df9359ff48981de HTTP/1.0" 202 Accepted 2026-03-31 16:53:16,666 - lava_callback - INFO - Starting async job submit for node 69cbfb978df9359ff48981de 2026-03-31 16:53:16,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:16,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb978df9359ff48981de HTTP/1.1" 200 None 2026-03-31 16:53:16,730 - lava_callback - DEBUG - Processing node 69cbfb978df9359ff48981de: baseline-arm 2026-03-31 16:53:17,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:17,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:17,518 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb978df9359ff48981de/log.txt.gz 2026-03-31 16:53:17,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:18,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:18,705 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb978df9359ff48981de/lava-logs.yaml 2026-03-31 16:53:18,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:19,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:19,598 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb978df9359ff48981de/lava_callback.json.gz 2026-03-31 16:53:19,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:19,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb978df9359ff48981de HTTP/1.1" 200 None 2026-03-31 16:53:19,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:19,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:53:19,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:19,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfb978df9359ff48981de HTTP/1.1" 200 None 2026-03-31 16:53:19,752 - lava_callback - INFO - Completed processing callback for node 69cbfb978df9359ff48981de 2026-03-31 16:53:21,203 - lava_callback - INFO - Received callback for node 69cbfbaa8df9359ff489820d INFO: 172.19.0.1:46200 - "POST /node/69cbfbaa8df9359ff489820d HTTP/1.0" 202 Accepted 2026-03-31 16:53:21,214 - lava_callback - INFO - Starting async job submit for node 69cbfbaa8df9359ff489820d 2026-03-31 16:53:21,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:21,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbaa8df9359ff489820d HTTP/1.1" 200 None 2026-03-31 16:53:21,293 - lava_callback - DEBUG - Processing node 69cbfbaa8df9359ff489820d: baseline-arm-broonie 2026-03-31 16:53:21,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:21,825 - lava_callback - INFO - Received callback for node 69cbfbc28df9359ff489826b 2026-03-31 16:53:21,834 - lava_callback - INFO - Starting async job submit for node 69cbfbc28df9359ff489826b INFO: 172.19.0.1:46214 - "POST /node/69cbfbc28df9359ff489826b HTTP/1.0" 202 Accepted 2026-03-31 16:53:21,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:21,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbc28df9359ff489826b HTTP/1.1" 200 None 2026-03-31 16:53:21,904 - lava_callback - DEBUG - Processing node 69cbfbc28df9359ff489826b: baseline-arm-pengutronix 2026-03-31 16:53:22,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:22,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:22,177 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfbaa8df9359ff489820d/log.txt.gz 2026-03-31 16:53:22,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:22,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:22,895 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc28df9359ff489826b/log.txt.gz 2026-03-31 16:53:23,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:23,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:23,160 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfbaa8df9359ff489820d/lava-logs.yaml 2026-03-31 16:53:23,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:23,195 - lava_callback - INFO - Received callback for node 69cbfbc08df9359ff4898268 2026-03-31 16:53:23,205 - lava_callback - INFO - Starting async job submit for node 69cbfbc08df9359ff4898268 INFO: 172.19.0.1:46216 - "POST /node/69cbfbc08df9359ff4898268 HTTP/1.0" 202 Accepted 2026-03-31 16:53:23,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:23,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbc08df9359ff4898268 HTTP/1.1" 200 None 2026-03-31 16:53:23,274 - lava_callback - DEBUG - Processing node 69cbfbc08df9359ff4898268: baseline-arm-pengutronix 2026-03-31 16:53:23,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:23,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:23,875 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfbaa8df9359ff489820d/lava_callback.json.gz 2026-03-31 16:53:23,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:23,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:23,929 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc08df9359ff4898268/log.txt.gz 2026-03-31 16:53:23,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbaa8df9359ff489820d HTTP/1.1" 200 None 2026-03-31 16:53:23,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:24,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:53:24,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:24,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbaa8df9359ff489820d HTTP/1.1" 200 None 2026-03-31 16:53:24,149 - lava_callback - INFO - Completed processing callback for node 69cbfbaa8df9359ff489820d 2026-03-31 16:53:24,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:24,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:24,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc28df9359ff489826b/lava-logs.yaml 2026-03-31 16:53:24,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:25,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:25,030 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc28df9359ff489826b/lava_callback.json.gz 2026-03-31 16:53:25,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:25,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbc28df9359ff489826b HTTP/1.1" 200 None 2026-03-31 16:53:25,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:25,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:53:25,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:25,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbc28df9359ff489826b HTTP/1.1" 200 None 2026-03-31 16:53:25,170 - lava_callback - INFO - Completed processing callback for node 69cbfbc28df9359ff489826b 2026-03-31 16:53:25,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:25,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc08df9359ff4898268/lava-logs.yaml 2026-03-31 16:53:25,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:26,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:26,091 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc08df9359ff4898268/lava_callback.json.gz 2026-03-31 16:53:26,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:26,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbc08df9359ff4898268 HTTP/1.1" 200 None 2026-03-31 16:53:26,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:26,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:53:26,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:26,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbc08df9359ff4898268 HTTP/1.1" 200 None 2026-03-31 16:53:26,274 - lava_callback - INFO - Completed processing callback for node 69cbfbc08df9359ff4898268 INFO: 172.19.0.1:46218 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:53:27,204 - lava_callback - INFO - Received callback for node 69c95901f2866272ea51e37b INFO: 172.19.0.1:46234 - "POST /node/69c95901f2866272ea51e37b HTTP/1.0" 202 Accepted 2026-03-31 16:53:27,212 - lava_callback - INFO - Starting async job submit for node 69c95901f2866272ea51e37b 2026-03-31 16:53:27,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:27,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95901f2866272ea51e37b HTTP/1.1" 200 None 2026-03-31 16:53:27,244 - lava_callback - DEBUG - Processing node 69c95901f2866272ea51e37b: baseline-arm64-clabbe 2026-03-31 16:53:27,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95901f2866272ea51e37b 2026-03-31 16:53:27,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:27,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95901f2866272ea51e37b HTTP/1.1" 200 None 2026-03-31 16:53:27,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:27,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 16:53:27,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:27,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95901f2866272ea51e37b HTTP/1.1" 200 None 2026-03-31 16:53:27,333 - lava_callback - ERROR - Error processing callback for node 69c95901f2866272ea51e37b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:53:27,861 - lava_callback - INFO - Received callback for node 69c95904f2866272ea51e37e INFO: 172.19.0.1:46248 - "POST /node/69c95904f2866272ea51e37e HTTP/1.0" 202 Accepted 2026-03-31 16:53:27,870 - lava_callback - INFO - Starting async job submit for node 69c95904f2866272ea51e37e 2026-03-31 16:53:27,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:27,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95904f2866272ea51e37e HTTP/1.1" 200 None 2026-03-31 16:53:27,896 - lava_callback - DEBUG - Processing node 69c95904f2866272ea51e37e: baseline-arm64-clabbe 2026-03-31 16:53:27,897 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95904f2866272ea51e37e 2026-03-31 16:53:27,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:27,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95904f2866272ea51e37e HTTP/1.1" 200 None 2026-03-31 16:53:27,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:27,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 16:53:27,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:27,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95904f2866272ea51e37e HTTP/1.1" 200 None 2026-03-31 16:53:27,995 - lava_callback - ERROR - Error processing callback for node 69c95904f2866272ea51e37e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:53:28,526 - lava_callback - INFO - Received callback for node 69c95905f2866272ea51e39b INFO: 172.19.0.1:46250 - "POST /node/69c95905f2866272ea51e39b HTTP/1.0" 202 Accepted 2026-03-31 16:53:28,535 - lava_callback - INFO - Starting async job submit for node 69c95905f2866272ea51e39b 2026-03-31 16:53:28,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:28,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:28,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95905f2866272ea51e39b HTTP/1.1" 200 None 2026-03-31 16:53:28,572 - lava_callback - DEBUG - Processing node 69c95905f2866272ea51e39b: baseline-arm64-clabbe 2026-03-31 16:53:28,573 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95905f2866272ea51e39b 2026-03-31 16:53:28,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:28,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 16:53:28,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95905f2866272ea51e39b HTTP/1.1" 200 None 2026-03-31 16:53:28,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:28,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 16:53:28,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:28,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95905f2866272ea51e39b HTTP/1.1" 200 None 2026-03-31 16:53:28,677 - lava_callback - ERROR - Error processing callback for node 69c95905f2866272ea51e39b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:53:31,895 - lava_callback - INFO - Received callback for node 69cbfbc58df9359ff4898275 INFO: 172.19.0.1:50978 - "POST /node/69cbfbc58df9359ff4898275 HTTP/1.0" 202 Accepted 2026-03-31 16:53:31,945 - lava_callback - INFO - Starting async job submit for node 69cbfbc58df9359ff4898275 2026-03-31 16:53:31,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:32,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbc58df9359ff4898275 HTTP/1.1" 200 None 2026-03-31 16:53:32,009 - lava_callback - DEBUG - Processing node 69cbfbc58df9359ff4898275: baseline-arm-pengutronix 2026-03-31 16:53:32,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:32,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:32,733 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc58df9359ff4898275/log.txt.gz 2026-03-31 16:53:32,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:33,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:33,459 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc58df9359ff4898275/lava-logs.yaml 2026-03-31 16:53:33,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:33,943 - lava_callback - INFO - Received callback for node 69cbfbfc8df9359ff48982f4 INFO: 172.19.0.1:50984 - "POST /node/69cbfbfc8df9359ff48982f4 HTTP/1.0" 202 Accepted 2026-03-31 16:53:33,953 - lava_callback - INFO - Starting async job submit for node 69cbfbfc8df9359ff48982f4 2026-03-31 16:53:33,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:33,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbfc8df9359ff48982f4 HTTP/1.1" 200 None 2026-03-31 16:53:33,987 - lava_callback - DEBUG - Processing node 69cbfbfc8df9359ff48982f4: baseline-arm 2026-03-31 16:53:33,995 - lava_callback - WARNING - Failed to upload log 2026-03-31 16:53:34,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:34,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:34,145 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc58df9359ff4898275/lava_callback.json.gz 2026-03-31 16:53:34,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:34,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbc58df9359ff4898275 HTTP/1.1" 200 None 2026-03-31 16:53:34,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:34,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:53:34,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:34,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbc58df9359ff4898275 HTTP/1.1" 200 None 2026-03-31 16:53:34,327 - lava_callback - INFO - Completed processing callback for node 69cbfbc58df9359ff4898275 2026-03-31 16:53:34,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:34,759 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfbfc8df9359ff48982f4/lava-logs.yaml 2026-03-31 16:53:34,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:35,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:35,430 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfbfc8df9359ff48982f4/lava_callback.json.gz 2026-03-31 16:53:35,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:35,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbfc8df9359ff48982f4 HTTP/1.1" 200 None 2026-03-31 16:53:35,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:35,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:53:35,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:35,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbfc8df9359ff48982f4 HTTP/1.1" 200 None 2026-03-31 16:53:35,556 - lava_callback - INFO - Completed processing callback for node 69cbfbfc8df9359ff48982f4 INFO: 172.19.0.1:50994 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:53:38,527 - lava_callback - INFO - Received callback for node 69cbfba88df9359ff489820a INFO: 172.19.0.1:51002 - "POST /node/69cbfba88df9359ff489820a HTTP/1.0" 202 Accepted 2026-03-31 16:53:38,537 - lava_callback - INFO - Starting async job submit for node 69cbfba88df9359ff489820a 2026-03-31 16:53:38,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:38,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfba88df9359ff489820a HTTP/1.1" 200 None 2026-03-31 16:53:38,603 - lava_callback - DEBUG - Processing node 69cbfba88df9359ff489820a: baseline-arm-broonie 2026-03-31 16:53:38,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:39,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:39,586 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfba88df9359ff489820a/log.txt.gz 2026-03-31 16:53:39,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:40,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:40,727 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfba88df9359ff489820a/lava-logs.yaml 2026-03-31 16:53:40,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:41,263 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:41,266 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfba88df9359ff489820a/lava_callback.json.gz 2026-03-31 16:53:41,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:41,334 - lava_callback - INFO - Received callback for node 69cbfbb48df9359ff4898247 INFO: 172.19.0.1:45882 - "POST /node/69cbfbb48df9359ff4898247 HTTP/1.0" 202 Accepted 2026-03-31 16:53:41,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfba88df9359ff489820a HTTP/1.1" 200 None 2026-03-31 16:53:41,348 - lava_callback - INFO - Starting async job submit for node 69cbfbb48df9359ff4898247 2026-03-31 16:53:41,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:41,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:41,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:53:41,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:41,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbb48df9359ff4898247 HTTP/1.1" 200 None 2026-03-31 16:53:41,444 - lava_callback - DEBUG - Processing node 69cbfbb48df9359ff4898247: baseline-arm-clabbe 2026-03-31 16:53:41,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfba88df9359ff489820a HTTP/1.1" 200 None 2026-03-31 16:53:41,525 - lava_callback - INFO - Completed processing callback for node 69cbfba88df9359ff489820a 2026-03-31 16:53:41,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:42,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:42,366 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cbfbb48df9359ff4898247/log.txt.gz 2026-03-31 16:53:42,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:43,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:43,351 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cbfbb48df9359ff4898247/lava-logs.yaml 2026-03-31 16:53:43,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:53:43,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:53:43,905 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cbfbb48df9359ff4898247/lava_callback.json.gz 2026-03-31 16:53:43,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:43,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbb48df9359ff4898247 HTTP/1.1" 200 None 2026-03-31 16:53:43,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:44,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:53:44,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:44,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbb48df9359ff4898247 HTTP/1.1" 200 None 2026-03-31 16:53:44,097 - lava_callback - INFO - Completed processing callback for node 69cbfbb48df9359ff4898247 INFO: 172.19.0.1:45896 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:53:47,325 - lava_callback - INFO - Received callback for node 69c95906f2866272ea51e39e INFO: 172.19.0.1:45900 - "POST /node/69c95906f2866272ea51e39e HTTP/1.0" 202 Accepted 2026-03-31 16:53:47,335 - lava_callback - INFO - Starting async job submit for node 69c95906f2866272ea51e39e 2026-03-31 16:53:47,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:47,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95906f2866272ea51e39e HTTP/1.1" 200 None 2026-03-31 16:53:47,367 - lava_callback - DEBUG - Processing node 69c95906f2866272ea51e39e: baseline-arm64-clabbe 2026-03-31 16:53:47,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95906f2866272ea51e39e 2026-03-31 16:53:47,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:47,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95906f2866272ea51e39e HTTP/1.1" 200 None 2026-03-31 16:53:47,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:47,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 16:53:47,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:47,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95906f2866272ea51e39e HTTP/1.1" 200 None 2026-03-31 16:53:47,462 - lava_callback - ERROR - Error processing callback for node 69c95906f2866272ea51e39e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:53:48,018 - lava_callback - INFO - Received callback for node 69c95909f2866272ea51e3ad INFO: 172.19.0.1:45906 - "POST /node/69c95909f2866272ea51e3ad HTTP/1.0" 202 Accepted 2026-03-31 16:53:48,027 - lava_callback - INFO - Starting async job submit for node 69c95909f2866272ea51e3ad 2026-03-31 16:53:48,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:48,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95909f2866272ea51e3ad HTTP/1.1" 200 None 2026-03-31 16:53:48,062 - lava_callback - DEBUG - Processing node 69c95909f2866272ea51e3ad: baseline-arm64-clabbe 2026-03-31 16:53:48,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95909f2866272ea51e3ad 2026-03-31 16:53:48,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:48,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95909f2866272ea51e3ad HTTP/1.1" 200 None 2026-03-31 16:53:48,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:48,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 16:53:48,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:48,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95909f2866272ea51e3ad HTTP/1.1" 200 None 2026-03-31 16:53:48,153 - lava_callback - ERROR - Error processing callback for node 69c95909f2866272ea51e3ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:53:48,711 - lava_callback - INFO - Received callback for node 69c95908f2866272ea51e3a6 INFO: 172.19.0.1:45920 - "POST /node/69c95908f2866272ea51e3a6 HTTP/1.0" 202 Accepted 2026-03-31 16:53:48,722 - lava_callback - INFO - Starting async job submit for node 69c95908f2866272ea51e3a6 2026-03-31 16:53:48,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:48,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95908f2866272ea51e3a6 HTTP/1.1" 200 None 2026-03-31 16:53:48,757 - lava_callback - DEBUG - Processing node 69c95908f2866272ea51e3a6: baseline-arm64-clabbe 2026-03-31 16:53:48,757 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95908f2866272ea51e3a6 2026-03-31 16:53:48,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:48,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95908f2866272ea51e3a6 HTTP/1.1" 200 None 2026-03-31 16:53:48,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:48,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 16:53:48,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:48,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95908f2866272ea51e3a6 HTTP/1.1" 200 None 2026-03-31 16:53:48,855 - lava_callback - ERROR - Error processing callback for node 69c95908f2866272ea51e3a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:53:53,319 - lava_callback - INFO - Received callback for node 69c95913f2866272ea51e3cb INFO: 172.19.0.1:49918 - "POST /node/69c95913f2866272ea51e3cb HTTP/1.0" 202 Accepted 2026-03-31 16:53:53,330 - lava_callback - INFO - Starting async job submit for node 69c95913f2866272ea51e3cb 2026-03-31 16:53:53,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:53,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95913f2866272ea51e3cb HTTP/1.1" 200 None 2026-03-31 16:53:53,363 - lava_callback - DEBUG - Processing node 69c95913f2866272ea51e3cb: baseline-arm-baylibre 2026-03-31 16:53:53,363 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95913f2866272ea51e3cb 2026-03-31 16:53:53,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:53,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95913f2866272ea51e3cb HTTP/1.1" 200 None 2026-03-31 16:53:53,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:53,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 16:53:53,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:53,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95913f2866272ea51e3cb HTTP/1.1" 200 None 2026-03-31 16:53:53,465 - lava_callback - ERROR - Error processing callback for node 69c95913f2866272ea51e3cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49924 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:53:58,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:53:58,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.19.0.1:47304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60216 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:54:51,448 - lava_callback - INFO - Received callback for node 69cbfbe08df9359ff48982ae INFO: 172.19.0.1:42964 - "POST /node/69cbfbe08df9359ff48982ae HTTP/1.0" 202 Accepted 2026-03-31 16:54:51,458 - lava_callback - INFO - Starting async job submit for node 69cbfbe08df9359ff48982ae 2026-03-31 16:54:51,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:54:51,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbe08df9359ff48982ae HTTP/1.1" 200 None 2026-03-31 16:54:51,535 - lava_callback - DEBUG - Processing node 69cbfbe08df9359ff48982ae: ltp-dio 2026-03-31 16:54:52,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:54:52,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:54:52,652 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69cbfbe08df9359ff48982ae/log.txt.gz 2026-03-31 16:54:53,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:54:53,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:54:53,940 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69cbfbe08df9359ff48982ae/lava-logs.yaml 2026-03-31 16:54:53,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:42976 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:54:54,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:54:54,644 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69cbfbe08df9359ff48982ae/lava_callback.json.gz 2026-03-31 16:54:54,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:54:54,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbe08df9359ff48982ae HTTP/1.1" 200 None 2026-03-31 16:54:54,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:54:54,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:54:54,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:54:54,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbe08df9359ff48982ae HTTP/1.1" 200 None 2026-03-31 16:54:54,820 - lava_callback - INFO - Completed processing callback for node 69cbfbe08df9359ff48982ae INFO: 172.19.0.1:42986 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:54:58,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:54:58,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:42686 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:55:13,165 - lava_callback - INFO - Received callback for node 69c9595df2866272ea51e488 INFO: 172.19.0.1:49418 - "POST /node/69c9595df2866272ea51e488 HTTP/1.0" 202 Accepted 2026-03-31 16:55:13,176 - lava_callback - INFO - Starting async job submit for node 69c9595df2866272ea51e488 2026-03-31 16:55:13,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:55:13,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9595df2866272ea51e488 HTTP/1.1" 200 None 2026-03-31 16:55:13,212 - lava_callback - DEBUG - Processing node 69c9595df2866272ea51e488: baseline-arm-baylibre 2026-03-31 16:55:13,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9595df2866272ea51e488 2026-03-31 16:55:13,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:55:13,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9595df2866272ea51e488 HTTP/1.1" 200 None 2026-03-31 16:55:13,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:55:13,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 16:55:13,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:55:13,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9595df2866272ea51e488 HTTP/1.1" 200 None 2026-03-31 16:55:13,318 - lava_callback - ERROR - Error processing callback for node 69c9595df2866272ea51e488: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49428 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:55:20,591 - lava_callback - INFO - Received callback for node 69cbfbb98df9359ff4898253 INFO: 172.19.0.1:33140 - "POST /node/69cbfbb98df9359ff4898253 HTTP/1.0" 202 Accepted 2026-03-31 16:55:20,597 - lava_callback - INFO - Starting async job submit for node 69cbfbb98df9359ff4898253 2026-03-31 16:55:20,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:55:20,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbb98df9359ff4898253 HTTP/1.1" 200 None 2026-03-31 16:55:20,640 - lava_callback - DEBUG - Processing node 69cbfbb98df9359ff4898253: baseline-arm-clabbe 2026-03-31 16:55:20,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:55:21,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:55:21,012 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cbfbb98df9359ff4898253/log.txt.gz 2026-03-31 16:55:21,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:55:21,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:55:21,351 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cbfbb98df9359ff4898253/lava-logs.yaml 2026-03-31 16:55:21,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:55:21,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:55:21,527 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cbfbb98df9359ff4898253/lava_callback.json.gz 2026-03-31 16:55:21,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:55:21,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbb98df9359ff4898253 HTTP/1.1" 200 None 2026-03-31 16:55:21,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:55:21,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:55:21,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:55:21,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbb98df9359ff4898253 HTTP/1.1" 200 None 2026-03-31 16:55:21,613 - lava_callback - INFO - Completed processing callback for node 69cbfbb98df9359ff4898253 INFO: 172.19.0.1:33146 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:55:28,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:55:28,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40976 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:56:13,156 - lava_callback - INFO - Received callback for node 69c959a9f2866272ea51e508 INFO: 172.19.0.1:33466 - "POST /node/69c959a9f2866272ea51e508 HTTP/1.0" 202 Accepted 2026-03-31 16:56:13,161 - lava_callback - INFO - Starting async job submit for node 69c959a9f2866272ea51e508 2026-03-31 16:56:13,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:13,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c959a9f2866272ea51e508 HTTP/1.1" 200 None 2026-03-31 16:56:13,178 - lava_callback - DEBUG - Processing node 69c959a9f2866272ea51e508: baseline-arm-baylibre 2026-03-31 16:56:13,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c959a9f2866272ea51e508 2026-03-31 16:56:13,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:13,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c959a9f2866272ea51e508 HTTP/1.1" 200 None 2026-03-31 16:56:13,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:13,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 16:56:13,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:13,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c959a9f2866272ea51e508 HTTP/1.1" 200 None 2026-03-31 16:56:13,234 - lava_callback - ERROR - Error processing callback for node 69c959a9f2866272ea51e508: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34490 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:56:31,964 - lava_callback - INFO - Received callback for node 69cbfc9c8df9359ff4898388 INFO: 172.19.0.1:35638 - "POST /node/69cbfc9c8df9359ff4898388 HTTP/1.0" 202 Accepted 2026-03-31 16:56:31,971 - lava_callback - INFO - Starting async job submit for node 69cbfc9c8df9359ff4898388 2026-03-31 16:56:31,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:32,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfc9c8df9359ff4898388 HTTP/1.1" 200 None 2026-03-31 16:56:32,015 - lava_callback - DEBUG - Processing node 69cbfc9c8df9359ff4898388: baseline-arm64 2026-03-31 16:56:32,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:32,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:32,531 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9c8df9359ff4898388/log.txt.gz 2026-03-31 16:56:32,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:32,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:32,877 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9c8df9359ff4898388/lava-logs.yaml 2026-03-31 16:56:32,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:33,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:33,017 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9c8df9359ff4898388/lava_callback.json.gz 2026-03-31 16:56:33,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:33,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfc9c8df9359ff4898388 HTTP/1.1" 200 None 2026-03-31 16:56:33,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:33,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:56:33,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:33,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfc9c8df9359ff4898388 HTTP/1.1" 200 None 2026-03-31 16:56:33,099 - lava_callback - INFO - Completed processing callback for node 69cbfc9c8df9359ff4898388 INFO: 172.19.0.1:35654 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:56:44,127 - lava_callback - INFO - Received callback for node 69cbfca98df9359ff48983b9 INFO: 172.19.0.1:40158 - "POST /node/69cbfca98df9359ff48983b9 HTTP/1.0" 202 Accepted 2026-03-31 16:56:44,133 - lava_callback - INFO - Starting async job submit for node 69cbfca98df9359ff48983b9 2026-03-31 16:56:44,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:44,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca98df9359ff48983b9 HTTP/1.1" 200 None 2026-03-31 16:56:44,168 - lava_callback - DEBUG - Processing node 69cbfca98df9359ff48983b9: baseline-arm64 2026-03-31 16:56:44,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:44,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:44,457 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca98df9359ff48983b9/log.txt.gz 2026-03-31 16:56:44,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:44,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:44,777 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca98df9359ff48983b9/lava-logs.yaml 2026-03-31 16:56:44,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:45,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:45,059 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca98df9359ff48983b9/lava_callback.json.gz 2026-03-31 16:56:45,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:45,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca98df9359ff48983b9 HTTP/1.1" 200 None 2026-03-31 16:56:45,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:45,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:56:45,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:45,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfca98df9359ff48983b9 HTTP/1.1" 200 None 2026-03-31 16:56:45,181 - lava_callback - INFO - Completed processing callback for node 69cbfca98df9359ff48983b9 INFO: 172.19.0.1:40162 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:56:49,359 - lava_callback - INFO - Received callback for node 69cbfcb98df9359ff48983e2 INFO: 172.19.0.1:40166 - "POST /node/69cbfcb98df9359ff48983e2 HTTP/1.0" 202 Accepted 2026-03-31 16:56:49,366 - lava_callback - INFO - Starting async job submit for node 69cbfcb98df9359ff48983e2 2026-03-31 16:56:49,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:49,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb98df9359ff48983e2 HTTP/1.1" 200 None 2026-03-31 16:56:49,391 - lava_callback - DEBUG - Processing node 69cbfcb98df9359ff48983e2: baseline-nfs-arm64 2026-03-31 16:56:49,392 - lava_callback - WARNING - Failed to upload log 2026-03-31 16:56:49,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:49,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:49,518 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb98df9359ff48983e2/lava-logs.yaml 2026-03-31 16:56:49,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:49,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:49,647 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb98df9359ff48983e2/lava_callback.json.gz 2026-03-31 16:56:49,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:49,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb98df9359ff48983e2 HTTP/1.1" 200 None 2026-03-31 16:56:49,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:49,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:56:49,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:49,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcb98df9359ff48983e2 HTTP/1.1" 200 None 2026-03-31 16:56:49,707 - lava_callback - INFO - Completed processing callback for node 69cbfcb98df9359ff48983e2 2026-03-31 16:56:53,185 - lava_callback - INFO - Received callback for node 69c959cff2866272ea51e51c INFO: 172.19.0.1:53846 - "POST /node/69c959cff2866272ea51e51c HTTP/1.0" 202 Accepted 2026-03-31 16:56:53,190 - lava_callback - INFO - Starting async job submit for node 69c959cff2866272ea51e51c 2026-03-31 16:56:53,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:53,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c959cff2866272ea51e51c HTTP/1.1" 200 None 2026-03-31 16:56:53,207 - lava_callback - DEBUG - Processing node 69c959cff2866272ea51e51c: baseline-arm-baylibre 2026-03-31 16:56:53,208 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c959cff2866272ea51e51c 2026-03-31 16:56:53,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:53,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c959cff2866272ea51e51c HTTP/1.1" 200 None 2026-03-31 16:56:53,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:53,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 16:56:53,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:53,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c959cff2866272ea51e51c HTTP/1.1" 200 None 2026-03-31 16:56:53,262 - lava_callback - ERROR - Error processing callback for node 69c959cff2866272ea51e51c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53862 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:56:58,017 - lava_callback - INFO - Received callback for node 69cbfca18df9359ff48983a3 INFO: 172.19.0.1:53864 - "POST /node/69cbfca18df9359ff48983a3 HTTP/1.0" 202 Accepted 2026-03-31 16:56:58,024 - lava_callback - INFO - Starting async job submit for node 69cbfca18df9359ff48983a3 2026-03-31 16:56:58,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:58,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca18df9359ff48983a3 HTTP/1.1" 200 None 2026-03-31 16:56:58,071 - lava_callback - DEBUG - Processing node 69cbfca18df9359ff48983a3: baseline-arm64 2026-03-31 16:56:58,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:58,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:58,598 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca18df9359ff48983a3/log.txt.gz 2026-03-31 16:56:58,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:58,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:58,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 16:56:58,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:58,900 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca18df9359ff48983a3/lava-logs.yaml 2026-03-31 16:56:58,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:56:59,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:56:59,027 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca18df9359ff48983a3/lava_callback.json.gz 2026-03-31 16:56:59,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:59,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca18df9359ff48983a3 HTTP/1.1" 200 None 2026-03-31 16:56:59,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:59,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:56:59,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:59,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfca18df9359ff48983a3 HTTP/1.1" 200 None 2026-03-31 16:56:59,235 - lava_callback - INFO - Completed processing callback for node 69cbfca18df9359ff48983a3 2026-03-31 16:56:59,506 - lava_callback - INFO - Received callback for node 69cbfbbb8df9359ff4898256 INFO: 172.19.0.1:53870 - "POST /node/69cbfbbb8df9359ff4898256 HTTP/1.0" 202 Accepted 2026-03-31 16:56:59,512 - lava_callback - INFO - Starting async job submit for node 69cbfbbb8df9359ff4898256 2026-03-31 16:56:59,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:56:59,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbbb8df9359ff4898256 HTTP/1.1" 200 None 2026-03-31 16:56:59,557 - lava_callback - DEBUG - Processing node 69cbfbbb8df9359ff4898256: baseline-arm-clabbe 2026-03-31 16:56:59,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:00,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:00,003 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cbfbbb8df9359ff4898256/log.txt.gz 2026-03-31 16:57:00,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:00,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:00,349 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cbfbbb8df9359ff4898256/lava-logs.yaml 2026-03-31 16:57:00,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:00,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:00,478 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cbfbbb8df9359ff4898256/lava_callback.json.gz 2026-03-31 16:57:00,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:00,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbbb8df9359ff4898256 HTTP/1.1" 200 None 2026-03-31 16:57:00,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:00,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:57:00,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:00,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbbb8df9359ff4898256 HTTP/1.1" 200 None 2026-03-31 16:57:00,570 - lava_callback - INFO - Completed processing callback for node 69cbfbbb8df9359ff4898256 INFO: 172.19.0.1:44386 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:57:09,642 - lava_callback - INFO - Received callback for node 69cbfc9e8df9359ff489838e INFO: 172.19.0.1:44390 - "POST /node/69cbfc9e8df9359ff489838e HTTP/1.0" 202 Accepted 2026-03-31 16:57:09,648 - lava_callback - INFO - Starting async job submit for node 69cbfc9e8df9359ff489838e 2026-03-31 16:57:09,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:09,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfc9e8df9359ff489838e HTTP/1.1" 200 None 2026-03-31 16:57:09,690 - lava_callback - DEBUG - Processing node 69cbfc9e8df9359ff489838e: baseline-arm64 2026-03-31 16:57:10,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:10,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:10,171 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9e8df9359ff489838e/log.txt.gz 2026-03-31 16:57:10,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:10,474 - lava_callback - INFO - Received callback for node 69cbfca48df9359ff48983a9 INFO: 172.19.0.1:49618 - "POST /node/69cbfca48df9359ff48983a9 HTTP/1.0" 202 Accepted 2026-03-31 16:57:10,481 - lava_callback - INFO - Starting async job submit for node 69cbfca48df9359ff48983a9 2026-03-31 16:57:10,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:10,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca48df9359ff48983a9 HTTP/1.1" 200 None 2026-03-31 16:57:10,531 - lava_callback - DEBUG - Processing node 69cbfca48df9359ff48983a9: baseline-arm64 2026-03-31 16:57:10,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:10,676 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9e8df9359ff489838e/lava-logs.yaml 2026-03-31 16:57:10,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:10,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:10,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:10,973 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9e8df9359ff489838e/lava_callback.json.gz 2026-03-31 16:57:10,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:10,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:10,998 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca48df9359ff48983a9/log.txt.gz 2026-03-31 16:57:11,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfc9e8df9359ff489838e HTTP/1.1" 200 None 2026-03-31 16:57:11,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:11,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:11,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:11,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:11,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfc9e8df9359ff489838e HTTP/1.1" 200 None 2026-03-31 16:57:11,239 - lava_callback - INFO - Completed processing callback for node 69cbfc9e8df9359ff489838e 2026-03-31 16:57:11,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:11,466 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca48df9359ff48983a9/lava-logs.yaml 2026-03-31 16:57:11,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:11,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:11,649 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca48df9359ff48983a9/lava_callback.json.gz 2026-03-31 16:57:11,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:11,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca48df9359ff48983a9 HTTP/1.1" 200 None 2026-03-31 16:57:11,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:11,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:11,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:11,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfca48df9359ff48983a9 HTTP/1.1" 200 None 2026-03-31 16:57:11,744 - lava_callback - INFO - Completed processing callback for node 69cbfca48df9359ff48983a9 INFO: 172.19.0.1:49622 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 16:57:16,453 - lava_callback - INFO - Received callback for node 69cbfcaa8df9359ff48983bc INFO: 172.19.0.1:49636 - "POST /node/69cbfcaa8df9359ff48983bc HTTP/1.0" 202 Accepted 2026-03-31 16:57:16,459 - lava_callback - INFO - Starting async job submit for node 69cbfcaa8df9359ff48983bc 2026-03-31 16:57:16,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:16,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcaa8df9359ff48983bc HTTP/1.1" 200 None 2026-03-31 16:57:16,497 - lava_callback - DEBUG - Processing node 69cbfcaa8df9359ff48983bc: baseline-arm64 INFO: 172.19.0.1:49648 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:57:16,657 - lava_callback - INFO - Received callback for node 69cbfcae8df9359ff48983c7 INFO: 172.19.0.1:49658 - "POST /node/69cbfcae8df9359ff48983c7 HTTP/1.0" 202 Accepted 2026-03-31 16:57:16,685 - lava_callback - INFO - Starting async job submit for node 69cbfcae8df9359ff48983c7 2026-03-31 16:57:16,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:16,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:16,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcae8df9359ff48983c7 HTTP/1.1" 200 None 2026-03-31 16:57:16,783 - lava_callback - DEBUG - Processing node 69cbfcae8df9359ff48983c7: baseline-arm64 2026-03-31 16:57:16,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:17,005 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcaa8df9359ff48983bc/log.txt.gz 2026-03-31 16:57:17,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:17,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:17,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:17,357 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcae8df9359ff48983c7/log.txt.gz 2026-03-31 16:57:17,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:17,462 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcaa8df9359ff48983bc/lava-logs.yaml 2026-03-31 16:57:17,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:17,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:17,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:17,677 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcaa8df9359ff48983bc/lava_callback.json.gz 2026-03-31 16:57:17,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:17,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcaa8df9359ff48983bc HTTP/1.1" 200 None 2026-03-31 16:57:17,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:17,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:17,723 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcae8df9359ff48983c7/lava-logs.yaml 2026-03-31 16:57:17,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:17,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:17,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:17,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcaa8df9359ff48983bc HTTP/1.1" 200 None 2026-03-31 16:57:17,767 - lava_callback - INFO - Completed processing callback for node 69cbfcaa8df9359ff48983bc 2026-03-31 16:57:17,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:17,913 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcae8df9359ff48983c7/lava_callback.json.gz 2026-03-31 16:57:17,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:17,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcae8df9359ff48983c7 HTTP/1.1" 200 None 2026-03-31 16:57:17,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:17,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:17,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:18,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcae8df9359ff48983c7 HTTP/1.1" 200 None 2026-03-31 16:57:18,019 - lava_callback - INFO - Completed processing callback for node 69cbfcae8df9359ff48983c7 2026-03-31 16:57:21,856 - lava_callback - INFO - Received callback for node 69cbfca78df9359ff48983b3 INFO: 172.19.0.1:58588 - "POST /node/69cbfca78df9359ff48983b3 HTTP/1.0" 202 Accepted 2026-03-31 16:57:21,863 - lava_callback - INFO - Starting async job submit for node 69cbfca78df9359ff48983b3 2026-03-31 16:57:21,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:21,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca78df9359ff48983b3 HTTP/1.1" 200 None 2026-03-31 16:57:21,920 - lava_callback - DEBUG - Processing node 69cbfca78df9359ff48983b3: baseline-arm64 2026-03-31 16:57:22,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:22,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:22,631 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca78df9359ff48983b3/log.txt.gz 2026-03-31 16:57:22,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:23,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:23,134 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca78df9359ff48983b3/lava-logs.yaml 2026-03-31 16:57:23,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:23,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:23,294 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca78df9359ff48983b3/lava_callback.json.gz 2026-03-31 16:57:23,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:23,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca78df9359ff48983b3 HTTP/1.1" 200 None 2026-03-31 16:57:23,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:23,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:23,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:23,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfca78df9359ff48983b3 HTTP/1.1" 200 None 2026-03-31 16:57:23,396 - lava_callback - INFO - Completed processing callback for node 69cbfca78df9359ff48983b3 2026-03-31 16:57:25,339 - lava_callback - INFO - Received callback for node 69cbfca08df9359ff48983a0 INFO: 172.19.0.1:58590 - "POST /node/69cbfca08df9359ff48983a0 HTTP/1.0" 202 Accepted 2026-03-31 16:57:25,349 - lava_callback - INFO - Starting async job submit for node 69cbfca08df9359ff48983a0 2026-03-31 16:57:25,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:25,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca08df9359ff48983a0 HTTP/1.1" 200 None 2026-03-31 16:57:25,412 - lava_callback - DEBUG - Processing node 69cbfca08df9359ff48983a0: baseline-arm64 2026-03-31 16:57:25,651 - lava_callback - WARNING - Failed to upload log 2026-03-31 16:57:25,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:25,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:25,879 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca08df9359ff48983a0/lava-logs.yaml 2026-03-31 16:57:25,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:26,042 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:26,044 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca08df9359ff48983a0/lava_callback.json.gz 2026-03-31 16:57:26,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:26,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca08df9359ff48983a0 HTTP/1.1" 200 None 2026-03-31 16:57:26,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:26,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:26,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:26,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfca08df9359ff48983a0 HTTP/1.1" 200 None 2026-03-31 16:57:26,199 - lava_callback - INFO - Completed processing callback for node 69cbfca08df9359ff48983a0 INFO: 172.19.0.1:58604 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:57:28,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:28,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 16:57:33,480 - lava_callback - INFO - Received callback for node 69cbfc9d8df9359ff489838b INFO: 172.19.0.1:45110 - "POST /node/69cbfc9d8df9359ff489838b HTTP/1.0" 202 Accepted 2026-03-31 16:57:33,486 - lava_callback - INFO - Starting async job submit for node 69cbfc9d8df9359ff489838b 2026-03-31 16:57:33,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:33,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfc9d8df9359ff489838b HTTP/1.1" 200 None 2026-03-31 16:57:33,524 - lava_callback - DEBUG - Processing node 69cbfc9d8df9359ff489838b: baseline-arm64 2026-03-31 16:57:33,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:33,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:33,869 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9d8df9359ff489838b/log.txt.gz 2026-03-31 16:57:34,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:34,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:34,181 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9d8df9359ff489838b/lava-logs.yaml 2026-03-31 16:57:34,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:34,324 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:34,326 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9d8df9359ff489838b/lava_callback.json.gz 2026-03-31 16:57:34,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:34,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfc9d8df9359ff489838b HTTP/1.1" 200 None 2026-03-31 16:57:34,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:34,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:34,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:34,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfc9d8df9359ff489838b HTTP/1.1" 200 None 2026-03-31 16:57:34,408 - lava_callback - INFO - Completed processing callback for node 69cbfc9d8df9359ff489838b INFO: 172.19.0.1:45116 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:57:37,829 - lava_callback - INFO - Received callback for node 69cbfc9f8df9359ff4898391 INFO: 172.19.0.1:45132 - "POST /node/69cbfc9f8df9359ff4898391 HTTP/1.0" 202 Accepted 2026-03-31 16:57:37,835 - lava_callback - INFO - Starting async job submit for node 69cbfc9f8df9359ff4898391 2026-03-31 16:57:37,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:37,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfc9f8df9359ff4898391 HTTP/1.1" 200 None 2026-03-31 16:57:37,882 - lava_callback - DEBUG - Processing node 69cbfc9f8df9359ff4898391: baseline-arm64 2026-03-31 16:57:38,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:38,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:38,412 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9f8df9359ff4898391/log.txt.gz 2026-03-31 16:57:38,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:38,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:38,841 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9f8df9359ff4898391/lava-logs.yaml 2026-03-31 16:57:38,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:39,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:39,085 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfc9f8df9359ff4898391/lava_callback.json.gz 2026-03-31 16:57:39,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:39,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfc9f8df9359ff4898391 HTTP/1.1" 200 None 2026-03-31 16:57:39,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:39,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:39,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:39,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfc9f8df9359ff4898391 HTTP/1.1" 200 None 2026-03-31 16:57:39,368 - lava_callback - INFO - Completed processing callback for node 69cbfc9f8df9359ff4898391 2026-03-31 16:57:39,692 - lava_callback - INFO - Received callback for node 69cbfcad8df9359ff48983c4 INFO: 172.19.0.1:45140 - "POST /node/69cbfcad8df9359ff48983c4 HTTP/1.0" 202 Accepted 2026-03-31 16:57:39,699 - lava_callback - INFO - Starting async job submit for node 69cbfcad8df9359ff48983c4 2026-03-31 16:57:39,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:39,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcad8df9359ff48983c4 HTTP/1.1" 200 None 2026-03-31 16:57:39,738 - lava_callback - DEBUG - Processing node 69cbfcad8df9359ff48983c4: baseline-arm64 2026-03-31 16:57:40,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:40,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:40,155 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcad8df9359ff48983c4/log.txt.gz 2026-03-31 16:57:40,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:40,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:40,556 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcad8df9359ff48983c4/lava-logs.yaml 2026-03-31 16:57:40,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:40,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:40,736 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcad8df9359ff48983c4/lava_callback.json.gz 2026-03-31 16:57:40,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:40,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcad8df9359ff48983c4 HTTP/1.1" 200 None 2026-03-31 16:57:40,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:40,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:40,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:40,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcad8df9359ff48983c4 HTTP/1.1" 200 None 2026-03-31 16:57:40,831 - lava_callback - INFO - Completed processing callback for node 69cbfcad8df9359ff48983c4 INFO: 172.19.0.1:52602 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:57:52,342 - lava_callback - INFO - Received callback for node 69cbfcb78df9359ff48983dc INFO: 172.19.0.1:43620 - "POST /node/69cbfcb78df9359ff48983dc HTTP/1.0" 202 Accepted 2026-03-31 16:57:52,352 - lava_callback - INFO - Starting async job submit for node 69cbfcb78df9359ff48983dc 2026-03-31 16:57:52,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:52,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb78df9359ff48983dc HTTP/1.1" 200 None 2026-03-31 16:57:52,404 - lava_callback - DEBUG - Processing node 69cbfcb78df9359ff48983dc: baseline-nfs-arm64 2026-03-31 16:57:53,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:53,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:53,132 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb78df9359ff48983dc/log.txt.gz 2026-03-31 16:57:53,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:53,688 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:53,691 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb78df9359ff48983dc/lava-logs.yaml 2026-03-31 16:57:53,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:53,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:53,873 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb78df9359ff48983dc/lava_callback.json.gz 2026-03-31 16:57:53,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:53,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb78df9359ff48983dc HTTP/1.1" 200 None 2026-03-31 16:57:53,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:53,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:53,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:53,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcb78df9359ff48983dc HTTP/1.1" 200 None 2026-03-31 16:57:53,979 - lava_callback - INFO - Completed processing callback for node 69cbfcb78df9359ff48983dc INFO: 172.19.0.1:43622 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:57:56,107 - lava_callback - INFO - Received callback for node 69cbfcbf8df9359ff489840b INFO: 172.19.0.1:43632 - "POST /node/69cbfcbf8df9359ff489840b HTTP/1.0" 202 Accepted 2026-03-31 16:57:56,114 - lava_callback - INFO - Starting async job submit for node 69cbfcbf8df9359ff489840b 2026-03-31 16:57:56,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:56,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcbf8df9359ff489840b HTTP/1.1" 200 None 2026-03-31 16:57:56,152 - lava_callback - DEBUG - Processing node 69cbfcbf8df9359ff489840b: baseline-nfs-arm64 2026-03-31 16:57:56,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:43634 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:57:56,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:56,643 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbf8df9359ff489840b/log.txt.gz 2026-03-31 16:57:56,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:57,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:57,077 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbf8df9359ff489840b/lava-logs.yaml 2026-03-31 16:57:57,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:57,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:57,237 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbf8df9359ff489840b/lava_callback.json.gz 2026-03-31 16:57:57,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:57,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcbf8df9359ff489840b HTTP/1.1" 200 None 2026-03-31 16:57:57,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:57,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:57,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:57,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcbf8df9359ff489840b HTTP/1.1" 200 None 2026-03-31 16:57:57,333 - lava_callback - INFO - Completed processing callback for node 69cbfcbf8df9359ff489840b 2026-03-31 16:57:57,973 - lava_callback - INFO - Received callback for node 69cbfcb08df9359ff48983ca INFO: 172.19.0.1:43646 - "POST /node/69cbfcb08df9359ff48983ca HTTP/1.0" 202 Accepted 2026-03-31 16:57:57,981 - lava_callback - INFO - Starting async job submit for node 69cbfcb08df9359ff48983ca 2026-03-31 16:57:58,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:58,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb08df9359ff48983ca HTTP/1.1" 200 None 2026-03-31 16:57:58,027 - lava_callback - DEBUG - Processing node 69cbfcb08df9359ff48983ca: baseline-arm64 2026-03-31 16:57:58,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:58,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:58,569 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcb08df9359ff48983ca/log.txt.gz 2026-03-31 16:57:58,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:58,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:58,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 16:57:59,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:59,004 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcb08df9359ff48983ca/lava-logs.yaml 2026-03-31 16:57:59,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:57:59,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:57:59,174 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcb08df9359ff48983ca/lava_callback.json.gz 2026-03-31 16:57:59,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:59,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb08df9359ff48983ca HTTP/1.1" 200 None 2026-03-31 16:57:59,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:59,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:57:59,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:57:59,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcb08df9359ff48983ca HTTP/1.1" 200 None 2026-03-31 16:57:59,275 - lava_callback - INFO - Completed processing callback for node 69cbfcb08df9359ff48983ca 2026-03-31 16:58:03,585 - lava_callback - INFO - Received callback for node 69cbfcb88df9359ff48983df INFO: 172.19.0.1:44992 - "POST /node/69cbfcb88df9359ff48983df HTTP/1.0" 202 Accepted 2026-03-31 16:58:03,596 - lava_callback - INFO - Starting async job submit for node 69cbfcb88df9359ff48983df 2026-03-31 16:58:03,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:03,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb88df9359ff48983df HTTP/1.1" 200 None 2026-03-31 16:58:03,647 - lava_callback - DEBUG - Processing node 69cbfcb88df9359ff48983df: baseline-nfs-arm64 2026-03-31 16:58:04,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:04,384 - lava_callback - INFO - Received callback for node 69cbfca88df9359ff48983b6 INFO: 172.19.0.1:44998 - "POST /node/69cbfca88df9359ff48983b6 HTTP/1.0" 202 Accepted 2026-03-31 16:58:04,396 - lava_callback - INFO - Starting async job submit for node 69cbfca88df9359ff48983b6 2026-03-31 16:58:04,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:04,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca88df9359ff48983b6 HTTP/1.1" 200 None 2026-03-31 16:58:04,437 - lava_callback - DEBUG - Processing node 69cbfca88df9359ff48983b6: baseline-arm64 2026-03-31 16:58:04,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:04,611 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb88df9359ff48983df/log.txt.gz 2026-03-31 16:58:05,261 - lava_callback - INFO - Received callback for node 69cbfcbe8df9359ff48983eb INFO: 172.19.0.1:45008 - "POST /node/69cbfcbe8df9359ff48983eb HTTP/1.0" 202 Accepted 2026-03-31 16:58:05,316 - lava_callback - INFO - Starting async job submit for node 69cbfcbe8df9359ff48983eb 2026-03-31 16:58:05,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:05,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:05,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:05,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcbe8df9359ff48983eb HTTP/1.1" 200 None 2026-03-31 16:58:05,440 - lava_callback - DEBUG - Processing node 69cbfcbe8df9359ff48983eb: baseline-nfs-arm64 2026-03-31 16:58:05,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:05,678 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca88df9359ff48983b6/log.txt.gz 2026-03-31 16:58:05,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:05,978 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb88df9359ff48983df/lava-logs.yaml 2026-03-31 16:58:06,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:06,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:06,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:06,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:06,350 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb88df9359ff48983df/lava_callback.json.gz 2026-03-31 16:58:06,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:06,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:06,390 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbe8df9359ff48983eb/log.txt.gz 2026-03-31 16:58:06,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb88df9359ff48983df HTTP/1.1" 200 None 2026-03-31 16:58:06,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:06,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:06,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:06,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:06,625 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca88df9359ff48983b6/lava-logs.yaml INFO: 172.19.0.1:45010 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:58:06,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:06,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:06,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcb88df9359ff48983df HTTP/1.1" 200 None 2026-03-31 16:58:06,697 - lava_callback - INFO - Completed processing callback for node 69cbfcb88df9359ff48983df 2026-03-31 16:58:06,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:06,973 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca88df9359ff48983b6/lava_callback.json.gz 2026-03-31 16:58:07,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:07,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:07,032 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbe8df9359ff48983eb/lava-logs.yaml 2026-03-31 16:58:07,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:07,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca88df9359ff48983b6 HTTP/1.1" 200 None 2026-03-31 16:58:07,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:07,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:07,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:07,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfca88df9359ff48983b6 HTTP/1.1" 200 None 2026-03-31 16:58:07,168 - lava_callback - INFO - Completed processing callback for node 69cbfca88df9359ff48983b6 2026-03-31 16:58:07,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:07,320 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbe8df9359ff48983eb/lava_callback.json.gz 2026-03-31 16:58:07,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:07,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcbe8df9359ff48983eb HTTP/1.1" 200 None 2026-03-31 16:58:07,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:07,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:07,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:07,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcbe8df9359ff48983eb HTTP/1.1" 200 None 2026-03-31 16:58:07,423 - lava_callback - INFO - Completed processing callback for node 69cbfcbe8df9359ff48983eb 2026-03-31 16:58:10,919 - lava_callback - INFO - Received callback for node 69cbfce98df9359ff489849c INFO: 172.19.0.1:49888 - "POST /node/69cbfce98df9359ff489849c HTTP/1.0" 202 Accepted 2026-03-31 16:58:10,926 - lava_callback - INFO - Starting async job submit for node 69cbfce98df9359ff489849c 2026-03-31 16:58:10,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:11,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce98df9359ff489849c HTTP/1.1" 200 None 2026-03-31 16:58:11,018 - lava_callback - DEBUG - Processing node 69cbfce98df9359ff489849c: baseline-arm64-foundriesio 2026-03-31 16:58:11,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:11,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:11,562 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cbfce98df9359ff489849c/log.txt.gz 2026-03-31 16:58:11,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:12,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:12,230 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cbfce98df9359ff489849c/lava-logs.yaml 2026-03-31 16:58:12,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:12,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:12,402 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cbfce98df9359ff489849c/lava_callback.json.gz 2026-03-31 16:58:12,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:12,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce98df9359ff489849c HTTP/1.1" 200 None 2026-03-31 16:58:12,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:12,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:12,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:12,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfce98df9359ff489849c HTTP/1.1" 200 None 2026-03-31 16:58:12,524 - lava_callback - INFO - Completed processing callback for node 69cbfce98df9359ff489849c INFO: 172.19.0.1:49894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49906 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:58:18,483 - lava_callback - INFO - Received callback for node 69cbfce58df9359ff489848b INFO: 172.19.0.1:49920 - "POST /node/69cbfce58df9359ff489848b HTTP/1.0" 202 Accepted 2026-03-31 16:58:18,492 - lava_callback - INFO - Starting async job submit for node 69cbfce58df9359ff489848b 2026-03-31 16:58:18,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:18,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce58df9359ff489848b HTTP/1.1" 200 None 2026-03-31 16:58:18,554 - lava_callback - DEBUG - Processing node 69cbfce58df9359ff489848b: baseline-arm64-pengutronix 2026-03-31 16:58:18,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:18,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:18,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cbfce58df9359ff489848b/log.txt.gz 2026-03-31 16:58:19,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:19,562 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:19,564 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cbfce58df9359ff489848b/lava-logs.yaml 2026-03-31 16:58:19,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:19,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:19,721 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cbfce58df9359ff489848b/lava_callback.json.gz 2026-03-31 16:58:19,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:19,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce58df9359ff489848b HTTP/1.1" 200 None 2026-03-31 16:58:19,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:19,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:19,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:19,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfce58df9359ff489848b HTTP/1.1" 200 None 2026-03-31 16:58:19,807 - lava_callback - INFO - Completed processing callback for node 69cbfce58df9359ff489848b 2026-03-31 16:58:19,858 - lava_callback - INFO - Received callback for node 69cbfcc08df9359ff489840e INFO: 172.19.0.1:37248 - "POST /node/69cbfcc08df9359ff489840e HTTP/1.0" 202 Accepted 2026-03-31 16:58:19,875 - lava_callback - INFO - Starting async job submit for node 69cbfcc08df9359ff489840e 2026-03-31 16:58:19,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:19,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcc08df9359ff489840e HTTP/1.1" 200 None 2026-03-31 16:58:19,987 - lava_callback - DEBUG - Processing node 69cbfcc08df9359ff489840e: baseline-nfs-arm64 2026-03-31 16:58:20,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:21,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:21,109 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcc08df9359ff489840e/log.txt.gz 2026-03-31 16:58:21,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:21,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:21,847 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcc08df9359ff489840e/lava-logs.yaml 2026-03-31 16:58:21,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:22,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:22,038 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcc08df9359ff489840e/lava_callback.json.gz 2026-03-31 16:58:22,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:22,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcc08df9359ff489840e HTTP/1.1" 200 None 2026-03-31 16:58:22,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:22,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:22,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:22,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcc08df9359ff489840e HTTP/1.1" 200 None 2026-03-31 16:58:22,303 - lava_callback - INFO - Completed processing callback for node 69cbfcc08df9359ff489840e 2026-03-31 16:58:26,000 - lava_callback - INFO - Received callback for node 69cbfce68df9359ff489848e INFO: 172.19.0.1:37254 - "POST /node/69cbfce68df9359ff489848e HTTP/1.0" 202 Accepted 2026-03-31 16:58:26,010 - lava_callback - INFO - Starting async job submit for node 69cbfce68df9359ff489848e 2026-03-31 16:58:26,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:26,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce68df9359ff489848e HTTP/1.1" 200 None 2026-03-31 16:58:26,076 - lava_callback - DEBUG - Processing node 69cbfce68df9359ff489848e: baseline-arm64-pengutronix 2026-03-31 16:58:26,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:37266 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:58:26,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:26,598 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cbfce68df9359ff489848e/log.txt.gz 2026-03-31 16:58:26,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:27,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:27,125 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cbfce68df9359ff489848e/lava-logs.yaml 2026-03-31 16:58:27,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:27,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:27,398 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cbfce68df9359ff489848e/lava_callback.json.gz 2026-03-31 16:58:27,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:27,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce68df9359ff489848e HTTP/1.1" 200 None 2026-03-31 16:58:27,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:27,452 - lava_callback - INFO - Received callback for node 69c95a1ef2866272ea51e53e 2026-03-31 16:58:27,458 - lava_callback - INFO - Starting async job submit for node 69c95a1ef2866272ea51e53e INFO: 172.19.0.1:37280 - "POST /node/69c95a1ef2866272ea51e53e HTTP/1.0" 202 Accepted 2026-03-31 16:58:27,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:27,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:27,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:27,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95a1ef2866272ea51e53e HTTP/1.1" 200 None 2026-03-31 16:58:27,478 - lava_callback - DEBUG - Processing node 69c95a1ef2866272ea51e53e: baseline-arm-clabbe 2026-03-31 16:58:27,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95a1ef2866272ea51e53e 2026-03-31 16:58:27,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:27,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfce68df9359ff489848e HTTP/1.1" 200 None 2026-03-31 16:58:27,506 - lava_callback - INFO - Completed processing callback for node 69cbfce68df9359ff489848e 2026-03-31 16:58:27,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95a1ef2866272ea51e53e HTTP/1.1" 200 None 2026-03-31 16:58:27,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:27,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 16:58:27,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:27,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95a1ef2866272ea51e53e HTTP/1.1" 200 None 2026-03-31 16:58:27,643 - lava_callback - ERROR - Error processing callback for node 69c95a1ef2866272ea51e53e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 16:58:28,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:28,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 16:58:35,674 - lava_callback - INFO - Received callback for node 69cbfce78df9359ff4898499 INFO: 172.19.0.1:34442 - "POST /node/69cbfce78df9359ff4898499 HTTP/1.0" 202 Accepted 2026-03-31 16:58:35,685 - lava_callback - INFO - Starting async job submit for node 69cbfce78df9359ff4898499 2026-03-31 16:58:35,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:35,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce78df9359ff4898499 HTTP/1.1" 200 None 2026-03-31 16:58:35,753 - lava_callback - DEBUG - Processing node 69cbfce78df9359ff4898499: baseline-arm64-pengutronix 2026-03-31 16:58:36,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:36,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:36,499 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cbfce78df9359ff4898499/log.txt.gz INFO: 172.19.0.1:34444 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:58:36,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:37,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:37,260 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cbfce78df9359ff4898499/lava-logs.yaml 2026-03-31 16:58:37,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:37,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:37,462 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cbfce78df9359ff4898499/lava_callback.json.gz 2026-03-31 16:58:37,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:37,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce78df9359ff4898499 HTTP/1.1" 200 None 2026-03-31 16:58:37,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:37,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:37,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:37,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfce78df9359ff4898499 HTTP/1.1" 200 None 2026-03-31 16:58:37,603 - lava_callback - INFO - Completed processing callback for node 69cbfce78df9359ff4898499 2026-03-31 16:58:38,161 - lava_callback - INFO - Received callback for node 69cbfbac8df9359ff4898212 INFO: 172.19.0.1:34454 - "POST /node/69cbfbac8df9359ff4898212 HTTP/1.0" 202 Accepted 2026-03-31 16:58:38,171 - lava_callback - INFO - Starting async job submit for node 69cbfbac8df9359ff4898212 2026-03-31 16:58:38,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:38,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbac8df9359ff4898212 HTTP/1.1" 200 None 2026-03-31 16:58:38,267 - lava_callback - DEBUG - Processing node 69cbfbac8df9359ff4898212: baseline-arm-broonie 2026-03-31 16:58:38,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:39,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:39,120 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfbac8df9359ff4898212/log.txt.gz 2026-03-31 16:58:39,203 - lava_callback - INFO - Received callback for node 69cbfcb58df9359ff48983d6 INFO: 172.19.0.1:34470 - "POST /node/69cbfcb58df9359ff48983d6 HTTP/1.0" 202 Accepted 2026-03-31 16:58:39,228 - lava_callback - INFO - Starting async job submit for node 69cbfcb58df9359ff48983d6 2026-03-31 16:58:39,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:39,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb58df9359ff48983d6 HTTP/1.1" 200 None 2026-03-31 16:58:39,354 - lava_callback - DEBUG - Processing node 69cbfcb58df9359ff48983d6: baseline-nfs-arm64 2026-03-31 16:58:39,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:39,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:39,993 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfbac8df9359ff4898212/lava-logs.yaml 2026-03-31 16:58:39,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:40,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:40,469 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:40,471 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb58df9359ff48983d6/log.txt.gz 2026-03-31 16:58:40,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:40,568 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfbac8df9359ff4898212/lava_callback.json.gz 2026-03-31 16:58:40,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:40,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbac8df9359ff4898212 HTTP/1.1" 200 None 2026-03-31 16:58:40,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:40,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:40,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 16:58:40,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:40,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbac8df9359ff4898212 HTTP/1.1" 200 None 2026-03-31 16:58:40,860 - lava_callback - INFO - Completed processing callback for node 69cbfbac8df9359ff4898212 2026-03-31 16:58:41,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:41,456 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb58df9359ff48983d6/lava-logs.yaml 2026-03-31 16:58:41,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:41,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:41,656 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb58df9359ff48983d6/lava_callback.json.gz 2026-03-31 16:58:41,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:41,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb58df9359ff48983d6 HTTP/1.1" 200 None 2026-03-31 16:58:41,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:41,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:41,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:41,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcb58df9359ff48983d6 HTTP/1.1" 200 None 2026-03-31 16:58:41,803 - lava_callback - INFO - Completed processing callback for node 69cbfcb58df9359ff48983d6 2026-03-31 16:58:42,628 - lava_callback - INFO - Received callback for node 69cbfce18df9359ff4898482 INFO: 172.19.0.1:50428 - "POST /node/69cbfce18df9359ff4898482 HTTP/1.0" 202 Accepted 2026-03-31 16:58:42,638 - lava_callback - INFO - Starting async job submit for node 69cbfce18df9359ff4898482 2026-03-31 16:58:42,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:42,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce18df9359ff4898482 HTTP/1.1" 200 None 2026-03-31 16:58:42,711 - lava_callback - DEBUG - Processing node 69cbfce18df9359ff4898482: baseline-arm64-clabbe 2026-03-31 16:58:43,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:43,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:43,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cbfce18df9359ff4898482/log.txt.gz 2026-03-31 16:58:43,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:43,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:43,808 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cbfce18df9359ff4898482/lava-logs.yaml 2026-03-31 16:58:43,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:43,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:43,971 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cbfce18df9359ff4898482/lava_callback.json.gz 2026-03-31 16:58:43,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:44,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce18df9359ff4898482 HTTP/1.1" 200 None 2026-03-31 16:58:44,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:44,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:44,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:44,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfce18df9359ff4898482 HTTP/1.1" 200 None 2026-03-31 16:58:44,100 - lava_callback - INFO - Completed processing callback for node 69cbfce18df9359ff4898482 INFO: 172.19.0.1:50432 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:58:46,751 - lava_callback - INFO - Received callback for node 69cbfcbb8df9359ff48983e5 INFO: 172.19.0.1:50448 - "POST /node/69cbfcbb8df9359ff48983e5 HTTP/1.0" 202 Accepted 2026-03-31 16:58:46,760 - lava_callback - INFO - Starting async job submit for node 69cbfcbb8df9359ff48983e5 2026-03-31 16:58:46,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:46,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcbb8df9359ff48983e5 HTTP/1.1" 200 None 2026-03-31 16:58:46,810 - lava_callback - DEBUG - Processing node 69cbfcbb8df9359ff48983e5: baseline-nfs-arm64 2026-03-31 16:58:47,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:47,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:47,609 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbb8df9359ff48983e5/log.txt.gz 2026-03-31 16:58:47,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:48,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:48,223 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbb8df9359ff48983e5/lava-logs.yaml 2026-03-31 16:58:48,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:48,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:48,360 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcbb8df9359ff48983e5/lava_callback.json.gz 2026-03-31 16:58:48,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:48,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcbb8df9359ff48983e5 HTTP/1.1" 200 None 2026-03-31 16:58:48,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:48,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:48,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:48,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcbb8df9359ff48983e5 HTTP/1.1" 200 None 2026-03-31 16:58:48,465 - lava_callback - INFO - Completed processing callback for node 69cbfcbb8df9359ff48983e5 2026-03-31 16:58:50,198 - lava_callback - INFO - Received callback for node 69cbfcb28df9359ff48983d0 INFO: 172.19.0.1:48612 - "POST /node/69cbfcb28df9359ff48983d0 HTTP/1.0" 202 Accepted 2026-03-31 16:58:50,209 - lava_callback - INFO - Starting async job submit for node 69cbfcb28df9359ff48983d0 2026-03-31 16:58:50,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:50,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb28df9359ff48983d0 HTTP/1.1" 200 None 2026-03-31 16:58:50,288 - lava_callback - DEBUG - Processing node 69cbfcb28df9359ff48983d0: baseline-nfs-arm64 2026-03-31 16:58:50,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:51,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:51,159 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb28df9359ff48983d0/log.txt.gz 2026-03-31 16:58:51,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:51,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:51,962 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb28df9359ff48983d0/lava-logs.yaml 2026-03-31 16:58:51,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:52,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:52,127 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb28df9359ff48983d0/lava_callback.json.gz 2026-03-31 16:58:52,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:52,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb28df9359ff48983d0 HTTP/1.1" 200 None 2026-03-31 16:58:52,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:52,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:52,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:52,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcb28df9359ff48983d0 HTTP/1.1" 200 None 2026-03-31 16:58:52,245 - lava_callback - INFO - Completed processing callback for node 69cbfcb28df9359ff48983d0 INFO: 172.19.0.1:48622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48634 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:58:58,072 - lava_callback - INFO - Received callback for node 69cbfcdb8df9359ff4898457 INFO: 172.19.0.1:48638 - "POST /node/69cbfcdb8df9359ff4898457 HTTP/1.0" 202 Accepted 2026-03-31 16:58:58,082 - lava_callback - INFO - Starting async job submit for node 69cbfcdb8df9359ff4898457 2026-03-31 16:58:58,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:58,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcdb8df9359ff4898457 HTTP/1.1" 200 None 2026-03-31 16:58:58,155 - lava_callback - DEBUG - Processing node 69cbfcdb8df9359ff4898457: baseline-arm64-broonie 2026-03-31 16:58:58,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:58,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:58,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-31 16:58:58,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:58,940 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcdb8df9359ff4898457/log.txt.gz 2026-03-31 16:58:59,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:59,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:59,712 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcdb8df9359ff4898457/lava-logs.yaml 2026-03-31 16:58:59,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:58:59,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:58:59,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcdb8df9359ff4898457/lava_callback.json.gz 2026-03-31 16:58:59,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:59,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcdb8df9359ff4898457 HTTP/1.1" 200 None 2026-03-31 16:58:59,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:59,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 16:58:59,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:58:59,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcdb8df9359ff4898457 HTTP/1.1" 200 None 2026-03-31 16:58:59,980 - lava_callback - INFO - Completed processing callback for node 69cbfcdb8df9359ff4898457 INFO: 172.19.0.1:46076 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:59:07,511 - lava_callback - INFO - Received callback for node 69c95a47f2866272ea51e54e INFO: 172.19.0.1:46080 - "POST /node/69c95a47f2866272ea51e54e HTTP/1.0" 202 Accepted 2026-03-31 16:59:07,517 - lava_callback - INFO - Starting async job submit for node 69c95a47f2866272ea51e54e 2026-03-31 16:59:07,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:07,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95a47f2866272ea51e54e HTTP/1.1" 200 None 2026-03-31 16:59:07,534 - lava_callback - DEBUG - Processing node 69c95a47f2866272ea51e54e: baseline-arm-clabbe 2026-03-31 16:59:07,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95a47f2866272ea51e54e 2026-03-31 16:59:07,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:07,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95a47f2866272ea51e54e HTTP/1.1" 200 None 2026-03-31 16:59:07,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:07,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 16:59:07,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:07,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95a47f2866272ea51e54e HTTP/1.1" 200 None 2026-03-31 16:59:07,605 - lava_callback - ERROR - Error processing callback for node 69c95a47f2866272ea51e54e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42956 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:59:28,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:28,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:50476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36444 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:59:47,612 - lava_callback - INFO - Received callback for node 69c95a6ff2866272ea51e568 INFO: 172.19.0.1:36458 - "POST /node/69c95a6ff2866272ea51e568 HTTP/1.0" 202 Accepted 2026-03-31 16:59:47,621 - lava_callback - INFO - Starting async job submit for node 69c95a6ff2866272ea51e568 2026-03-31 16:59:47,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:47,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95a6ff2866272ea51e568 HTTP/1.1" 200 None 2026-03-31 16:59:47,649 - lava_callback - DEBUG - Processing node 69c95a6ff2866272ea51e568: baseline-arm-clabbe 2026-03-31 16:59:47,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95a6ff2866272ea51e568 2026-03-31 16:59:47,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:47,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95a6ff2866272ea51e568 HTTP/1.1" 200 None 2026-03-31 16:59:47,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:47,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 16:59:47,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:47,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95a6ff2866272ea51e568 HTTP/1.1" 200 None 2026-03-31 16:59:47,735 - lava_callback - ERROR - Error processing callback for node 69c95a6ff2866272ea51e568: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59258 - "GET / HTTP/1.0" 200 OK 2026-03-31 16:59:57,512 - lava_callback - INFO - Received callback for node 69cbfca58df9359ff48983b0 INFO: 172.19.0.1:59270 - "POST /node/69cbfca58df9359ff48983b0 HTTP/1.0" 202 Accepted 2026-03-31 16:59:57,522 - lava_callback - INFO - Starting async job submit for node 69cbfca58df9359ff48983b0 2026-03-31 16:59:57,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:57,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca58df9359ff48983b0 HTTP/1.1" 200 None 2026-03-31 16:59:57,601 - lava_callback - DEBUG - Processing node 69cbfca58df9359ff48983b0: baseline-arm64 2026-03-31 16:59:58,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:59:58,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:59:58,517 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca58df9359ff48983b0/log.txt.gz 2026-03-31 16:59:58,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:59:59,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:59:59,698 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca58df9359ff48983b0/lava-logs.yaml 2026-03-31 16:59:59,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 16:59:59,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 16:59:59,887 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca58df9359ff48983b0/lava_callback.json.gz 2026-03-31 16:59:59,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 16:59:59,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca58df9359ff48983b0 HTTP/1.1" 200 None 2026-03-31 16:59:59,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:00,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:00:00,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:00,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfca58df9359ff48983b0 HTTP/1.1" 200 None 2026-03-31 17:00:00,070 - lava_callback - INFO - Completed processing callback for node 69cbfca58df9359ff48983b0 INFO: 172.19.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47502 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:00:20,190 - lava_callback - INFO - Received callback for node 69cbfcd48df9359ff4898441 INFO: 172.19.0.1:33930 - "POST /node/69cbfcd48df9359ff4898441 HTTP/1.0" 202 Accepted 2026-03-31 17:00:20,200 - lava_callback - INFO - Starting async job submit for node 69cbfcd48df9359ff4898441 2026-03-31 17:00:20,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:20,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcd48df9359ff4898441 HTTP/1.1" 200 None 2026-03-31 17:00:20,256 - lava_callback - DEBUG - Processing node 69cbfcd48df9359ff4898441: baseline-arm64-broonie 2026-03-31 17:00:20,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:00:20,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:00:20,952 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcd48df9359ff4898441/log.txt.gz 2026-03-31 17:00:21,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:00:21,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:00:21,642 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcd48df9359ff4898441/lava-logs.yaml 2026-03-31 17:00:21,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:00:21,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:00:21,822 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcd48df9359ff4898441/lava_callback.json.gz 2026-03-31 17:00:21,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:21,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcd48df9359ff4898441 HTTP/1.1" 200 None 2026-03-31 17:00:21,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:21,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:00:21,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:21,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcd48df9359ff4898441 HTTP/1.1" 200 None 2026-03-31 17:00:21,927 - lava_callback - INFO - Completed processing callback for node 69cbfcd48df9359ff4898441 INFO: 172.19.0.1:33934 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:00:28,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:28,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:39578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55510 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:00:47,638 - lava_callback - INFO - Received callback for node 69c95abbf2866272ea51e58b INFO: 172.19.0.1:55520 - "POST /node/69c95abbf2866272ea51e58b HTTP/1.0" 202 Accepted 2026-03-31 17:00:47,648 - lava_callback - INFO - Starting async job submit for node 69c95abbf2866272ea51e58b 2026-03-31 17:00:47,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:47,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95abbf2866272ea51e58b HTTP/1.1" 200 None 2026-03-31 17:00:47,677 - lava_callback - DEBUG - Processing node 69c95abbf2866272ea51e58b: baseline-arm-clabbe 2026-03-31 17:00:47,678 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95abbf2866272ea51e58b 2026-03-31 17:00:47,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:47,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95abbf2866272ea51e58b HTTP/1.1" 200 None 2026-03-31 17:00:47,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:47,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 17:00:47,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:00:47,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95abbf2866272ea51e58b HTTP/1.1" 200 None 2026-03-31 17:00:47,751 - lava_callback - ERROR - Error processing callback for node 69c95abbf2866272ea51e58b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47252 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:01:27,614 - lava_callback - INFO - Received callback for node 69c95ae1f2866272ea51e599 INFO: 172.19.0.1:47254 - "POST /node/69c95ae1f2866272ea51e599 HTTP/1.0" 202 Accepted 2026-03-31 17:01:27,622 - lava_callback - INFO - Starting async job submit for node 69c95ae1f2866272ea51e599 2026-03-31 17:01:27,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:01:27,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95ae1f2866272ea51e599 HTTP/1.1" 200 None 2026-03-31 17:01:27,641 - lava_callback - DEBUG - Processing node 69c95ae1f2866272ea51e599: baseline-arm-clabbe 2026-03-31 17:01:27,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95ae1f2866272ea51e599 2026-03-31 17:01:27,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:01:27,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95ae1f2866272ea51e599 HTTP/1.1" 200 None 2026-03-31 17:01:27,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:01:27,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 17:01:27,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:01:27,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95ae1f2866272ea51e599 HTTP/1.1" 200 None 2026-03-31 17:01:27,704 - lava_callback - ERROR - Error processing callback for node 69c95ae1f2866272ea51e599: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50966 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:01:58,285 - lava_callback - INFO - Received callback for node 69cbfbf08df9359ff48982e0 INFO: 172.19.0.1:50980 - "POST /node/69cbfbf08df9359ff48982e0 HTTP/1.0" 202 Accepted 2026-03-31 17:01:58,297 - lava_callback - INFO - Starting async job submit for node 69cbfbf08df9359ff48982e0 2026-03-31 17:01:58,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:01:58,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbf08df9359ff48982e0 HTTP/1.1" 200 None 2026-03-31 17:01:58,358 - lava_callback - DEBUG - Processing node 69cbfbf08df9359ff48982e0: ltp-syscalls 2026-03-31 17:01:59,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:01:59,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:01:59,731 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cbfbf08df9359ff48982e0/log.txt.gz 2026-03-31 17:02:00,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:02:00,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:02:00,543 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cbfbf08df9359ff48982e0/lava-logs.yaml 2026-03-31 17:02:00,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:02:00,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:02:00,762 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cbfbf08df9359ff48982e0/lava_callback.json.gz 2026-03-31 17:02:00,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:00,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbf08df9359ff48982e0 HTTP/1.1" 200 None 2026-03-31 17:02:00,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:00,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 17:02:00,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:00,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbf08df9359ff48982e0 HTTP/1.1" 200 None 2026-03-31 17:02:00,855 - lava_callback - INFO - Completed processing callback for node 69cbfbf08df9359ff48982e0 INFO: 172.19.0.1:33344 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:02:13,529 - lava_callback - INFO - Received callback for node 69c95b07f2866272ea51e5a7 INFO: 172.19.0.1:37834 - "POST /node/69c95b07f2866272ea51e5a7 HTTP/1.0" 202 Accepted 2026-03-31 17:02:13,536 - lava_callback - INFO - Starting async job submit for node 69c95b07f2866272ea51e5a7 2026-03-31 17:02:13,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:13,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95b07f2866272ea51e5a7 HTTP/1.1" 200 None 2026-03-31 17:02:13,554 - lava_callback - DEBUG - Processing node 69c95b07f2866272ea51e5a7: baseline-arm-baylibre 2026-03-31 17:02:13,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95b07f2866272ea51e5a7 2026-03-31 17:02:13,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:13,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95b07f2866272ea51e5a7 HTTP/1.1" 200 None 2026-03-31 17:02:13,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:13,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 17:02:13,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:13,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95b07f2866272ea51e5a7 HTTP/1.1" 200 None 2026-03-31 17:02:13,618 - lava_callback - ERROR - Error processing callback for node 69c95b07f2866272ea51e5a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60640 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:02:28,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:28,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:33812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52200 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:02:49,862 - lava_callback - INFO - Received callback for node 69cbfe1b8df9359ff48985be INFO: 172.19.0.1:38928 - "POST /node/69cbfe1b8df9359ff48985be HTTP/1.0" 202 Accepted 2026-03-31 17:02:49,868 - lava_callback - INFO - Starting async job submit for node 69cbfe1b8df9359ff48985be 2026-03-31 17:02:49,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:49,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfe1b8df9359ff48985be HTTP/1.1" 200 None 2026-03-31 17:02:49,904 - lava_callback - DEBUG - Processing node 69cbfe1b8df9359ff48985be: baseline-x86 2026-03-31 17:02:50,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:02:50,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:02:50,228 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe1b8df9359ff48985be/log.txt.gz 2026-03-31 17:02:50,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:02:50,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:02:50,541 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe1b8df9359ff48985be/lava-logs.yaml 2026-03-31 17:02:50,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:02:50,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:02:50,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe1b8df9359ff48985be/lava_callback.json.gz 2026-03-31 17:02:50,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:50,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfe1b8df9359ff48985be HTTP/1.1" 200 None 2026-03-31 17:02:50,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:50,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4418df9359ff4897d50 HTTP/1.1" 200 None 2026-03-31 17:02:50,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:50,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfe1b8df9359ff48985be HTTP/1.1" 200 None 2026-03-31 17:02:50,755 - lava_callback - INFO - Completed processing callback for node 69cbfe1b8df9359ff48985be INFO: 172.19.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38958 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:02:58,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:58,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 17:02:59,473 - lava_callback - INFO - Received callback for node 69cbfe198df9359ff48985b5 INFO: 172.19.0.1:38968 - "POST /node/69cbfe198df9359ff48985b5 HTTP/1.0" 202 Accepted 2026-03-31 17:02:59,483 - lava_callback - INFO - Starting async job submit for node 69cbfe198df9359ff48985b5 2026-03-31 17:02:59,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:02:59,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfe198df9359ff48985b5 HTTP/1.1" 200 None 2026-03-31 17:02:59,530 - lava_callback - DEBUG - Processing node 69cbfe198df9359ff48985b5: baseline-x86 2026-03-31 17:02:59,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:02:59,852 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:02:59,854 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe198df9359ff48985b5/log.txt.gz 2026-03-31 17:02:59,949 - lava_callback - INFO - Received callback for node 69cbfce98df9359ff489849f INFO: 172.19.0.1:44526 - "POST /node/69cbfce98df9359ff489849f HTTP/1.0" 202 Accepted 2026-03-31 17:02:59,977 - lava_callback - INFO - Starting async job submit for node 69cbfce98df9359ff489849f 2026-03-31 17:03:00,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:00,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:00,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce98df9359ff489849f HTTP/1.1" 200 None 2026-03-31 17:03:00,058 - lava_callback - DEBUG - Processing node 69cbfce98df9359ff489849f: baseline-arm64-foundriesio 2026-03-31 17:03:00,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:00,307 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe198df9359ff48985b5/lava-logs.yaml 2026-03-31 17:03:00,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:00,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:00,683 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe198df9359ff48985b5/lava_callback.json.gz 2026-03-31 17:03:00,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:00,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfe198df9359ff48985b5 HTTP/1.1" 200 None 2026-03-31 17:03:00,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:00,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4418df9359ff4897d50 HTTP/1.1" 200 None 2026-03-31 17:03:00,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:01,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfe198df9359ff48985b5 HTTP/1.1" 200 None 2026-03-31 17:03:01,108 - lava_callback - INFO - Completed processing callback for node 69cbfe198df9359ff48985b5 2026-03-31 17:03:01,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:01,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:01,634 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cbfce98df9359ff489849f/log.txt.gz 2026-03-31 17:03:02,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:02,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:02,575 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cbfce98df9359ff489849f/lava-logs.yaml 2026-03-31 17:03:02,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:02,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:02,781 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cbfce98df9359ff489849f/lava_callback.json.gz DEBUG: Setup failed for 69cbfce98df9359ff489849f. Transitting job node result: incomplete -> fail 2026-03-31 17:03:02,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:02,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfce98df9359ff489849f HTTP/1.1" 200 None 2026-03-31 17:03:02,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:02,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:03:02,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:02,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfce98df9359ff489849f HTTP/1.1" 200 None 2026-03-31 17:03:03,055 - lava_callback - INFO - Completed processing callback for node 69cbfce98df9359ff489849f INFO: 172.19.0.1:44536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40852 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:03:27,699 - lava_callback - INFO - Received callback for node 69c95b53f2866272ea51e5ca INFO: 172.19.0.1:40856 - "POST /node/69c95b53f2866272ea51e5ca HTTP/1.0" 202 Accepted 2026-03-31 17:03:27,704 - lava_callback - INFO - Starting async job submit for node 69c95b53f2866272ea51e5ca 2026-03-31 17:03:27,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:27,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95b53f2866272ea51e5ca HTTP/1.1" 200 None 2026-03-31 17:03:27,720 - lava_callback - DEBUG - Processing node 69c95b53f2866272ea51e5ca: baseline-arm-clabbe 2026-03-31 17:03:27,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95b53f2866272ea51e5ca 2026-03-31 17:03:27,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:27,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95b53f2866272ea51e5ca HTTP/1.1" 200 None 2026-03-31 17:03:27,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:27,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 17:03:27,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:27,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95b53f2866272ea51e5ca HTTP/1.1" 200 None 2026-03-31 17:03:27,777 - lava_callback - ERROR - Error processing callback for node 69c95b53f2866272ea51e5ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 17:03:28,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:29,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:60928 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:03:39,528 - lava_callback - INFO - Received callback for node 69cbfe1a8df9359ff48985bb INFO: 172.19.0.1:60944 - "POST /node/69cbfe1a8df9359ff48985bb HTTP/1.0" 202 Accepted 2026-03-31 17:03:39,534 - lava_callback - INFO - Starting async job submit for node 69cbfe1a8df9359ff48985bb 2026-03-31 17:03:39,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:39,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfe1a8df9359ff48985bb HTTP/1.1" 200 None 2026-03-31 17:03:39,588 - lava_callback - DEBUG - Processing node 69cbfe1a8df9359ff48985bb: baseline-x86 2026-03-31 17:03:39,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:40,009 - lava_callback - INFO - Received callback for node 69cbfcb18df9359ff48983cd INFO: 172.19.0.1:50600 - "POST /node/69cbfcb18df9359ff48983cd HTTP/1.0" 202 Accepted 2026-03-31 17:03:40,016 - lava_callback - INFO - Starting async job submit for node 69cbfcb18df9359ff48983cd 2026-03-31 17:03:40,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:40,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb18df9359ff48983cd HTTP/1.1" 200 None 2026-03-31 17:03:40,077 - lava_callback - DEBUG - Processing node 69cbfcb18df9359ff48983cd: baseline-arm64 2026-03-31 17:03:40,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:40,239 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe1a8df9359ff48985bb/log.txt.gz 2026-03-31 17:03:40,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:40,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:40,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:40,795 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcb18df9359ff48983cd/log.txt.gz 2026-03-31 17:03:40,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:40,931 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe1a8df9359ff48985bb/lava-logs.yaml 2026-03-31 17:03:40,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:41,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:41,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:41,178 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe1a8df9359ff48985bb/lava_callback.json.gz 2026-03-31 17:03:41,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:41,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfe1a8df9359ff48985bb HTTP/1.1" 200 None 2026-03-31 17:03:41,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:41,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4418df9359ff4897d50 HTTP/1.1" 200 None 2026-03-31 17:03:41,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:41,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfe1a8df9359ff48985bb HTTP/1.1" 200 None 2026-03-31 17:03:41,264 - lava_callback - INFO - Completed processing callback for node 69cbfe1a8df9359ff48985bb 2026-03-31 17:03:41,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:41,288 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcb18df9359ff48983cd/lava-logs.yaml 2026-03-31 17:03:41,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:03:41,499 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:03:41,501 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfcb18df9359ff48983cd/lava_callback.json.gz DEBUG: Setup failed for 69cbfcb18df9359ff48983cd. Transitting job node result: incomplete -> fail 2026-03-31 17:03:41,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:41,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb18df9359ff48983cd HTTP/1.1" 200 None 2026-03-31 17:03:41,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:41,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:03:41,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:41,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcb18df9359ff48983cd HTTP/1.1" 200 None 2026-03-31 17:03:41,727 - lava_callback - INFO - Completed processing callback for node 69cbfcb18df9359ff48983cd INFO: 172.19.0.1:50616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51294 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:03:59,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:03:59,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:56194 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:04:13,362 - lava_callback - INFO - Received callback for node 69cbfe198df9359ff48985b8 INFO: 172.19.0.1:57432 - "POST /node/69cbfe198df9359ff48985b8 HTTP/1.0" 202 Accepted 2026-03-31 17:04:13,368 - lava_callback - INFO - Starting async job submit for node 69cbfe198df9359ff48985b8 2026-03-31 17:04:13,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:04:13,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfe198df9359ff48985b8 HTTP/1.1" 200 None 2026-03-31 17:04:13,409 - lava_callback - DEBUG - Processing node 69cbfe198df9359ff48985b8: baseline-x86 2026-03-31 17:04:13,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:04:14,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:04:14,024 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe198df9359ff48985b8/log.txt.gz 2026-03-31 17:04:14,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:04:14,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:04:14,443 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe198df9359ff48985b8/lava-logs.yaml 2026-03-31 17:04:14,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:04:14,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:04:14,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe198df9359ff48985b8/lava_callback.json.gz 2026-03-31 17:04:14,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:04:14,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfe198df9359ff48985b8 HTTP/1.1" 200 None 2026-03-31 17:04:14,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:04:14,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4418df9359ff4897d50 HTTP/1.1" 200 None 2026-03-31 17:04:14,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:04:14,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfe198df9359ff48985b8 HTTP/1.1" 200 None 2026-03-31 17:04:14,737 - lava_callback - INFO - Completed processing callback for node 69cbfe198df9359ff48985b8 INFO: 172.19.0.1:57438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39846 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:04:29,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:04:29,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37672 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:06:27,823 - lava_callback - INFO - Received callback for node 69c95c0df2866272ea51e606 INFO: 172.19.0.1:37684 - "POST /node/69c95c0df2866272ea51e606 HTTP/1.0" 202 Accepted 2026-03-31 17:06:27,829 - lava_callback - INFO - Starting async job submit for node 69c95c0df2866272ea51e606 2026-03-31 17:06:27,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:06:27,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95c0df2866272ea51e606 HTTP/1.1" 200 None 2026-03-31 17:06:27,846 - lava_callback - DEBUG - Processing node 69c95c0df2866272ea51e606: baseline-arm-clabbe 2026-03-31 17:06:27,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95c0df2866272ea51e606 2026-03-31 17:06:27,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:06:27,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95c0df2866272ea51e606 HTTP/1.1" 200 None 2026-03-31 17:06:27,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:06:27,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 17:06:27,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:06:27,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95c0df2866272ea51e606 HTTP/1.1" 200 None 2026-03-31 17:06:27,905 - lava_callback - ERROR - Error processing callback for node 69c95c0df2866272ea51e606: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44554 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:07:47,840 - lava_callback - INFO - Received callback for node 69c95c54f2866272ea51e620 INFO: 172.19.0.1:44560 - "POST /node/69c95c54f2866272ea51e620 HTTP/1.0" 202 Accepted 2026-03-31 17:07:47,846 - lava_callback - INFO - Starting async job submit for node 69c95c54f2866272ea51e620 2026-03-31 17:07:47,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:07:47,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95c54f2866272ea51e620 HTTP/1.1" 200 None 2026-03-31 17:07:47,868 - lava_callback - DEBUG - Processing node 69c95c54f2866272ea51e620: baseline-arm-clabbe 2026-03-31 17:07:47,868 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95c54f2866272ea51e620 2026-03-31 17:07:47,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:07:47,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95c54f2866272ea51e620 HTTP/1.1" 200 None 2026-03-31 17:07:47,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:07:47,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 17:07:47,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:07:47,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95c54f2866272ea51e620 HTTP/1.1" 200 None 2026-03-31 17:07:47,933 - lava_callback - ERROR - Error processing callback for node 69c95c54f2866272ea51e620: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53204 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:07:58,525 - lava_callback - INFO - Received callback for node 69cbfcd38df9359ff489843e INFO: 172.19.0.1:53216 - "POST /node/69cbfcd38df9359ff489843e HTTP/1.0" 202 Accepted 2026-03-31 17:07:58,532 - lava_callback - INFO - Starting async job submit for node 69cbfcd38df9359ff489843e 2026-03-31 17:07:58,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:07:58,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcd38df9359ff489843e HTTP/1.1" 200 None 2026-03-31 17:07:58,571 - lava_callback - DEBUG - Processing node 69cbfcd38df9359ff489843e: baseline-arm64-broonie 2026-03-31 17:07:58,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:07:58,953 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:07:58,954 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcd38df9359ff489843e/log.txt.gz 2026-03-31 17:07:59,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:07:59,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:07:59,313 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcd38df9359ff489843e/lava-logs.yaml 2026-03-31 17:07:59,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:07:59,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:07:59,472 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cbfcd38df9359ff489843e/lava_callback.json.gz 2026-03-31 17:07:59,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:07:59,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcd38df9359ff489843e HTTP/1.1" 200 None 2026-03-31 17:07:59,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:07:59,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:07:59,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:07:59,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcd38df9359ff489843e HTTP/1.1" 200 None 2026-03-31 17:07:59,556 - lava_callback - INFO - Completed processing callback for node 69cbfcd38df9359ff489843e INFO: 172.19.0.1:57282 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:08:14,262 - lava_callback - INFO - Received callback for node 69cbfb998df9359ff48981e7 INFO: 172.19.0.1:33674 - "POST /node/69cbfb998df9359ff48981e7 HTTP/1.0" 202 Accepted 2026-03-31 17:08:14,271 - lava_callback - INFO - Starting async job submit for node 69cbfb998df9359ff48981e7 2026-03-31 17:08:14,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:08:14,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb998df9359ff48981e7 HTTP/1.1" 200 None 2026-03-31 17:08:14,365 - lava_callback - DEBUG - Processing node 69cbfb998df9359ff48981e7: baseline-arm 2026-03-31 17:08:14,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:08:15,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:08:15,130 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb998df9359ff48981e7/log.txt.gz INFO: 172.19.0.1:33676 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 17:08:15,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:08:15,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:08:15,697 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb998df9359ff48981e7/lava-logs.yaml 2026-03-31 17:08:15,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:08:15,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:08:15,882 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cbfb998df9359ff48981e7/lava_callback.json.gz DEBUG: Setup failed for 69cbfb998df9359ff48981e7. Transitting job node result: incomplete -> fail 2026-03-31 17:08:16,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:08:16,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb998df9359ff48981e7 HTTP/1.1" 200 None 2026-03-31 17:08:16,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:08:16,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 17:08:16,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:08:16,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfb998df9359ff48981e7 HTTP/1.1" 200 None 2026-03-31 17:08:16,224 - lava_callback - INFO - Completed processing callback for node 69cbfb998df9359ff48981e7 INFO: 172.19.0.1:33688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33170 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:08:29,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:08:29,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:43254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43520 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:08:46,791 - lava_callback - INFO - Received callback for node 69cbff918df9359ff489865e INFO: 172.19.0.1:43524 - "POST /node/69cbff918df9359ff489865e HTTP/1.0" 202 Accepted 2026-03-31 17:08:46,798 - lava_callback - INFO - Starting async job submit for node 69cbff918df9359ff489865e 2026-03-31 17:08:46,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:08:46,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbff918df9359ff489865e HTTP/1.1" 200 None 2026-03-31 17:08:46,816 - lava_callback - DEBUG - Processing node 69cbff918df9359ff489865e: baseline-arm 2026-03-31 17:08:46,821 - lava_callback - WARNING - Failed to upload log 2026-03-31 17:08:46,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:08:46,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:08:46,942 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cbff918df9359ff489865e/lava-logs.yaml 2026-03-31 17:08:46,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:08:47,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:08:47,060 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cbff918df9359ff489865e/lava_callback.json.gz 2026-03-31 17:08:47,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:08:47,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbff918df9359ff489865e HTTP/1.1" 200 None 2026-03-31 17:08:47,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:08:47,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 17:08:47,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:08:47,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbff918df9359ff489865e HTTP/1.1" 200 None 2026-03-31 17:08:47,136 - lava_callback - INFO - Completed processing callback for node 69cbff918df9359ff489865e INFO: 172.19.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40416 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:08:59,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:08:59,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40050 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:10:54,047 - lava_callback - INFO - Received callback for node 69c95d1af2866272ea51e642 INFO: 172.19.0.1:55302 - "POST /node/69c95d1af2866272ea51e642 HTTP/1.0" 202 Accepted 2026-03-31 17:10:54,056 - lava_callback - INFO - Starting async job submit for node 69c95d1af2866272ea51e642 2026-03-31 17:10:54,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:10:54,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95d1af2866272ea51e642 HTTP/1.1" 200 None 2026-03-31 17:10:54,088 - lava_callback - DEBUG - Processing node 69c95d1af2866272ea51e642: baseline-arm64-baylibre 2026-03-31 17:10:54,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95d1af2866272ea51e642 2026-03-31 17:10:54,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:10:54,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95d1af2866272ea51e642 HTTP/1.1" 200 None 2026-03-31 17:10:54,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:10:54,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:10:54,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:10:54,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95d1af2866272ea51e642 HTTP/1.1" 200 None 2026-03-31 17:10:54,201 - lava_callback - ERROR - Error processing callback for node 69c95d1af2866272ea51e642: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35314 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:11:59,976 - lava_callback - INFO - Received callback for node 69cbfbc38df9359ff4898272 INFO: 172.19.0.1:55916 - "POST /node/69cbfbc38df9359ff4898272 HTTP/1.0" 202 Accepted 2026-03-31 17:12:00,028 - lava_callback - INFO - Starting async job submit for node 69cbfbc38df9359ff4898272 2026-03-31 17:12:00,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:12:00,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbc38df9359ff4898272 HTTP/1.1" 200 None 2026-03-31 17:12:00,100 - lava_callback - DEBUG - Processing node 69cbfbc38df9359ff4898272: baseline-arm-pengutronix 2026-03-31 17:12:00,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:12:00,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:12:00,469 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc38df9359ff4898272/log.txt.gz 2026-03-31 17:12:00,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:12:01,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:12:01,067 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc38df9359ff4898272/lava-logs.yaml 2026-03-31 17:12:01,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:12:01,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:12:01,257 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cbfbc38df9359ff4898272/lava_callback.json.gz DEBUG: Setup failed for 69cbfbc38df9359ff4898272. Transitting job node result: incomplete -> fail 2026-03-31 17:12:01,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:12:01,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbc38df9359ff4898272 HTTP/1.1" 200 None 2026-03-31 17:12:01,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:12:01,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 17:12:01,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:12:01,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbc38df9359ff4898272 HTTP/1.1" 200 None 2026-03-31 17:12:01,474 - lava_callback - INFO - Completed processing callback for node 69cbfbc38df9359ff4898272 INFO: 172.19.0.1:55932 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:12:14,085 - lava_callback - INFO - Received callback for node 69c95d66f2866272ea51e656 INFO: 172.19.0.1:42400 - "POST /node/69c95d66f2866272ea51e656 HTTP/1.0" 202 Accepted 2026-03-31 17:12:14,093 - lava_callback - INFO - Starting async job submit for node 69c95d66f2866272ea51e656 2026-03-31 17:12:14,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:12:14,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95d66f2866272ea51e656 HTTP/1.1" 200 None 2026-03-31 17:12:14,117 - lava_callback - DEBUG - Processing node 69c95d66f2866272ea51e656: baseline-arm64-baylibre 2026-03-31 17:12:14,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95d66f2866272ea51e656 2026-03-31 17:12:14,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:12:14,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95d66f2866272ea51e656 HTTP/1.1" 200 None 2026-03-31 17:12:14,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:12:14,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:12:14,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:12:14,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95d66f2866272ea51e656 HTTP/1.1" 200 None 2026-03-31 17:12:14,199 - lava_callback - ERROR - Error processing callback for node 69c95d66f2866272ea51e656: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52976 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:12:29,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:12:29,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:33264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43180 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:13:34,163 - lava_callback - INFO - Received callback for node 69c95db4f2866272ea51e665 INFO: 172.19.0.1:45836 - "POST /node/69c95db4f2866272ea51e665 HTTP/1.0" 202 Accepted 2026-03-31 17:13:34,169 - lava_callback - INFO - Starting async job submit for node 69c95db4f2866272ea51e665 2026-03-31 17:13:34,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:13:34,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95db4f2866272ea51e665 HTTP/1.1" 200 None 2026-03-31 17:13:34,191 - lava_callback - DEBUG - Processing node 69c95db4f2866272ea51e665: baseline-arm64-baylibre 2026-03-31 17:13:34,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95db4f2866272ea51e665 2026-03-31 17:13:34,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:13:34,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95db4f2866272ea51e665 HTTP/1.1" 200 None 2026-03-31 17:13:34,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:13:34,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:13:34,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:13:34,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95db4f2866272ea51e665 HTTP/1.1" 200 None 2026-03-31 17:13:34,249 - lava_callback - ERROR - Error processing callback for node 69c95db4f2866272ea51e665: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36960 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:14:54,231 - lava_callback - INFO - Received callback for node 69c95dfdf2866272ea51e674 INFO: 172.19.0.1:51148 - "POST /node/69c95dfdf2866272ea51e674 HTTP/1.0" 202 Accepted 2026-03-31 17:14:54,237 - lava_callback - INFO - Starting async job submit for node 69c95dfdf2866272ea51e674 2026-03-31 17:14:54,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:14:54,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95dfdf2866272ea51e674 HTTP/1.1" 200 None 2026-03-31 17:14:54,267 - lava_callback - DEBUG - Processing node 69c95dfdf2866272ea51e674: baseline-arm64-baylibre 2026-03-31 17:14:54,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95dfdf2866272ea51e674 2026-03-31 17:14:54,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:14:54,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95dfdf2866272ea51e674 HTTP/1.1" 200 None 2026-03-31 17:14:54,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:14:54,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:14:54,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:14:54,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95dfdf2866272ea51e674 HTTP/1.1" 200 None 2026-03-31 17:14:54,367 - lava_callback - ERROR - Error processing callback for node 69c95dfdf2866272ea51e674: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50948 - "GET /node/693f135c24745582e399a6d6%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-31 17:16:14,711 - lava_callback - INFO - Received callback for node 69c95e4af2866272ea51e67c INFO: 172.19.0.1:50960 - "POST /node/69c95e4af2866272ea51e67c HTTP/1.0" 202 Accepted 2026-03-31 17:16:14,717 - lava_callback - INFO - Starting async job submit for node 69c95e4af2866272ea51e67c 2026-03-31 17:16:14,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:16:14,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95e4af2866272ea51e67c HTTP/1.1" 200 None 2026-03-31 17:16:14,734 - lava_callback - DEBUG - Processing node 69c95e4af2866272ea51e67c: baseline-arm64-baylibre 2026-03-31 17:16:14,734 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95e4af2866272ea51e67c 2026-03-31 17:16:14,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:16:14,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95e4af2866272ea51e67c HTTP/1.1" 200 None 2026-03-31 17:16:14,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:16:14,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:16:14,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:16:14,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95e4af2866272ea51e67c HTTP/1.1" 200 None 2026-03-31 17:16:14,787 - lava_callback - ERROR - Error processing callback for node 69c95e4af2866272ea51e67c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52366 - "GET /node/698b0f383b3beb19e7618436%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:52374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33284 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:17:14,300 - lava_callback - INFO - Received callback for node 69c95e98f2866272ea51e68d INFO: 172.19.0.1:58712 - "POST /node/69c95e98f2866272ea51e68d HTTP/1.0" 202 Accepted 2026-03-31 17:17:14,305 - lava_callback - INFO - Starting async job submit for node 69c95e98f2866272ea51e68d 2026-03-31 17:17:14,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:17:14,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95e98f2866272ea51e68d HTTP/1.1" 200 None 2026-03-31 17:17:14,323 - lava_callback - DEBUG - Processing node 69c95e98f2866272ea51e68d: baseline-arm64-baylibre 2026-03-31 17:17:14,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95e98f2866272ea51e68d 2026-03-31 17:17:14,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:17:14,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95e98f2866272ea51e68d HTTP/1.1" 200 None 2026-03-31 17:17:14,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:17:14,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:17:14,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:17:14,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95e98f2866272ea51e68d HTTP/1.1" 200 None 2026-03-31 17:17:14,381 - lava_callback - ERROR - Error processing callback for node 69c95e98f2866272ea51e68d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36292 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:18:48,358 - lava_callback - INFO - Received callback for node 69c95ee9f2866272ea51e69c INFO: 172.19.0.1:36308 - "POST /node/69c95ee9f2866272ea51e69c HTTP/1.0" 202 Accepted 2026-03-31 17:18:48,364 - lava_callback - INFO - Starting async job submit for node 69c95ee9f2866272ea51e69c 2026-03-31 17:18:48,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:18:48,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95ee9f2866272ea51e69c HTTP/1.1" 200 None 2026-03-31 17:18:48,380 - lava_callback - DEBUG - Processing node 69c95ee9f2866272ea51e69c: baseline-arm64-clabbe 2026-03-31 17:18:48,380 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95ee9f2866272ea51e69c 2026-03-31 17:18:48,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:18:48,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95ee9f2866272ea51e69c HTTP/1.1" 200 None 2026-03-31 17:18:48,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:18:48,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:18:48,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:18:48,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95ee9f2866272ea51e69c HTTP/1.1" 200 None 2026-03-31 17:18:48,446 - lava_callback - ERROR - Error processing callback for node 69c95ee9f2866272ea51e69c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60636 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:19:36,676 - lava_callback - INFO - Received callback for node 69cbfcb68df9359ff48983d9 INFO: 172.19.0.1:60644 - "POST /node/69cbfcb68df9359ff48983d9 HTTP/1.0" 202 Accepted 2026-03-31 17:19:36,685 - lava_callback - INFO - Starting async job submit for node 69cbfcb68df9359ff48983d9 2026-03-31 17:19:36,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:19:36,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb68df9359ff48983d9 HTTP/1.1" 200 None 2026-03-31 17:19:36,764 - lava_callback - DEBUG - Processing node 69cbfcb68df9359ff48983d9: baseline-nfs-arm64 2026-03-31 17:19:38,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:19:38,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:19:38,311 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb68df9359ff48983d9/log.txt.gz 2026-03-31 17:19:38,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:19:39,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:19:39,216 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb68df9359ff48983d9/lava-logs.yaml 2026-03-31 17:19:39,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:19:39,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:19:39,429 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb68df9359ff48983d9/lava_callback.json.gz DEBUG: Setup failed for 69cbfcb68df9359ff48983d9. Transitting job node result: incomplete -> fail 2026-03-31 17:19:39,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:19:39,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb68df9359ff48983d9 HTTP/1.1" 200 None 2026-03-31 17:19:39,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:19:39,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:19:39,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:19:39,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcb68df9359ff48983d9 HTTP/1.1" 200 None 2026-03-31 17:19:39,669 - lava_callback - INFO - Completed processing callback for node 69cbfcb68df9359ff48983d9 INFO: 172.19.0.1:34502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35170 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:19:59,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:19:59,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:45682 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:20:08,577 - lava_callback - INFO - Received callback for node 69c95f36f2866272ea51e6a3 INFO: 172.19.0.1:45688 - "POST /node/69c95f36f2866272ea51e6a3 HTTP/1.0" 202 Accepted 2026-03-31 17:20:08,585 - lava_callback - INFO - Starting async job submit for node 69c95f36f2866272ea51e6a3 2026-03-31 17:20:08,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:20:08,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95f36f2866272ea51e6a3 HTTP/1.1" 200 None 2026-03-31 17:20:08,615 - lava_callback - DEBUG - Processing node 69c95f36f2866272ea51e6a3: baseline-arm64-clabbe 2026-03-31 17:20:08,616 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95f36f2866272ea51e6a3 2026-03-31 17:20:08,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:20:08,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95f36f2866272ea51e6a3 HTTP/1.1" 200 None 2026-03-31 17:20:08,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:20:08,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:20:08,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:20:08,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95f36f2866272ea51e6a3 HTTP/1.1" 200 None 2026-03-31 17:20:08,678 - lava_callback - ERROR - Error processing callback for node 69c95f36f2866272ea51e6a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48924 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:20:29,842 - lava_callback - INFO - Received callback for node 69cc02548df9359ff489876b INFO: 172.19.0.1:45278 - "POST /node/69cc02548df9359ff489876b HTTP/1.0" 202 Accepted 2026-03-31 17:20:29,851 - lava_callback - INFO - Starting async job submit for node 69cc02548df9359ff489876b 2026-03-31 17:20:29,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:20:29,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc02548df9359ff489876b HTTP/1.1" 200 None 2026-03-31 17:20:29,883 - lava_callback - DEBUG - Processing node 69cc02548df9359ff489876b: baseline-nfs-arm64 2026-03-31 17:20:29,885 - lava_callback - WARNING - Failed to upload log 2026-03-31 17:20:29,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:20:30,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:20:30,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc02548df9359ff489876b/lava-logs.yaml 2026-03-31 17:20:30,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:20:30,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:20:30,159 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc02548df9359ff489876b/lava_callback.json.gz 2026-03-31 17:20:30,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:20:30,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc02548df9359ff489876b HTTP/1.1" 200 None 2026-03-31 17:20:30,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:20:30,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:20:30,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:20:30,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc02548df9359ff489876b HTTP/1.1" 200 None 2026-03-31 17:20:30,250 - lava_callback - INFO - Completed processing callback for node 69cc02548df9359ff489876b INFO: 172.19.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47686 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:20:59,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:20:59,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:48988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60218 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:21:28,498 - lava_callback - INFO - Received callback for node 69c95f84f2866272ea51e6b8 INFO: 172.19.0.1:60220 - "POST /node/69c95f84f2866272ea51e6b8 HTTP/1.0" 202 Accepted 2026-03-31 17:21:28,504 - lava_callback - INFO - Starting async job submit for node 69c95f84f2866272ea51e6b8 2026-03-31 17:21:28,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:21:28,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95f84f2866272ea51e6b8 HTTP/1.1" 200 None 2026-03-31 17:21:28,530 - lava_callback - DEBUG - Processing node 69c95f84f2866272ea51e6b8: baseline-arm64-clabbe 2026-03-31 17:21:28,531 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95f84f2866272ea51e6b8 2026-03-31 17:21:28,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:21:28,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95f84f2866272ea51e6b8 HTTP/1.1" 200 None 2026-03-31 17:21:28,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:21:28,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:21:28,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:21:28,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95f84f2866272ea51e6b8 HTTP/1.1" 200 None 2026-03-31 17:21:28,607 - lava_callback - ERROR - Error processing callback for node 69c95f84f2866272ea51e6b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50432 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:22:28,498 - lava_callback - INFO - Received callback for node 69c95fd1f2866272ea51e6c1 INFO: 172.19.0.1:50448 - "POST /node/69c95fd1f2866272ea51e6c1 HTTP/1.0" 202 Accepted 2026-03-31 17:22:28,504 - lava_callback - INFO - Starting async job submit for node 69c95fd1f2866272ea51e6c1 2026-03-31 17:22:28,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:28,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95fd1f2866272ea51e6c1 HTTP/1.1" 200 None 2026-03-31 17:22:28,524 - lava_callback - DEBUG - Processing node 69c95fd1f2866272ea51e6c1: baseline-arm64-clabbe 2026-03-31 17:22:28,524 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c95fd1f2866272ea51e6c1 2026-03-31 17:22:28,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:28,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c95fd1f2866272ea51e6c1 HTTP/1.1" 200 None 2026-03-31 17:22:28,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:28,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:22:28,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:28,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c95fd1f2866272ea51e6c1 HTTP/1.1" 200 None 2026-03-31 17:22:28,582 - lava_callback - ERROR - Error processing callback for node 69c95fd1f2866272ea51e6c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 17:22:32,604 - lava_callback - INFO - Received callback for node 69cc02a08df9359ff4898779 INFO: 172.19.0.1:39268 - "POST /node/69cc02a08df9359ff4898779 HTTP/1.0" 202 Accepted 2026-03-31 17:22:32,610 - lava_callback - INFO - Starting async job submit for node 69cc02a08df9359ff4898779 2026-03-31 17:22:32,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:32,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc02a08df9359ff4898779 HTTP/1.1" 200 None 2026-03-31 17:22:32,679 - lava_callback - DEBUG - Processing node 69cc02a08df9359ff4898779: baseline-arm64 2026-03-31 17:22:32,814 - lava_callback - WARNING - Failed to upload log 2026-03-31 17:22:32,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:22:33,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:22:33,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc02a08df9359ff4898779/lava-logs.yaml 2026-03-31 17:22:33,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:22:33,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:22:33,255 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc02a08df9359ff4898779/lava_callback.json.gz 2026-03-31 17:22:33,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:33,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc02a08df9359ff4898779 HTTP/1.1" 200 None 2026-03-31 17:22:33,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:33,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:22:33,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:33,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc02a08df9359ff4898779 HTTP/1.1" 200 None 2026-03-31 17:22:33,467 - lava_callback - INFO - Completed processing callback for node 69cc02a08df9359ff4898779 INFO: 172.19.0.1:39274 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:22:40,426 - lava_callback - INFO - Received callback for node 69cbfb838df9359ff48981c6 INFO: 172.19.0.1:38876 - "POST /node/69cbfb838df9359ff48981c6 HTTP/1.0" 202 Accepted 2026-03-31 17:22:40,432 - lava_callback - INFO - Starting async job submit for node 69cbfb838df9359ff48981c6 2026-03-31 17:22:40,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:40,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb838df9359ff48981c6 HTTP/1.1" 200 None 2026-03-31 17:22:40,509 - lava_callback - DEBUG - Processing node 69cbfb838df9359ff48981c6: ltp-smoke 2026-03-31 17:22:41,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:22:41,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:22:41,167 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cbfb838df9359ff48981c6/log.txt.gz 2026-03-31 17:22:41,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:22:41,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:22:41,843 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cbfb838df9359ff48981c6/lava-logs.yaml 2026-03-31 17:22:41,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:22:42,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:22:42,038 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cbfb838df9359ff48981c6/lava_callback.json.gz 2026-03-31 17:22:42,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:42,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfb838df9359ff48981c6 HTTP/1.1" 200 None 2026-03-31 17:22:42,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:42,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4198df9359ff4897d2e HTTP/1.1" 200 None 2026-03-31 17:22:42,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:42,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfb838df9359ff48981c6 HTTP/1.1" 200 None 2026-03-31 17:22:42,346 - lava_callback - INFO - Completed processing callback for node 69cbfb838df9359ff48981c6 INFO: 172.19.0.1:38882 - "GET /node/693f3c4724745582e399ea25%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:38890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49762 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:22:59,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:22:59,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59192 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:23:48,569 - lava_callback - INFO - Received callback for node 69c9601ef2866272ea51e6ce INFO: 172.19.0.1:59202 - "POST /node/69c9601ef2866272ea51e6ce HTTP/1.0" 202 Accepted 2026-03-31 17:23:48,579 - lava_callback - INFO - Starting async job submit for node 69c9601ef2866272ea51e6ce 2026-03-31 17:23:48,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:23:48,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9601ef2866272ea51e6ce HTTP/1.1" 200 None 2026-03-31 17:23:48,609 - lava_callback - DEBUG - Processing node 69c9601ef2866272ea51e6ce: baseline-arm64-clabbe 2026-03-31 17:23:48,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9601ef2866272ea51e6ce 2026-03-31 17:23:48,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:23:48,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9601ef2866272ea51e6ce HTTP/1.1" 200 None 2026-03-31 17:23:48,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:23:48,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:23:48,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:23:48,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9601ef2866272ea51e6ce HTTP/1.1" 200 None 2026-03-31 17:23:48,710 - lava_callback - ERROR - Error processing callback for node 69c9601ef2866272ea51e6ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49702 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:23:55,381 - lava_callback - INFO - Received callback for node 69cc02f28df9359ff489878c INFO: 172.19.0.1:49706 - "POST /node/69cc02f28df9359ff489878c HTTP/1.0" 202 Accepted 2026-03-31 17:23:55,392 - lava_callback - INFO - Starting async job submit for node 69cc02f28df9359ff489878c 2026-03-31 17:23:55,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:23:55,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc02f28df9359ff489878c HTTP/1.1" 200 None 2026-03-31 17:23:55,505 - lava_callback - DEBUG - Processing node 69cc02f28df9359ff489878c: baseline-arm64 2026-03-31 17:23:55,725 - lava_callback - WARNING - Failed to upload log 2026-03-31 17:23:55,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:23:56,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:23:56,334 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc02f28df9359ff489878c/lava-logs.yaml 2026-03-31 17:23:56,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:23:56,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:23:56,464 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc02f28df9359ff489878c/lava_callback.json.gz 2026-03-31 17:23:56,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:23:56,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc02f28df9359ff489878c HTTP/1.1" 200 None INFO: 172.19.0.1:49710 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:23:56,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:23:56,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:23:56,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:23:56,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc02f28df9359ff489878c HTTP/1.1" 200 None 2026-03-31 17:23:56,641 - lava_callback - INFO - Completed processing callback for node 69cc02f28df9359ff489878c 2026-03-31 17:23:59,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:23:59,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:58028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52982 - "GET /node/69bd8fea0e93e231472415f2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:40774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56358 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:25:08,477 - lava_callback - INFO - Received callback for node 69c9606af2866272ea51e6d9 INFO: 172.19.0.1:56362 - "POST /node/69c9606af2866272ea51e6d9 HTTP/1.0" 202 Accepted 2026-03-31 17:25:08,487 - lava_callback - INFO - Starting async job submit for node 69c9606af2866272ea51e6d9 2026-03-31 17:25:08,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:25:08,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9606af2866272ea51e6d9 HTTP/1.1" 200 None 2026-03-31 17:25:08,513 - lava_callback - DEBUG - Processing node 69c9606af2866272ea51e6d9: baseline-arm64-clabbe 2026-03-31 17:25:08,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9606af2866272ea51e6d9 2026-03-31 17:25:08,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:25:08,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9606af2866272ea51e6d9 HTTP/1.1" 200 None 2026-03-31 17:25:08,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:25:08,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 17:25:08,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:25:08,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9606af2866272ea51e6d9 HTTP/1.1" 200 None 2026-03-31 17:25:08,592 - lava_callback - ERROR - Error processing callback for node 69c9606af2866272ea51e6d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48284 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:25:45,242 - lava_callback - INFO - Received callback for node 69cc03468df9359ff489879f INFO: 172.19.0.1:54058 - "POST /node/69cc03468df9359ff489879f HTTP/1.0" 202 Accepted 2026-03-31 17:25:45,251 - lava_callback - INFO - Starting async job submit for node 69cc03468df9359ff489879f 2026-03-31 17:25:45,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:25:45,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc03468df9359ff489879f HTTP/1.1" 200 None 2026-03-31 17:25:45,295 - lava_callback - DEBUG - Processing node 69cc03468df9359ff489879f: baseline-arm64 2026-03-31 17:25:45,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:25:45,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:25:45,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc03468df9359ff489879f/log.txt.gz 2026-03-31 17:25:46,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:25:46,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:25:46,271 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc03468df9359ff489879f/lava-logs.yaml 2026-03-31 17:25:46,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:25:46,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:25:46,444 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc03468df9359ff489879f/lava_callback.json.gz 2026-03-31 17:25:46,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:25:46,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc03468df9359ff489879f HTTP/1.1" 200 None 2026-03-31 17:25:46,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:25:46,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:25:46,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:54060 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:25:46,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc03468df9359ff489879f HTTP/1.1" 200 None 2026-03-31 17:25:46,553 - lava_callback - INFO - Completed processing callback for node 69cc03468df9359ff489879f INFO: 172.19.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52266 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:25:59,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:25:59,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:57018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59426 - "GET /node/694a599583c67c43d1ed9f90%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-31 17:26:14,828 - lava_callback - INFO - Received callback for node 69c960aff2866272ea51e6e2 INFO: 172.19.0.1:59434 - "POST /node/69c960aff2866272ea51e6e2 HTTP/1.0" 202 Accepted 2026-03-31 17:26:14,834 - lava_callback - INFO - Starting async job submit for node 69c960aff2866272ea51e6e2 2026-03-31 17:26:14,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:26:14,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c960aff2866272ea51e6e2 HTTP/1.1" 200 None 2026-03-31 17:26:14,850 - lava_callback - DEBUG - Processing node 69c960aff2866272ea51e6e2: baseline-arm64-baylibre 2026-03-31 17:26:14,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c960aff2866272ea51e6e2 2026-03-31 17:26:14,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:26:14,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c960aff2866272ea51e6e2 HTTP/1.1" 200 None 2026-03-31 17:26:14,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:26:14,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 17:26:14,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:26:14,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c960aff2866272ea51e6e2 HTTP/1.1" 200 None 2026-03-31 17:26:14,915 - lava_callback - ERROR - Error processing callback for node 69c960aff2866272ea51e6e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52918 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:27:46,213 - lava_callback - INFO - Received callback for node 69cc03c88df9359ff48987c0 INFO: 172.19.0.1:46542 - "POST /node/69cc03c88df9359ff48987c0 HTTP/1.0" 202 Accepted 2026-03-31 17:27:46,219 - lava_callback - INFO - Starting async job submit for node 69cc03c88df9359ff48987c0 2026-03-31 17:27:46,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:27:46,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc03c88df9359ff48987c0 HTTP/1.1" 200 None 2026-03-31 17:27:46,259 - lava_callback - DEBUG - Processing node 69cc03c88df9359ff48987c0: baseline-arm64 INFO: 172.19.0.1:46550 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:27:46,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:27:46,874 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:27:46,875 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc03c88df9359ff48987c0/log.txt.gz 2026-03-31 17:27:47,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:27:47,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:27:47,356 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc03c88df9359ff48987c0/lava-logs.yaml 2026-03-31 17:27:47,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:27:47,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:27:47,521 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc03c88df9359ff48987c0/lava_callback.json.gz 2026-03-31 17:27:47,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:27:47,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc03c88df9359ff48987c0 HTTP/1.1" 200 None 2026-03-31 17:27:47,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:27:47,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:27:47,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:27:47,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc03c88df9359ff48987c0 HTTP/1.1" 200 None 2026-03-31 17:27:47,606 - lava_callback - INFO - Completed processing callback for node 69cc03c88df9359ff48987c0 INFO: 172.19.0.1:56652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56662 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:27:59,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:27:59,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:35520 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:28:14,924 - lava_callback - INFO - Received callback for node 69c96122f2866272ea51e6f8 INFO: 172.19.0.1:58434 - "POST /node/69c96122f2866272ea51e6f8 HTTP/1.0" 202 Accepted 2026-03-31 17:28:14,934 - lava_callback - INFO - Starting async job submit for node 69c96122f2866272ea51e6f8 2026-03-31 17:28:14,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:28:14,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96122f2866272ea51e6f8 HTTP/1.1" 200 None 2026-03-31 17:28:14,965 - lava_callback - DEBUG - Processing node 69c96122f2866272ea51e6f8: baseline-arm64-baylibre 2026-03-31 17:28:14,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c96122f2866272ea51e6f8 2026-03-31 17:28:14,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:28:14,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96122f2866272ea51e6f8 HTTP/1.1" 200 None 2026-03-31 17:28:14,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:28:15,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 17:28:15,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:28:15,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96122f2866272ea51e6f8 HTTP/1.1" 200 None 2026-03-31 17:28:15,090 - lava_callback - ERROR - Error processing callback for node 69c96122f2866272ea51e6f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54048 - "GET /node/693b3e604b47c3f87266e3f7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:54054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54066 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:29:17,217 - lava_callback - INFO - Received callback for node 69cc04228df9359ff48987d2 INFO: 172.19.0.1:54068 - "POST /node/69cc04228df9359ff48987d2 HTTP/1.0" 202 Accepted 2026-03-31 17:29:17,225 - lava_callback - INFO - Starting async job submit for node 69cc04228df9359ff48987d2 2026-03-31 17:29:17,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:29:17,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc04228df9359ff48987d2 HTTP/1.1" 200 None 2026-03-31 17:29:17,276 - lava_callback - DEBUG - Processing node 69cc04228df9359ff48987d2: baseline-nfs-arm64 2026-03-31 17:29:17,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:29:17,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:29:17,836 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc04228df9359ff48987d2/log.txt.gz 2026-03-31 17:29:18,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:29:18,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:29:18,282 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc04228df9359ff48987d2/lava-logs.yaml 2026-03-31 17:29:18,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:29:18,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:29:18,486 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc04228df9359ff48987d2/lava_callback.json.gz 2026-03-31 17:29:18,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:29:18,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc04228df9359ff48987d2 HTTP/1.1" 200 None 2026-03-31 17:29:18,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:29:18,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:29:18,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:29:18,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc04228df9359ff48987d2 HTTP/1.1" 200 None 2026-03-31 17:29:18,587 - lava_callback - INFO - Completed processing callback for node 69cc04228df9359ff48987d2 INFO: 172.19.0.1:42214 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:29:29,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:29:29,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:57114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58680 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:30:15,163 - lava_callback - INFO - Received callback for node 69c96192f2866272ea51e70c INFO: 172.19.0.1:54870 - "POST /node/69c96192f2866272ea51e70c HTTP/1.0" 202 Accepted 2026-03-31 17:30:15,169 - lava_callback - INFO - Starting async job submit for node 69c96192f2866272ea51e70c INFO: 172.19.0.1:54882 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 17:30:15,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:30:15,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96192f2866272ea51e70c HTTP/1.1" 200 None 2026-03-31 17:30:15,192 - lava_callback - DEBUG - Processing node 69c96192f2866272ea51e70c: baseline-arm64-baylibre 2026-03-31 17:30:15,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c96192f2866272ea51e70c 2026-03-31 17:30:15,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:30:15,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96192f2866272ea51e70c HTTP/1.1" 200 None 2026-03-31 17:30:15,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:30:15,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 17:30:15,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:30:15,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96192f2866272ea51e70c HTTP/1.1" 200 None 2026-03-31 17:30:15,254 - lava_callback - ERROR - Error processing callback for node 69c96192f2866272ea51e70c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36408 - "GET /node/6939ad945206d8a4495e959d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53246 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:31:08,815 - lava_callback - INFO - Received callback for node 69c961d5f2866272ea51e71a INFO: 172.19.0.1:53254 - "POST /node/69c961d5f2866272ea51e71a HTTP/1.0" 202 Accepted 2026-03-31 17:31:08,821 - lava_callback - INFO - Starting async job submit for node 69c961d5f2866272ea51e71a 2026-03-31 17:31:08,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:31:08,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c961d5f2866272ea51e71a HTTP/1.1" 200 None 2026-03-31 17:31:08,840 - lava_callback - DEBUG - Processing node 69c961d5f2866272ea51e71a: baseline-arm-clabbe 2026-03-31 17:31:08,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c961d5f2866272ea51e71a 2026-03-31 17:31:08,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:31:08,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c961d5f2866272ea51e71a HTTP/1.1" 200 None 2026-03-31 17:31:08,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:31:08,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 17:31:08,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:31:08,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c961d5f2866272ea51e71a HTTP/1.1" 200 None 2026-03-31 17:31:08,902 - lava_callback - ERROR - Error processing callback for node 69c961d5f2866272ea51e71a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55200 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39128 - "GET /node/69bde8c7d54b5122738c6479%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:39134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35224 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46492 - "GET /node/69862f66177ca0f097bd2421%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:46494 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:33:55,474 - lava_callback - INFO - Received callback for node 69c96272f2866272ea51e73a INFO: 172.19.0.1:46502 - "POST /node/69c96272f2866272ea51e73a HTTP/1.0" 202 Accepted 2026-03-31 17:33:55,480 - lava_callback - INFO - Starting async job submit for node 69c96272f2866272ea51e73a 2026-03-31 17:33:55,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:33:55,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96272f2866272ea51e73a HTTP/1.1" 200 None 2026-03-31 17:33:55,496 - lava_callback - DEBUG - Processing node 69c96272f2866272ea51e73a: baseline-arm64-baylibre 2026-03-31 17:33:55,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c96272f2866272ea51e73a 2026-03-31 17:33:55,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:33:55,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96272f2866272ea51e73a HTTP/1.1" 200 None 2026-03-31 17:33:55,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:33:55,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 17:33:55,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:33:55,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96272f2866272ea51e73a HTTP/1.1" 200 None 2026-03-31 17:33:55,550 - lava_callback - ERROR - Error processing callback for node 69c96272f2866272ea51e73a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43880 - "GET /node/69bde8c3d54b5122738c646c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:43896 - "GET /node/68ba037f09efb230970abe5e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:33060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53178 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:36:29,070 - lava_callback - INFO - Received callback for node 69c96318f2866272ea51e754 INFO: 172.19.0.1:53186 - "POST /node/69c96318f2866272ea51e754 HTTP/1.0" 202 Accepted 2026-03-31 17:36:29,076 - lava_callback - INFO - Starting async job submit for node 69c96318f2866272ea51e754 2026-03-31 17:36:29,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:36:29,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96318f2866272ea51e754 HTTP/1.1" 200 None 2026-03-31 17:36:29,092 - lava_callback - DEBUG - Processing node 69c96318f2866272ea51e754: baseline-arm64-clabbe 2026-03-31 17:36:29,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c96318f2866272ea51e754 2026-03-31 17:36:29,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:36:29,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96318f2866272ea51e754 HTTP/1.1" 200 None 2026-03-31 17:36:29,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:36:29,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 17:36:29,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:36:29,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96318f2866272ea51e754 HTTP/1.1" 200 None 2026-03-31 17:36:29,179 - lava_callback - ERROR - Error processing callback for node 69c96318f2866272ea51e754: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41696 - "GET /node/69bd7c890e93e2314723e938%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:41700 - "GET /node/694aad2183c67c43d1ede5ba%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:41702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33170 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:37:26,759 - lava_callback - INFO - Received callback for node 69cc05a08df9359ff4898824 INFO: 172.19.0.1:33172 - "POST /node/69cc05a08df9359ff4898824 HTTP/1.0" 202 Accepted 2026-03-31 17:37:26,767 - lava_callback - INFO - Starting async job submit for node 69cc05a08df9359ff4898824 2026-03-31 17:37:26,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:26,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc05a08df9359ff4898824 HTTP/1.1" 200 None 2026-03-31 17:37:26,821 - lava_callback - DEBUG - Processing node 69cc05a08df9359ff4898824: baseline-arm64-foundriesio 2026-03-31 17:37:27,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:37:27,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:37:27,880 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc05a08df9359ff4898824/log.txt.gz 2026-03-31 17:37:28,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:37:28,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:37:28,621 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc05a08df9359ff4898824/lava-logs.yaml 2026-03-31 17:37:28,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:37:28,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:37:28,849 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc05a08df9359ff4898824/lava_callback.json.gz DEBUG: 69cc05a08df9359ff4898824 Transitting job node result: pass -> incomplete 2026-03-31 17:37:28,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:28,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc05a08df9359ff4898824 HTTP/1.1" 200 None 2026-03-31 17:37:28,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:28,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:37:28,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:28,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc05a08df9359ff4898824 HTTP/1.1" 200 None 2026-03-31 17:37:28,946 - lava_callback - INFO - Completed processing callback for node 69cc05a08df9359ff4898824 2026-03-31 17:37:29,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:29,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 17:37:35,031 - lava_callback - INFO - Received callback for node 69cc06578df9359ff4898840 INFO: 172.19.0.1:57814 - "POST /node/69cc06578df9359ff4898840 HTTP/1.0" 202 Accepted 2026-03-31 17:37:35,037 - lava_callback - INFO - Starting async job submit for node 69cc06578df9359ff4898840 2026-03-31 17:37:35,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:35,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc06578df9359ff4898840 HTTP/1.1" 200 None 2026-03-31 17:37:35,055 - lava_callback - DEBUG - Processing node 69cc06578df9359ff4898840: baseline-arm 2026-03-31 17:37:35,061 - lava_callback - WARNING - Failed to upload log 2026-03-31 17:37:35,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:37:35,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:37:35,196 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc06578df9359ff4898840/lava-logs.yaml 2026-03-31 17:37:35,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:37:35,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:37:35,321 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc06578df9359ff4898840/lava_callback.json.gz 2026-03-31 17:37:35,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:35,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc06578df9359ff4898840 HTTP/1.1" 200 None 2026-03-31 17:37:35,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:35,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 17:37:35,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:35,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc06578df9359ff4898840 HTTP/1.1" 200 None 2026-03-31 17:37:35,387 - lava_callback - INFO - Completed processing callback for node 69cc06578df9359ff4898840 INFO: 172.19.0.1:57824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54480 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:37:49,110 - lava_callback - INFO - Received callback for node 69c96368f2866272ea51e769 INFO: 172.19.0.1:54486 - "POST /node/69c96368f2866272ea51e769 HTTP/1.0" 202 Accepted 2026-03-31 17:37:49,115 - lava_callback - INFO - Starting async job submit for node 69c96368f2866272ea51e769 2026-03-31 17:37:49,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:49,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96368f2866272ea51e769 HTTP/1.1" 200 None 2026-03-31 17:37:49,151 - lava_callback - DEBUG - Processing node 69c96368f2866272ea51e769: baseline-arm64-clabbe 2026-03-31 17:37:49,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c96368f2866272ea51e769 2026-03-31 17:37:49,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:49,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96368f2866272ea51e769 HTTP/1.1" 200 None 2026-03-31 17:37:49,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:49,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 17:37:49,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:49,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96368f2866272ea51e769 HTTP/1.1" 200 None 2026-03-31 17:37:49,217 - lava_callback - ERROR - Error processing callback for node 69c96368f2866272ea51e769: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34018 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:37:59,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:37:59,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:43544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49910 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35304 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:39:35,538 - lava_callback - INFO - Received callback for node 69c963c7f2866272ea51e780 INFO: 172.19.0.1:35830 - "POST /node/69c963c7f2866272ea51e780 HTTP/1.0" 202 Accepted 2026-03-31 17:39:35,543 - lava_callback - INFO - Starting async job submit for node 69c963c7f2866272ea51e780 2026-03-31 17:39:35,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:39:35,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c963c7f2866272ea51e780 HTTP/1.1" 200 None 2026-03-31 17:39:35,558 - lava_callback - DEBUG - Processing node 69c963c7f2866272ea51e780: baseline-arm-baylibre 2026-03-31 17:39:35,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c963c7f2866272ea51e780 2026-03-31 17:39:35,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:39:35,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c963c7f2866272ea51e780 HTTP/1.1" 200 None 2026-03-31 17:39:35,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:39:35,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:39:35,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:39:35,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c963c7f2866272ea51e780 HTTP/1.1" 200 None 2026-03-31 17:39:35,631 - lava_callback - ERROR - Error processing callback for node 69c963c7f2866272ea51e780: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59516 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 17:40:15,504 - lava_callback - INFO - Received callback for node 69c963ecf2866272ea51e78a INFO: 172.19.0.1:59526 - "POST /node/69c963ecf2866272ea51e78a HTTP/1.0" 202 Accepted 2026-03-31 17:40:15,511 - lava_callback - INFO - Starting async job submit for node 69c963ecf2866272ea51e78a 2026-03-31 17:40:15,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:40:15,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c963ecf2866272ea51e78a HTTP/1.1" 200 None 2026-03-31 17:40:15,559 - lava_callback - DEBUG - Processing node 69c963ecf2866272ea51e78a: baseline-arm-baylibre 2026-03-31 17:40:15,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c963ecf2866272ea51e78a 2026-03-31 17:40:15,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:40:15,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c963ecf2866272ea51e78a HTTP/1.1" 200 None 2026-03-31 17:40:15,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:40:15,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:40:15,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:40:15,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c963ecf2866272ea51e78a HTTP/1.1" 200 None 2026-03-31 17:40:15,615 - lava_callback - ERROR - Error processing callback for node 69c963ecf2866272ea51e78a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51558 - "GET /node/698b071d3b3beb19e7617fb4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:51566 - "GET /node/69861eea177ca0f097bd180d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:51574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43990 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:41:09,337 - lava_callback - INFO - Received callback for node 69c96424f2866272ea51e796 INFO: 172.19.0.1:43994 - "POST /node/69c96424f2866272ea51e796 HTTP/1.0" 202 Accepted 2026-03-31 17:41:09,343 - lava_callback - INFO - Starting async job submit for node 69c96424f2866272ea51e796 2026-03-31 17:41:09,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:41:09,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96424f2866272ea51e796 HTTP/1.1" 200 None 2026-03-31 17:41:09,359 - lava_callback - DEBUG - Processing node 69c96424f2866272ea51e796: baseline-arm64-clabbe 2026-03-31 17:41:09,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c96424f2866272ea51e796 2026-03-31 17:41:09,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:41:09,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96424f2866272ea51e796 HTTP/1.1" 200 None 2026-03-31 17:41:09,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:41:09,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 17:41:09,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:41:09,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96424f2866272ea51e796 HTTP/1.1" 200 None 2026-03-31 17:41:09,424 - lava_callback - ERROR - Error processing callback for node 69c96424f2866272ea51e796: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52984 - "GET /node/69926a2298a8bdb31c392531%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:47418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57212 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:41:55,716 - lava_callback - INFO - Received callback for node 69c96461f2866272ea51e7a2 INFO: 172.19.0.1:57218 - "POST /node/69c96461f2866272ea51e7a2 HTTP/1.0" 202 Accepted 2026-03-31 17:41:55,724 - lava_callback - INFO - Starting async job submit for node 69c96461f2866272ea51e7a2 2026-03-31 17:41:55,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:41:55,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96461f2866272ea51e7a2 HTTP/1.1" 200 None 2026-03-31 17:41:55,741 - lava_callback - DEBUG - Processing node 69c96461f2866272ea51e7a2: baseline-arm-baylibre 2026-03-31 17:41:55,741 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c96461f2866272ea51e7a2 2026-03-31 17:41:55,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:41:55,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96461f2866272ea51e7a2 HTTP/1.1" 200 None 2026-03-31 17:41:55,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:41:55,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:41:55,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:41:55,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96461f2866272ea51e7a2 HTTP/1.1" 200 None 2026-03-31 17:41:55,802 - lava_callback - ERROR - Error processing callback for node 69c96461f2866272ea51e7a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55804 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:42:35,645 - lava_callback - INFO - Received callback for node 69c96486f2866272ea51e7b4 INFO: 172.19.0.1:43226 - "POST /node/69c96486f2866272ea51e7b4 HTTP/1.0" 202 Accepted 2026-03-31 17:42:35,651 - lava_callback - INFO - Starting async job submit for node 69c96486f2866272ea51e7b4 2026-03-31 17:42:35,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:42:35,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96486f2866272ea51e7b4 HTTP/1.1" 200 None 2026-03-31 17:42:35,676 - lava_callback - DEBUG - Processing node 69c96486f2866272ea51e7b4: baseline-arm-baylibre 2026-03-31 17:42:35,676 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c96486f2866272ea51e7b4 2026-03-31 17:42:35,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:42:35,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96486f2866272ea51e7b4 HTTP/1.1" 200 None 2026-03-31 17:42:35,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:42:35,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:42:35,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:42:35,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96486f2866272ea51e7b4 HTTP/1.1" 200 None 2026-03-31 17:42:35,736 - lava_callback - ERROR - Error processing callback for node 69c96486f2866272ea51e7b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51942 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:43:14,810 - lava_callback - INFO - Received callback for node 69cc05e08df9359ff4898829 INFO: 172.19.0.1:41448 - "POST /node/69cc05e08df9359ff4898829 HTTP/1.0" 202 Accepted 2026-03-31 17:43:14,818 - lava_callback - INFO - Starting async job submit for node 69cc05e08df9359ff4898829 2026-03-31 17:43:14,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:14,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc05e08df9359ff4898829 HTTP/1.1" 200 None 2026-03-31 17:43:14,868 - lava_callback - DEBUG - Processing node 69cc05e08df9359ff4898829: baseline-arm64 INFO: 172.19.0.1:41464 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 17:43:15,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:43:15,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:43:15,442 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc05e08df9359ff4898829/log.txt.gz 2026-03-31 17:43:15,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:43:15,743 - lava_callback - INFO - Received callback for node 69c964acf2866272ea51e7be INFO: 172.19.0.1:41468 - "POST /node/69c964acf2866272ea51e7be HTTP/1.0" 202 Accepted 2026-03-31 17:43:15,753 - lava_callback - INFO - Starting async job submit for node 69c964acf2866272ea51e7be 2026-03-31 17:43:15,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:15,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c964acf2866272ea51e7be HTTP/1.1" 200 None 2026-03-31 17:43:15,770 - lava_callback - DEBUG - Processing node 69c964acf2866272ea51e7be: baseline-arm-baylibre 2026-03-31 17:43:15,770 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c964acf2866272ea51e7be 2026-03-31 17:43:15,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:15,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c964acf2866272ea51e7be HTTP/1.1" 200 None 2026-03-31 17:43:15,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:15,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:43:15,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:15,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c964acf2866272ea51e7be HTTP/1.1" 200 None 2026-03-31 17:43:15,841 - lava_callback - ERROR - Error processing callback for node 69c964acf2866272ea51e7be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 17:43:15,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:43:15,977 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc05e08df9359ff4898829/lava-logs.yaml 2026-03-31 17:43:15,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:43:16,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:43:16,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc05e08df9359ff4898829/lava_callback.json.gz DEBUG: Setup failed for 69cc05e08df9359ff4898829. Transitting job node result: incomplete -> fail 2026-03-31 17:43:16,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:16,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc05e08df9359ff4898829 HTTP/1.1" 200 None 2026-03-31 17:43:16,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:16,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:43:16,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:16,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc05e08df9359ff4898829 HTTP/1.1" 200 None 2026-03-31 17:43:16,406 - lava_callback - INFO - Completed processing callback for node 69cc05e08df9359ff4898829 INFO: 172.19.0.1:41472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50570 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:43:29,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:29,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:39368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55010 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:43:55,703 - lava_callback - INFO - Received callback for node 69c964d0f2866272ea51e7c7 INFO: 172.19.0.1:55018 - "POST /node/69c964d0f2866272ea51e7c7 HTTP/1.0" 202 Accepted 2026-03-31 17:43:55,709 - lava_callback - INFO - Starting async job submit for node 69c964d0f2866272ea51e7c7 2026-03-31 17:43:55,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:55,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c964d0f2866272ea51e7c7 HTTP/1.1" 200 None 2026-03-31 17:43:55,727 - lava_callback - DEBUG - Processing node 69c964d0f2866272ea51e7c7: baseline-arm-baylibre 2026-03-31 17:43:55,727 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c964d0f2866272ea51e7c7 2026-03-31 17:43:55,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:55,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c964d0f2866272ea51e7c7 HTTP/1.1" 200 None 2026-03-31 17:43:55,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:55,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:43:55,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:43:55,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c964d0f2866272ea51e7c7 HTTP/1.1" 200 None 2026-03-31 17:43:55,791 - lava_callback - ERROR - Error processing callback for node 69c964d0f2866272ea51e7c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54900 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:44:29,463 - lava_callback - INFO - Received callback for node 69c964f7f2866272ea51e7d3 INFO: 172.19.0.1:54902 - "POST /node/69c964f7f2866272ea51e7d3 HTTP/1.0" 202 Accepted 2026-03-31 17:44:29,469 - lava_callback - INFO - Starting async job submit for node 69c964f7f2866272ea51e7d3 2026-03-31 17:44:29,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:44:29,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c964f7f2866272ea51e7d3 HTTP/1.1" 200 None 2026-03-31 17:44:29,486 - lava_callback - DEBUG - Processing node 69c964f7f2866272ea51e7d3: baseline-arm-clabbe 2026-03-31 17:44:29,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c964f7f2866272ea51e7d3 2026-03-31 17:44:29,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:44:29,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c964f7f2866272ea51e7d3 HTTP/1.1" 200 None 2026-03-31 17:44:29,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:44:29,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:44:29,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:44:29,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c964f7f2866272ea51e7d3 HTTP/1.1" 200 None 2026-03-31 17:44:29,612 - lava_callback - ERROR - Error processing callback for node 69c964f7f2866272ea51e7d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59104 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:45:09,419 - lava_callback - INFO - Received callback for node 69c9651df2866272ea51e7de INFO: 172.19.0.1:59106 - "POST /node/69c9651df2866272ea51e7de HTTP/1.0" 202 Accepted 2026-03-31 17:45:09,425 - lava_callback - INFO - Starting async job submit for node 69c9651df2866272ea51e7de 2026-03-31 17:45:09,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:45:09,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9651df2866272ea51e7de HTTP/1.1" 200 None 2026-03-31 17:45:09,444 - lava_callback - DEBUG - Processing node 69c9651df2866272ea51e7de: baseline-arm-clabbe 2026-03-31 17:45:09,444 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9651df2866272ea51e7de 2026-03-31 17:45:09,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:45:09,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9651df2866272ea51e7de HTTP/1.1" 200 None 2026-03-31 17:45:09,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:45:09,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:45:09,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:45:09,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9651df2866272ea51e7de HTTP/1.1" 200 None 2026-03-31 17:45:09,504 - lava_callback - ERROR - Error processing callback for node 69c9651df2866272ea51e7de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54626 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:45:49,538 - lava_callback - INFO - Received callback for node 69c96541f2866272ea51e7e8 INFO: 172.19.0.1:54636 - "POST /node/69c96541f2866272ea51e7e8 HTTP/1.0" 202 Accepted 2026-03-31 17:45:49,544 - lava_callback - INFO - Starting async job submit for node 69c96541f2866272ea51e7e8 2026-03-31 17:45:49,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:45:49,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96541f2866272ea51e7e8 HTTP/1.1" 200 None 2026-03-31 17:45:49,565 - lava_callback - DEBUG - Processing node 69c96541f2866272ea51e7e8: baseline-arm-clabbe 2026-03-31 17:45:49,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c96541f2866272ea51e7e8 2026-03-31 17:45:49,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:45:49,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96541f2866272ea51e7e8 HTTP/1.1" 200 None 2026-03-31 17:45:49,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:45:49,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:45:49,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:45:49,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96541f2866272ea51e7e8 HTTP/1.1" 200 None 2026-03-31 17:45:49,642 - lava_callback - ERROR - Error processing callback for node 69c96541f2866272ea51e7e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53698 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:46:29,508 - lava_callback - INFO - Received callback for node 69c96569f2866272ea51e7f2 INFO: 172.19.0.1:53704 - "POST /node/69c96569f2866272ea51e7f2 HTTP/1.0" 202 Accepted 2026-03-31 17:46:29,514 - lava_callback - INFO - Starting async job submit for node 69c96569f2866272ea51e7f2 2026-03-31 17:46:29,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:46:29,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96569f2866272ea51e7f2 HTTP/1.1" 200 None 2026-03-31 17:46:29,530 - lava_callback - DEBUG - Processing node 69c96569f2866272ea51e7f2: baseline-arm-clabbe 2026-03-31 17:46:29,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c96569f2866272ea51e7f2 2026-03-31 17:46:29,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:46:29,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c96569f2866272ea51e7f2 HTTP/1.1" 200 None 2026-03-31 17:46:29,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:46:29,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:46:29,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:46:29,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c96569f2866272ea51e7f2 HTTP/1.1" 200 None 2026-03-31 17:46:29,602 - lava_callback - ERROR - Error processing callback for node 69c96569f2866272ea51e7f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45176 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:47:09,589 - lava_callback - INFO - Received callback for node 69c9658ef2866272ea51e7fc INFO: 172.19.0.1:45190 - "POST /node/69c9658ef2866272ea51e7fc HTTP/1.0" 202 Accepted 2026-03-31 17:47:09,594 - lava_callback - INFO - Starting async job submit for node 69c9658ef2866272ea51e7fc 2026-03-31 17:47:09,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:47:09,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9658ef2866272ea51e7fc HTTP/1.1" 200 None 2026-03-31 17:47:09,611 - lava_callback - DEBUG - Processing node 69c9658ef2866272ea51e7fc: baseline-arm-clabbe 2026-03-31 17:47:09,611 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9658ef2866272ea51e7fc 2026-03-31 17:47:09,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:47:09,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9658ef2866272ea51e7fc HTTP/1.1" 200 None 2026-03-31 17:47:09,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:47:09,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-03-31 17:47:09,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:47:09,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9658ef2866272ea51e7fc HTTP/1.1" 200 None 2026-03-31 17:47:09,674 - lava_callback - ERROR - Error processing callback for node 69c9658ef2866272ea51e7fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49666 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:49:09,683 - lava_callback - INFO - Received callback for node 69c9660ef2866272ea51e818 INFO: 172.19.0.1:49668 - "POST /node/69c9660ef2866272ea51e818 HTTP/1.0" 202 Accepted 2026-03-31 17:49:09,689 - lava_callback - INFO - Starting async job submit for node 69c9660ef2866272ea51e818 2026-03-31 17:49:09,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:49:09,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9660ef2866272ea51e818 HTTP/1.1" 200 None 2026-03-31 17:49:09,705 - lava_callback - DEBUG - Processing node 69c9660ef2866272ea51e818: baseline-arm64-clabbe 2026-03-31 17:49:09,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9660ef2866272ea51e818 2026-03-31 17:49:09,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:49:09,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9660ef2866272ea51e818 HTTP/1.1" 200 None 2026-03-31 17:49:09,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:49:09,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 17:49:09,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:49:09,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9660ef2866272ea51e818 HTTP/1.1" 200 None 2026-03-31 17:49:09,769 - lava_callback - ERROR - Error processing callback for node 69c9660ef2866272ea51e818: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43168 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53826 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:53:09,844 - lava_callback - INFO - Received callback for node 69c966f8f2866272ea51e83e INFO: 172.19.0.1:34740 - "POST /node/69c966f8f2866272ea51e83e HTTP/1.0" 202 Accepted 2026-03-31 17:53:09,850 - lava_callback - INFO - Starting async job submit for node 69c966f8f2866272ea51e83e 2026-03-31 17:53:09,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:53:09,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c966f8f2866272ea51e83e HTTP/1.1" 200 None 2026-03-31 17:53:09,867 - lava_callback - DEBUG - Processing node 69c966f8f2866272ea51e83e: baseline-arm64-clabbe 2026-03-31 17:53:09,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c966f8f2866272ea51e83e 2026-03-31 17:53:09,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:53:09,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c966f8f2866272ea51e83e HTTP/1.1" 200 None 2026-03-31 17:53:09,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:53:09,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 17:53:09,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:53:09,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c966f8f2866272ea51e83e HTTP/1.1" 200 None 2026-03-31 17:53:09,929 - lava_callback - ERROR - Error processing callback for node 69c966f8f2866272ea51e83e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38794 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:54:29,883 - lava_callback - INFO - Received callback for node 69c9674bf2866272ea51e84e INFO: 172.19.0.1:52668 - "POST /node/69c9674bf2866272ea51e84e HTTP/1.0" 202 Accepted 2026-03-31 17:54:29,889 - lava_callback - INFO - Starting async job submit for node 69c9674bf2866272ea51e84e 2026-03-31 17:54:29,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:54:29,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9674bf2866272ea51e84e HTTP/1.1" 200 None 2026-03-31 17:54:29,906 - lava_callback - DEBUG - Processing node 69c9674bf2866272ea51e84e: baseline-arm64-clabbe 2026-03-31 17:54:29,907 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9674bf2866272ea51e84e 2026-03-31 17:54:29,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:54:29,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9674bf2866272ea51e84e HTTP/1.1" 200 None 2026-03-31 17:54:29,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:54:29,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 17:54:29,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:54:29,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9674bf2866272ea51e84e HTTP/1.1" 200 None 2026-03-31 17:54:29,962 - lava_callback - ERROR - Error processing callback for node 69c9674bf2866272ea51e84e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60200 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:55:06,722 - lava_callback - INFO - Received callback for node 69cc062f8df9359ff4898834 INFO: 172.19.0.1:60208 - "POST /node/69cc062f8df9359ff4898834 HTTP/1.0" 202 Accepted 2026-03-31 17:55:06,729 - lava_callback - INFO - Starting async job submit for node 69cc062f8df9359ff4898834 2026-03-31 17:55:06,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:55:06,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc062f8df9359ff4898834 HTTP/1.1" 200 None 2026-03-31 17:55:06,797 - lava_callback - DEBUG - Processing node 69cc062f8df9359ff4898834: baseline-arm 2026-03-31 17:55:07,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:55:07,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:55:07,562 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc062f8df9359ff4898834/log.txt.gz 2026-03-31 17:55:07,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:55:08,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:55:08,231 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc062f8df9359ff4898834/lava-logs.yaml 2026-03-31 17:55:08,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:55:08,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:55:08,432 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc062f8df9359ff4898834/lava_callback.json.gz DEBUG: Setup failed for 69cc062f8df9359ff4898834. Transitting job node result: incomplete -> fail 2026-03-31 17:55:08,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:55:08,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc062f8df9359ff4898834 HTTP/1.1" 200 None 2026-03-31 17:55:08,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:55:08,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 17:55:08,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:55:08,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc062f8df9359ff4898834 HTTP/1.1" 200 None 2026-03-31 17:55:08,753 - lava_callback - INFO - Completed processing callback for node 69cc062f8df9359ff4898834 INFO: 172.19.0.1:39742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43812 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:55:29,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:55:29,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-31 17:55:29,941 - lava_callback - INFO - Received callback for node 69c9678df2866272ea51e858 INFO: 172.19.0.1:52316 - "POST /node/69c9678df2866272ea51e858 HTTP/1.0" 202 Accepted 2026-03-31 17:55:29,946 - lava_callback - INFO - Starting async job submit for node 69c9678df2866272ea51e858 2026-03-31 17:55:29,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:55:29,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9678df2866272ea51e858 HTTP/1.1" 200 None 2026-03-31 17:55:29,965 - lava_callback - DEBUG - Processing node 69c9678df2866272ea51e858: baseline-arm-clabbe 2026-03-31 17:55:29,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9678df2866272ea51e858 2026-03-31 17:55:29,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:55:29,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9678df2866272ea51e858 HTTP/1.1" 200 None 2026-03-31 17:55:29,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:55:30,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-31 17:55:30,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:55:30,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9678df2866272ea51e858 HTTP/1.1" 200 None 2026-03-31 17:55:30,033 - lava_callback - ERROR - Error processing callback for node 69c9678df2866272ea51e858: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34760 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:56:31,497 - lava_callback - INFO - Received callback for node 69cc0abb8df9359ff489890f INFO: 172.19.0.1:33686 - "POST /node/69cc0abb8df9359ff489890f HTTP/1.0" 202 Accepted 2026-03-31 17:56:31,503 - lava_callback - INFO - Starting async job submit for node 69cc0abb8df9359ff489890f 2026-03-31 17:56:31,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:56:31,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc0abb8df9359ff489890f HTTP/1.1" 200 None 2026-03-31 17:56:31,521 - lava_callback - DEBUG - Processing node 69cc0abb8df9359ff489890f: baseline-nfs-arm64 2026-03-31 17:56:31,523 - lava_callback - WARNING - Failed to upload log 2026-03-31 17:56:31,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:56:31,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:56:31,647 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc0abb8df9359ff489890f/lava-logs.yaml 2026-03-31 17:56:31,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 17:56:31,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 17:56:31,775 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc0abb8df9359ff489890f/lava_callback.json.gz 2026-03-31 17:56:31,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:56:31,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc0abb8df9359ff489890f HTTP/1.1" 200 None 2026-03-31 17:56:31,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:56:31,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 17:56:31,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:56:31,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc0abb8df9359ff489890f HTTP/1.1" 200 None 2026-03-31 17:56:31,831 - lava_callback - INFO - Completed processing callback for node 69cc0abb8df9359ff489890f INFO: 172.19.0.1:33688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33768 - "GET / HTTP/1.0" 200 OK 2026-03-31 17:56:59,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 17:56:59,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:49112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52174 - "GET /node/69bd7b060e93e2314723e7b3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35234 - "GET /node/693b3ada4b47c3f87266d864%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36286 - "GET /node/69392f7a6e7b0b7f22e518dc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:36292 - "GET /node/693f032d24745582e3995ddb%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:45994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40426 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:01:21,479 - lava_callback - INFO - Received callback for node 69cc0bb08df9359ff489892a INFO: 172.19.0.1:60134 - "POST /node/69cc0bb08df9359ff489892a HTTP/1.0" 202 Accepted 2026-03-31 18:01:21,487 - lava_callback - INFO - Starting async job submit for node 69cc0bb08df9359ff489892a 2026-03-31 18:01:21,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:01:21,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc0bb08df9359ff489892a HTTP/1.1" 200 None 2026-03-31 18:01:21,533 - lava_callback - DEBUG - Processing node 69cc0bb08df9359ff489892a: baseline-arm64 INFO: 172.19.0.1:60142 - "GET /node/69bde8c4d54b5122738c646f%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-31 18:01:21,751 - lava_callback - WARNING - Failed to upload log 2026-03-31 18:01:21,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:01:22,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:01:22,144 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc0bb08df9359ff489892a/lava-logs.yaml 2026-03-31 18:01:22,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:01:22,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:01:22,303 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc0bb08df9359ff489892a/lava_callback.json.gz 2026-03-31 18:01:22,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:01:22,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc0bb08df9359ff489892a HTTP/1.1" 200 None 2026-03-31 18:01:22,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:01:22,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:01:22,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:01:22,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc0bb08df9359ff489892a HTTP/1.1" 200 None 2026-03-31 18:01:22,463 - lava_callback - INFO - Completed processing callback for node 69cc0bb08df9359ff489892a INFO: 172.19.0.1:60154 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:01:29,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:01:29,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33208 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:02:28,114 - lava_callback - INFO - Received callback for node 69cbfca38df9359ff48983a6 INFO: 172.19.0.1:33216 - "POST /node/69cbfca38df9359ff48983a6 HTTP/1.0" 202 Accepted 2026-03-31 18:02:28,120 - lava_callback - INFO - Starting async job submit for node 69cbfca38df9359ff48983a6 2026-03-31 18:02:28,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:02:28,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca38df9359ff48983a6 HTTP/1.1" 200 None 2026-03-31 18:02:28,153 - lava_callback - DEBUG - Processing node 69cbfca38df9359ff48983a6: baseline-arm64 2026-03-31 18:02:28,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:02:28,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:02:28,650 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca38df9359ff48983a6/log.txt.gz 2026-03-31 18:02:28,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:02:29,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:02:29,072 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca38df9359ff48983a6/lava-logs.yaml 2026-03-31 18:02:29,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:02:29,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:02:29,217 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cbfca38df9359ff48983a6/lava_callback.json.gz 2026-03-31 18:02:29,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:02:29,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfca38df9359ff48983a6 HTTP/1.1" 200 None 2026-03-31 18:02:29,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:02:29,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:02:29,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:02:29,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfca38df9359ff48983a6 HTTP/1.1" 200 None 2026-03-31 18:02:29,368 - lava_callback - INFO - Completed processing callback for node 69cbfca38df9359ff48983a6 2026-03-31 18:02:29,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:02:29,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:33456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36320 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:03:40,691 - lava_callback - INFO - Received callback for node 69cbfcb38df9359ff48983d3 INFO: 172.19.0.1:50352 - "POST /node/69cbfcb38df9359ff48983d3 HTTP/1.0" 202 Accepted 2026-03-31 18:03:40,697 - lava_callback - INFO - Starting async job submit for node 69cbfcb38df9359ff48983d3 2026-03-31 18:03:40,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:03:40,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb38df9359ff48983d3 HTTP/1.1" 200 None 2026-03-31 18:03:40,739 - lava_callback - DEBUG - Processing node 69cbfcb38df9359ff48983d3: baseline-nfs-arm64 2026-03-31 18:03:41,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:03:41,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:03:41,331 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb38df9359ff48983d3/log.txt.gz 2026-03-31 18:03:41,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:03:41,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:03:41,798 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb38df9359ff48983d3/lava-logs.yaml 2026-03-31 18:03:41,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:03:42,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:03:42,105 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cbfcb38df9359ff48983d3/lava_callback.json.gz 2026-03-31 18:03:42,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:03:42,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfcb38df9359ff48983d3 HTTP/1.1" 200 None 2026-03-31 18:03:42,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:03:42,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:03:42,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:03:42,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfcb38df9359ff48983d3 HTTP/1.1" 200 None 2026-03-31 18:03:42,194 - lava_callback - INFO - Completed processing callback for node 69cbfcb38df9359ff48983d3 INFO: 172.19.0.1:50356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59112 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:03:59,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:03:59,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:41488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51210 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:04:17,016 - lava_callback - INFO - Received callback for node 69cc0c5f8df9359ff489894d INFO: 172.19.0.1:51214 - "POST /node/69cc0c5f8df9359ff489894d HTTP/1.0" 202 Accepted 2026-03-31 18:04:17,023 - lava_callback - INFO - Starting async job submit for node 69cc0c5f8df9359ff489894d 2026-03-31 18:04:17,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:04:17,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc0c5f8df9359ff489894d HTTP/1.1" 200 None 2026-03-31 18:04:17,079 - lava_callback - DEBUG - Processing node 69cc0c5f8df9359ff489894d: baseline-arm64 2026-03-31 18:04:17,211 - lava_callback - WARNING - Failed to upload log 2026-03-31 18:04:17,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:04:17,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:04:17,468 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc0c5f8df9359ff489894d/lava-logs.yaml 2026-03-31 18:04:17,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:04:17,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:04:17,579 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc0c5f8df9359ff489894d/lava_callback.json.gz 2026-03-31 18:04:17,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:04:17,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc0c5f8df9359ff489894d HTTP/1.1" 200 None 2026-03-31 18:04:17,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:04:17,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:04:17,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:04:17,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc0c5f8df9359ff489894d HTTP/1.1" 200 None 2026-03-31 18:04:17,753 - lava_callback - INFO - Completed processing callback for node 69cc0c5f8df9359ff489894d INFO: 172.19.0.1:57462 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:04:29,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:04:29,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:50810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39154 - "GET /node/694ab01e83c67c43d1ede856%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51826 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50350 - "GET /node/6990b18ec45febfe44523008%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:50366 - "GET /node/694ac99d6c96ed51d316f4b3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:50378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42426 - "GET /node/694aac4a83c67c43d1ede3aa%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45410 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:12:35,598 - lava_callback - INFO - Received callback for node 69cc09e48df9359ff48988e7 INFO: 172.19.0.1:50780 - "POST /node/69cc09e48df9359ff48988e7 HTTP/1.0" 202 Accepted 2026-03-31 18:12:35,606 - lava_callback - INFO - Starting async job submit for node 69cc09e48df9359ff48988e7 2026-03-31 18:12:35,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:12:35,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc09e48df9359ff48988e7 HTTP/1.1" 200 None 2026-03-31 18:12:35,668 - lava_callback - DEBUG - Processing node 69cc09e48df9359ff48988e7: baseline-nfs-arm64 INFO: 172.19.0.1:50786 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:12:36,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:12:37,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:12:37,137 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc09e48df9359ff48988e7/log.txt.gz 2026-03-31 18:12:37,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:12:38,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:12:38,080 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc09e48df9359ff48988e7/lava-logs.yaml 2026-03-31 18:12:38,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:12:38,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:12:38,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc09e48df9359ff48988e7/lava_callback.json.gz DEBUG: Setup failed for 69cc09e48df9359ff48988e7. Transitting job node result: incomplete -> fail 2026-03-31 18:12:38,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:12:38,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc09e48df9359ff48988e7 HTTP/1.1" 200 None 2026-03-31 18:12:38,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:12:38,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:12:38,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:12:38,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc09e48df9359ff48988e7 HTTP/1.1" 200 None 2026-03-31 18:12:38,519 - lava_callback - INFO - Completed processing callback for node 69cc09e48df9359ff48988e7 INFO: 172.19.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33434 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:12:59,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:12:59,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:47066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47674 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38374 - "GET /node/69a168eba41e9451dcf419f0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:38384 - "GET /node/691057558b15debe9dd4c21d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:38386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55308 - "GET /node/694ab01383c67c43d1ede851%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35160 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:19:01,039 - lava_callback - INFO - Received callback for node 69cbfba78df9359ff4898207 INFO: 172.19.0.1:34790 - "POST /node/69cbfba78df9359ff4898207 HTTP/1.0" 202 Accepted 2026-03-31 18:19:01,044 - lava_callback - INFO - Starting async job submit for node 69cbfba78df9359ff4898207 2026-03-31 18:19:01,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:19:01,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfba78df9359ff4898207 HTTP/1.1" 200 None 2026-03-31 18:19:01,082 - lava_callback - DEBUG - Processing node 69cbfba78df9359ff4898207: baseline-arm-broonie 2026-03-31 18:19:01,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:19:01,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:19:01,468 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfba78df9359ff4898207/log.txt.gz 2026-03-31 18:19:01,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:19:01,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:19:01,942 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfba78df9359ff4898207/lava-logs.yaml 2026-03-31 18:19:01,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:19:02,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:19:02,078 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cbfba78df9359ff4898207/lava_callback.json.gz 2026-03-31 18:19:02,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:19:02,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfba78df9359ff4898207 HTTP/1.1" 200 None 2026-03-31 18:19:02,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:19:02,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 18:19:02,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:19:02,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfba78df9359ff4898207 HTTP/1.1" 200 None 2026-03-31 18:19:02,161 - lava_callback - INFO - Completed processing callback for node 69cbfba78df9359ff4898207 INFO: 172.19.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42218 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:19:29,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:19:29,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48014 - "GET /node/698a48ad22b4cc0f227b0d31%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:48030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48034 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:20:24,962 - lava_callback - INFO - Received callback for node 69cc0f158df9359ff48989ad INFO: 172.19.0.1:34284 - "POST /node/69cc0f158df9359ff48989ad HTTP/1.0" 202 Accepted 2026-03-31 18:20:24,971 - lava_callback - INFO - Starting async job submit for node 69cc0f158df9359ff48989ad 2026-03-31 18:20:25,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:20:25,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc0f158df9359ff48989ad HTTP/1.1" 200 None 2026-03-31 18:20:25,041 - lava_callback - DEBUG - Processing node 69cc0f158df9359ff48989ad: baseline-arm64-foundriesio 2026-03-31 18:20:26,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:20:26,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:20:26,442 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc0f158df9359ff48989ad/log.txt.gz INFO: 172.19.0.1:34286 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:20:27,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:20:27,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:20:27,403 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc0f158df9359ff48989ad/lava-logs.yaml 2026-03-31 18:20:27,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:20:27,622 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:20:27,624 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc0f158df9359ff48989ad/lava_callback.json.gz DEBUG: Setup failed for 69cc0f158df9359ff48989ad. Transitting job node result: incomplete -> fail 2026-03-31 18:20:27,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:20:27,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc0f158df9359ff48989ad HTTP/1.1" 200 None 2026-03-31 18:20:27,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:20:27,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:20:27,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:20:27,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc0f158df9359ff48989ad HTTP/1.1" 200 None 2026-03-31 18:20:27,908 - lava_callback - INFO - Completed processing callback for node 69cc0f158df9359ff48989ad 2026-03-31 18:20:29,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:20:29,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60592 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:21:21,486 - lava_callback - INFO - Received callback for node 69cbfbd88df9359ff48982a2 INFO: 172.19.0.1:35180 - "POST /node/69cbfbd88df9359ff48982a2 HTTP/1.0" 202 Accepted 2026-03-31 18:21:21,496 - lava_callback - INFO - Starting async job submit for node 69cbfbd88df9359ff48982a2 2026-03-31 18:21:21,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:21:21,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbd88df9359ff48982a2 HTTP/1.1" 200 None 2026-03-31 18:21:21,542 - lava_callback - DEBUG - Processing node 69cbfbd88df9359ff48982a2: ltp-cap_bounds 2026-03-31 18:21:21,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:21:22,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:21:22,027 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cbfbd88df9359ff48982a2/log.txt.gz 2026-03-31 18:21:22,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:21:22,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:21:22,441 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cbfbd88df9359ff48982a2/lava-logs.yaml 2026-03-31 18:21:22,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:21:22,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:21:22,629 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cbfbd88df9359ff48982a2/lava_callback.json.gz 2026-03-31 18:21:22,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:21:22,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbd88df9359ff48982a2 HTTP/1.1" 200 None 2026-03-31 18:21:22,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:21:22,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 18:21:22,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:21:22,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbd88df9359ff48982a2 HTTP/1.1" 200 None 2026-03-31 18:21:22,714 - lava_callback - INFO - Completed processing callback for node 69cbfbd88df9359ff48982a2 INFO: 172.19.0.1:35194 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:21:29,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:21:29,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:33004 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:21:42,043 - lava_callback - INFO - Received callback for node 69cbfbda8df9359ff48982a5 INFO: 172.19.0.1:38112 - "POST /node/69cbfbda8df9359ff48982a5 HTTP/1.0" 202 Accepted 2026-03-31 18:21:42,049 - lava_callback - INFO - Starting async job submit for node 69cbfbda8df9359ff48982a5 2026-03-31 18:21:42,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:21:42,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbda8df9359ff48982a5 HTTP/1.1" 200 None 2026-03-31 18:21:42,093 - lava_callback - DEBUG - Processing node 69cbfbda8df9359ff48982a5: ltp-containers 2026-03-31 18:21:42,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:21:42,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:21:42,649 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69cbfbda8df9359ff48982a5/log.txt.gz 2026-03-31 18:21:42,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:21:43,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:21:43,095 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69cbfbda8df9359ff48982a5/lava-logs.yaml 2026-03-31 18:21:43,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:21:43,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:21:43,256 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69cbfbda8df9359ff48982a5/lava_callback.json.gz 2026-03-31 18:21:43,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:21:43,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbda8df9359ff48982a5 HTTP/1.1" 200 None 2026-03-31 18:21:43,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:21:43,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 18:21:43,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:21:43,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbda8df9359ff48982a5 HTTP/1.1" 200 None 2026-03-31 18:21:43,340 - lava_callback - INFO - Completed processing callback for node 69cbfbda8df9359ff48982a5 INFO: 172.19.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40314 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:21:59,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:21:59,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:51664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34974 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:22:41,406 - lava_callback - INFO - Received callback for node 69cbfbdc8df9359ff48982a8 INFO: 172.19.0.1:52016 - "POST /node/69cbfbdc8df9359ff48982a8 HTTP/1.0" 202 Accepted 2026-03-31 18:22:41,412 - lava_callback - INFO - Starting async job submit for node 69cbfbdc8df9359ff48982a8 2026-03-31 18:22:41,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:22:41,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbdc8df9359ff48982a8 HTTP/1.1" 200 None 2026-03-31 18:22:41,452 - lava_callback - DEBUG - Processing node 69cbfbdc8df9359ff48982a8: ltp-controllers 2026-03-31 18:22:41,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:22:42,234 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:22:42,236 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69cbfbdc8df9359ff48982a8/log.txt.gz 2026-03-31 18:22:42,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:22:42,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:22:42,706 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69cbfbdc8df9359ff48982a8/lava-logs.yaml 2026-03-31 18:22:42,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:22:42,894 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:22:42,896 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69cbfbdc8df9359ff48982a8/lava_callback.json.gz 2026-03-31 18:22:42,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:22:42,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbdc8df9359ff48982a8 HTTP/1.1" 200 None 2026-03-31 18:22:42,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:22:42,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 18:22:42,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:22:43,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbdc8df9359ff48982a8 HTTP/1.1" 200 None 2026-03-31 18:22:43,019 - lava_callback - INFO - Completed processing callback for node 69cbfbdc8df9359ff48982a8 INFO: 172.19.0.1:52032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60988 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:22:59,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:22:59,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:33000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33328 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:23:35,679 - lava_callback - INFO - Received callback for node 69cbfe1c8df9359ff48985c1 INFO: 172.19.0.1:53774 - "POST /node/69cbfe1c8df9359ff48985c1 HTTP/1.0" 202 Accepted 2026-03-31 18:23:35,685 - lava_callback - INFO - Starting async job submit for node 69cbfe1c8df9359ff48985c1 2026-03-31 18:23:35,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:23:35,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfe1c8df9359ff48985c1 HTTP/1.1" 200 None 2026-03-31 18:23:35,731 - lava_callback - DEBUG - Processing node 69cbfe1c8df9359ff48985c1: baseline-x86 2026-03-31 18:23:36,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:23:36,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:23:36,281 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe1c8df9359ff48985c1/log.txt.gz 2026-03-31 18:23:36,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:53788 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:23:36,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:23:36,662 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe1c8df9359ff48985c1/lava-logs.yaml 2026-03-31 18:23:36,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:23:36,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:23:36,805 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cbfe1c8df9359ff48985c1/lava_callback.json.gz 2026-03-31 18:23:36,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:23:36,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfe1c8df9359ff48985c1 HTTP/1.1" 200 None 2026-03-31 18:23:36,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:23:36,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4418df9359ff4897d50 HTTP/1.1" 200 None 2026-03-31 18:23:36,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:23:36,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfe1c8df9359ff48985c1 HTTP/1.1" 200 None 2026-03-31 18:23:36,882 - lava_callback - INFO - Completed processing callback for node 69cbfe1c8df9359ff48985c1 INFO: 172.19.0.1:50954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37268 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:23:59,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:23:59,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:54264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45364 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:27:21,493 - lava_callback - INFO - Received callback for node 69cbfbde8df9359ff48982ab INFO: 172.19.0.1:47190 - "POST /node/69cbfbde8df9359ff48982ab HTTP/1.0" 202 Accepted 2026-03-31 18:27:21,500 - lava_callback - INFO - Starting async job submit for node 69cbfbde8df9359ff48982ab 2026-03-31 18:27:21,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:27:21,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbde8df9359ff48982ab HTTP/1.1" 200 None 2026-03-31 18:27:21,541 - lava_callback - DEBUG - Processing node 69cbfbde8df9359ff48982ab: ltp-cve 2026-03-31 18:27:21,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:27:22,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:27:22,152 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69cbfbde8df9359ff48982ab/log.txt.gz 2026-03-31 18:27:22,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:27:22,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:27:22,678 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69cbfbde8df9359ff48982ab/lava-logs.yaml 2026-03-31 18:27:22,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:27:22,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:27:22,861 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69cbfbde8df9359ff48982ab/lava_callback.json.gz 2026-03-31 18:27:22,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:27:22,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbde8df9359ff48982ab HTTP/1.1" 200 None 2026-03-31 18:27:22,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:27:22,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 18:27:22,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:27:22,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbde8df9359ff48982ab HTTP/1.1" 200 None 2026-03-31 18:27:22,957 - lava_callback - INFO - Completed processing callback for node 69cbfbde8df9359ff48982ab INFO: 172.19.0.1:47206 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:27:29,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:27:29,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:51338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54450 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:29:24,885 - lava_callback - INFO - Received callback for node 69cc10c38df9359ff4898a0d INFO: 172.19.0.1:41614 - "POST /node/69cc10c38df9359ff4898a0d HTTP/1.0" 202 Accepted 2026-03-31 18:29:24,893 - lava_callback - INFO - Starting async job submit for node 69cc10c38df9359ff4898a0d 2026-03-31 18:29:24,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:29:24,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc10c38df9359ff4898a0d HTTP/1.1" 200 None 2026-03-31 18:29:24,943 - lava_callback - DEBUG - Processing node 69cc10c38df9359ff4898a0d: baseline-arm64 2026-03-31 18:29:25,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:29:25,454 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:29:25,456 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc10c38df9359ff4898a0d/log.txt.gz 2026-03-31 18:29:25,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:29:25,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:29:25,917 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc10c38df9359ff4898a0d/lava-logs.yaml 2026-03-31 18:29:25,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:29:26,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:29:26,101 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc10c38df9359ff4898a0d/lava_callback.json.gz DEBUG: Setup failed for 69cc10c38df9359ff4898a0d. Transitting job node result: incomplete -> fail 2026-03-31 18:29:26,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:29:26,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc10c38df9359ff4898a0d HTTP/1.1" 200 None 2026-03-31 18:29:26,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:29:26,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:29:26,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:29:26,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc10c38df9359ff4898a0d HTTP/1.1" 200 None 2026-03-31 18:29:26,340 - lava_callback - INFO - Completed processing callback for node 69cc10c38df9359ff4898a0d INFO: 172.19.0.1:41620 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:29:29,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:29:29,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36294 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:34:12,655 - lava_callback - INFO - Received callback for node 69cc138d8df9359ff4898b00 INFO: 172.19.0.1:41928 - "POST /node/69cc138d8df9359ff4898b00 HTTP/1.0" 202 Accepted 2026-03-31 18:34:12,664 - lava_callback - INFO - Starting async job submit for node 69cc138d8df9359ff4898b00 2026-03-31 18:34:12,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:34:12,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc138d8df9359ff4898b00 HTTP/1.1" 200 None 2026-03-31 18:34:12,681 - lava_callback - DEBUG - Processing node 69cc138d8df9359ff4898b00: baseline-nfs-arm64 2026-03-31 18:34:12,682 - lava_callback - WARNING - Failed to upload log 2026-03-31 18:34:12,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:34:12,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:34:12,845 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc138d8df9359ff4898b00/lava-logs.yaml 2026-03-31 18:34:12,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:34:12,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:34:12,981 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc138d8df9359ff4898b00/lava_callback.json.gz 2026-03-31 18:34:12,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:34:12,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc138d8df9359ff4898b00 HTTP/1.1" 200 None 2026-03-31 18:34:13,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:34:13,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:34:13,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:34:13,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc138d8df9359ff4898b00 HTTP/1.1" 200 None 2026-03-31 18:34:13,057 - lava_callback - INFO - Completed processing callback for node 69cc138d8df9359ff4898b00 INFO: 172.19.0.1:41932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34418 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:34:29,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:34:29,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:41398 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:34:38,591 - lava_callback - INFO - Received callback for node 69c970b7f2866272ea51e97e INFO: 172.19.0.1:41412 - "POST /node/69c970b7f2866272ea51e97e HTTP/1.0" 202 Accepted 2026-03-31 18:34:38,597 - lava_callback - INFO - Starting async job submit for node 69c970b7f2866272ea51e97e 2026-03-31 18:34:38,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:34:38,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c970b7f2866272ea51e97e HTTP/1.1" 200 None 2026-03-31 18:34:38,612 - lava_callback - DEBUG - Processing node 69c970b7f2866272ea51e97e: baseline-arm64-baylibre 2026-03-31 18:34:38,612 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c970b7f2866272ea51e97e 2026-03-31 18:34:38,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:34:38,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c970b7f2866272ea51e97e HTTP/1.1" 200 None 2026-03-31 18:34:38,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:34:38,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:34:38,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:34:38,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c970b7f2866272ea51e97e HTTP/1.1" 200 None 2026-03-31 18:34:38,674 - lava_callback - ERROR - Error processing callback for node 69c970b7f2866272ea51e97e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58914 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:36:38,630 - lava_callback - INFO - Received callback for node 69c97134f2866272ea51e987 INFO: 172.19.0.1:58922 - "POST /node/69c97134f2866272ea51e987 HTTP/1.0" 202 Accepted 2026-03-31 18:36:38,636 - lava_callback - INFO - Starting async job submit for node 69c97134f2866272ea51e987 2026-03-31 18:36:38,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:36:38,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97134f2866272ea51e987 HTTP/1.1" 200 None 2026-03-31 18:36:38,652 - lava_callback - DEBUG - Processing node 69c97134f2866272ea51e987: baseline-arm64-baylibre 2026-03-31 18:36:38,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c97134f2866272ea51e987 2026-03-31 18:36:38,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:36:38,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97134f2866272ea51e987 HTTP/1.1" 200 None 2026-03-31 18:36:38,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:36:38,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:36:38,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:36:38,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c97134f2866272ea51e987 HTTP/1.1" 200 None 2026-03-31 18:36:38,708 - lava_callback - ERROR - Error processing callback for node 69c97134f2866272ea51e987: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46132 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:38:38,753 - lava_callback - INFO - Received callback for node 69c971aaf2866272ea51e9ad INFO: 172.19.0.1:46134 - "POST /node/69c971aaf2866272ea51e9ad HTTP/1.0" 202 Accepted 2026-03-31 18:38:38,759 - lava_callback - INFO - Starting async job submit for node 69c971aaf2866272ea51e9ad 2026-03-31 18:38:38,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:38:38,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c971aaf2866272ea51e9ad HTTP/1.1" 200 None 2026-03-31 18:38:38,799 - lava_callback - DEBUG - Processing node 69c971aaf2866272ea51e9ad: baseline-arm64-baylibre 2026-03-31 18:38:38,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c971aaf2866272ea51e9ad 2026-03-31 18:38:38,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:38:38,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c971aaf2866272ea51e9ad HTTP/1.1" 200 None 2026-03-31 18:38:38,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:38:38,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:38:38,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:38:38,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c971aaf2866272ea51e9ad HTTP/1.1" 200 None 2026-03-31 18:38:38,855 - lava_callback - ERROR - Error processing callback for node 69c971aaf2866272ea51e9ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58068 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:38:55,711 - lava_callback - INFO - Received callback for node 69cc14888df9359ff4898b99 INFO: 172.19.0.1:46644 - "POST /node/69cc14888df9359ff4898b99 HTTP/1.0" 202 Accepted 2026-03-31 18:38:55,718 - lava_callback - INFO - Starting async job submit for node 69cc14888df9359ff4898b99 2026-03-31 18:38:55,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:38:55,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc14888df9359ff4898b99 HTTP/1.1" 200 None 2026-03-31 18:38:55,763 - lava_callback - DEBUG - Processing node 69cc14888df9359ff4898b99: baseline-arm64 INFO: 172.19.0.1:46650 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:38:55,896 - lava_callback - WARNING - Failed to upload log 2026-03-31 18:38:55,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:38:56,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:38:56,166 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc14888df9359ff4898b99/lava-logs.yaml 2026-03-31 18:38:56,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:38:56,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:38:56,305 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc14888df9359ff4898b99/lava_callback.json.gz 2026-03-31 18:38:56,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:38:56,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc14888df9359ff4898b99 HTTP/1.1" 200 None 2026-03-31 18:38:56,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:38:56,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:38:56,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:38:56,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc14888df9359ff4898b99 HTTP/1.1" 200 None 2026-03-31 18:38:56,460 - lava_callback - INFO - Completed processing callback for node 69cc14888df9359ff4898b99 INFO: 172.19.0.1:46652 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:38:59,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:38:59,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:55618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51258 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:39:58,913 - lava_callback - INFO - Received callback for node 69c971f9f2866272ea51e9be INFO: 172.19.0.1:51260 - "POST /node/69c971f9f2866272ea51e9be HTTP/1.0" 202 Accepted 2026-03-31 18:39:58,918 - lava_callback - INFO - Starting async job submit for node 69c971f9f2866272ea51e9be 2026-03-31 18:39:58,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:39:58,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c971f9f2866272ea51e9be HTTP/1.1" 200 None 2026-03-31 18:39:58,933 - lava_callback - DEBUG - Processing node 69c971f9f2866272ea51e9be: baseline-arm64-baylibre 2026-03-31 18:39:58,934 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c971f9f2866272ea51e9be 2026-03-31 18:39:58,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:39:58,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c971f9f2866272ea51e9be HTTP/1.1" 200 None 2026-03-31 18:39:58,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:39:58,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:39:58,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:39:58,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c971f9f2866272ea51e9be HTTP/1.1" 200 None 2026-03-31 18:39:58,987 - lava_callback - ERROR - Error processing callback for node 69c971f9f2866272ea51e9be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-31 18:40:00,546 - lava_callback - INFO - Received callback for node 69cc14308df9359ff4898b51 INFO: 172.19.0.1:59810 - "POST /node/69cc14308df9359ff4898b51 HTTP/1.0" 202 Accepted 2026-03-31 18:40:00,552 - lava_callback - INFO - Starting async job submit for node 69cc14308df9359ff4898b51 2026-03-31 18:40:00,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:40:00,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc14308df9359ff4898b51 HTTP/1.1" 200 None 2026-03-31 18:40:00,607 - lava_callback - DEBUG - Processing node 69cc14308df9359ff4898b51: baseline-arm64 2026-03-31 18:40:00,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:40:01,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:40:01,206 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc14308df9359ff4898b51/log.txt.gz 2026-03-31 18:40:01,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:40:01,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:40:01,738 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc14308df9359ff4898b51/lava-logs.yaml 2026-03-31 18:40:01,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:40:01,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:40:01,912 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc14308df9359ff4898b51/lava_callback.json.gz 2026-03-31 18:40:01,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:40:01,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc14308df9359ff4898b51 HTTP/1.1" 200 None 2026-03-31 18:40:01,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:40:01,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:40:01,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:40:01,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc14308df9359ff4898b51 HTTP/1.1" 200 None 2026-03-31 18:40:01,999 - lava_callback - INFO - Completed processing callback for node 69cc14308df9359ff4898b51 INFO: 172.19.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41444 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:40:29,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:40:29,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49688 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:40:56,696 - lava_callback - INFO - Received callback for node 69cc13db8df9359ff4898b36 INFO: 172.19.0.1:49704 - "POST /node/69cc13db8df9359ff4898b36 HTTP/1.0" 202 Accepted 2026-03-31 18:40:56,703 - lava_callback - INFO - Starting async job submit for node 69cc13db8df9359ff4898b36 2026-03-31 18:40:56,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:40:56,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc13db8df9359ff4898b36 HTTP/1.1" 200 None 2026-03-31 18:40:56,747 - lava_callback - DEBUG - Processing node 69cc13db8df9359ff4898b36: baseline-arm64 2026-03-31 18:40:56,944 - lava_callback - WARNING - Failed to upload log 2026-03-31 18:40:57,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:40:57,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:40:57,231 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc13db8df9359ff4898b36/lava-logs.yaml 2026-03-31 18:40:57,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:40:57,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:40:57,355 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc13db8df9359ff4898b36/lava_callback.json.gz 2026-03-31 18:40:57,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:40:57,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc13db8df9359ff4898b36 HTTP/1.1" 200 None 2026-03-31 18:40:57,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:40:57,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:40:57,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:40:57,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc13db8df9359ff4898b36 HTTP/1.1" 200 None 2026-03-31 18:40:57,503 - lava_callback - INFO - Completed processing callback for node 69cc13db8df9359ff4898b36 2026-03-31 18:40:59,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:40:59,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54362 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42232 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:42:39,057 - lava_callback - INFO - Received callback for node 69c9729bf2866272ea51e9e6 INFO: 172.19.0.1:42240 - "POST /node/69c9729bf2866272ea51e9e6 HTTP/1.0" 202 Accepted 2026-03-31 18:42:39,062 - lava_callback - INFO - Starting async job submit for node 69c9729bf2866272ea51e9e6 2026-03-31 18:42:39,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:42:39,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9729bf2866272ea51e9e6 HTTP/1.1" 200 None 2026-03-31 18:42:39,081 - lava_callback - DEBUG - Processing node 69c9729bf2866272ea51e9e6: baseline-arm64-baylibre 2026-03-31 18:42:39,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9729bf2866272ea51e9e6 2026-03-31 18:42:39,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:42:39,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9729bf2866272ea51e9e6 HTTP/1.1" 200 None 2026-03-31 18:42:39,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:42:39,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:42:39,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:42:39,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9729bf2866272ea51e9e6 HTTP/1.1" 200 None 2026-03-31 18:42:39,143 - lava_callback - ERROR - Error processing callback for node 69c9729bf2866272ea51e9e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43610 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:43:58,975 - lava_callback - INFO - Received callback for node 69c972edf2866272ea51e9ef INFO: 172.19.0.1:43620 - "POST /node/69c972edf2866272ea51e9ef HTTP/1.0" 202 Accepted 2026-03-31 18:43:58,980 - lava_callback - INFO - Starting async job submit for node 69c972edf2866272ea51e9ef 2026-03-31 18:43:58,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:43:59,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c972edf2866272ea51e9ef HTTP/1.1" 200 None 2026-03-31 18:43:59,010 - lava_callback - DEBUG - Processing node 69c972edf2866272ea51e9ef: baseline-arm64-baylibre 2026-03-31 18:43:59,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c972edf2866272ea51e9ef 2026-03-31 18:43:59,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:43:59,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c972edf2866272ea51e9ef HTTP/1.1" 200 None 2026-03-31 18:43:59,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:43:59,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:43:59,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:43:59,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c972edf2866272ea51e9ef HTTP/1.1" 200 None 2026-03-31 18:43:59,067 - lava_callback - ERROR - Error processing callback for node 69c972edf2866272ea51e9ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51828 - "GET /node/6993284931166481c8de8f66%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:51836 - "GET /node/694acb696c96ed51d316f72a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:46910 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:45:32,196 - lava_callback - INFO - Received callback for node 69c97344f2866272ea51ea02 INFO: 172.19.0.1:57692 - "POST /node/69c97344f2866272ea51ea02 HTTP/1.0" 202 Accepted 2026-03-31 18:45:32,202 - lava_callback - INFO - Starting async job submit for node 69c97344f2866272ea51ea02 2026-03-31 18:45:32,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:45:32,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97344f2866272ea51ea02 HTTP/1.1" 200 None 2026-03-31 18:45:32,228 - lava_callback - DEBUG - Processing node 69c97344f2866272ea51ea02: baseline-arm64-clabbe 2026-03-31 18:45:32,228 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c97344f2866272ea51ea02 2026-03-31 18:45:32,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:45:32,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97344f2866272ea51ea02 HTTP/1.1" 200 None 2026-03-31 18:45:32,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:45:32,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:45:32,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:45:32,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c97344f2866272ea51ea02 HTTP/1.1" 200 None 2026-03-31 18:45:32,293 - lava_callback - ERROR - Error processing callback for node 69c97344f2866272ea51ea02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59102 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:47:12,259 - lava_callback - INFO - Received callback for node 69c973a4f2866272ea51ea12 INFO: 172.19.0.1:53370 - "POST /node/69c973a4f2866272ea51ea12 HTTP/1.0" 202 Accepted 2026-03-31 18:47:12,267 - lava_callback - INFO - Starting async job submit for node 69c973a4f2866272ea51ea12 2026-03-31 18:47:12,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:47:12,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c973a4f2866272ea51ea12 HTTP/1.1" 200 None 2026-03-31 18:47:12,291 - lava_callback - DEBUG - Processing node 69c973a4f2866272ea51ea12: baseline-arm64-clabbe 2026-03-31 18:47:12,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c973a4f2866272ea51ea12 2026-03-31 18:47:12,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:47:12,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c973a4f2866272ea51ea12 HTTP/1.1" 200 None 2026-03-31 18:47:12,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:47:12,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:47:12,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:47:12,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c973a4f2866272ea51ea12 HTTP/1.1" 200 None 2026-03-31 18:47:12,345 - lava_callback - ERROR - Error processing callback for node 69c973a4f2866272ea51ea12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34682 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:47:44,585 - lava_callback - INFO - Received callback for node 69cc15908df9359ff4898bfe INFO: 172.19.0.1:36068 - "POST /node/69cc15908df9359ff4898bfe HTTP/1.0" 202 Accepted 2026-03-31 18:47:44,593 - lava_callback - INFO - Starting async job submit for node 69cc15908df9359ff4898bfe 2026-03-31 18:47:44,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:47:44,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc15908df9359ff4898bfe HTTP/1.1" 200 None 2026-03-31 18:47:44,657 - lava_callback - DEBUG - Processing node 69cc15908df9359ff4898bfe: baseline-arm64-foundriesio 2026-03-31 18:47:45,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:47:46,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:47:46,057 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc15908df9359ff4898bfe/log.txt.gz INFO: 172.19.0.1:36072 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:47:46,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:47:46,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:47:46,994 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc15908df9359ff4898bfe/lava-logs.yaml 2026-03-31 18:47:47,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:47:47,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:47:47,204 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cc15908df9359ff4898bfe/lava_callback.json.gz DEBUG: 69cc15908df9359ff4898bfe Transitting job node result: pass -> incomplete 2026-03-31 18:47:47,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:47:47,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc15908df9359ff4898bfe HTTP/1.1" 200 None 2026-03-31 18:47:47,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:47:47,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:47:47,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:47:47,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc15908df9359ff4898bfe HTTP/1.1" 200 None 2026-03-31 18:47:47,334 - lava_callback - INFO - Completed processing callback for node 69cc15908df9359ff4898bfe INFO: 172.19.0.1:48198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48202 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:47:59,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:47:59,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:50106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59592 - "GET /node/698460f63a8748f2820864d3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:59604 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58982 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:49:12,300 - lava_callback - INFO - Received callback for node 69c9741af2866272ea51ea28 INFO: 172.19.0.1:52660 - "POST /node/69c9741af2866272ea51ea28 HTTP/1.0" 202 Accepted 2026-03-31 18:49:12,306 - lava_callback - INFO - Starting async job submit for node 69c9741af2866272ea51ea28 2026-03-31 18:49:12,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:49:12,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9741af2866272ea51ea28 HTTP/1.1" 200 None 2026-03-31 18:49:12,323 - lava_callback - DEBUG - Processing node 69c9741af2866272ea51ea28: baseline-arm64-clabbe 2026-03-31 18:49:12,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9741af2866272ea51ea28 2026-03-31 18:49:12,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:49:12,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9741af2866272ea51ea28 HTTP/1.1" 200 None 2026-03-31 18:49:12,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:49:12,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:49:12,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:49:12,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9741af2866272ea51ea28 HTTP/1.1" 200 None 2026-03-31 18:49:12,390 - lava_callback - ERROR - Error processing callback for node 69c9741af2866272ea51ea28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49404 - "GET /node/694acd896c96ed51d316fe6b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:46754 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:50:39,427 - lava_callback - INFO - Received callback for node 69c9747cf2866272ea51ea44 INFO: 172.19.0.1:46764 - "POST /node/69c9747cf2866272ea51ea44 HTTP/1.0" 202 Accepted 2026-03-31 18:50:39,433 - lava_callback - INFO - Starting async job submit for node 69c9747cf2866272ea51ea44 2026-03-31 18:50:39,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:50:39,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9747cf2866272ea51ea44 HTTP/1.1" 200 None 2026-03-31 18:50:39,449 - lava_callback - DEBUG - Processing node 69c9747cf2866272ea51ea44: baseline-arm-baylibre 2026-03-31 18:50:39,449 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9747cf2866272ea51ea44 2026-03-31 18:50:39,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:50:39,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9747cf2866272ea51ea44 HTTP/1.1" 200 None 2026-03-31 18:50:39,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:50:39,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 18:50:39,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:50:39,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9747cf2866272ea51ea44 HTTP/1.1" 200 None 2026-03-31 18:50:39,518 - lava_callback - ERROR - Error processing callback for node 69c9747cf2866272ea51ea44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53998 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:50:55,071 - lava_callback - INFO - Received callback for node 69cc135e8df9359ff4898ae2 INFO: 172.19.0.1:48522 - "POST /node/69cc135e8df9359ff4898ae2 HTTP/1.0" 202 Accepted 2026-03-31 18:50:55,080 - lava_callback - INFO - Starting async job submit for node 69cc135e8df9359ff4898ae2 2026-03-31 18:50:55,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:50:55,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc135e8df9359ff4898ae2 HTTP/1.1" 200 None 2026-03-31 18:50:55,179 - lava_callback - DEBUG - Processing node 69cc135e8df9359ff4898ae2: baseline-arm 2026-03-31 18:50:55,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:50:56,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:50:56,019 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc135e8df9359ff4898ae2/log.txt.gz INFO: 172.19.0.1:48534 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:50:56,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.19.0.1:48544 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:50:56,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:50:56,580 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc135e8df9359ff4898ae2/lava-logs.yaml 2026-03-31 18:50:56,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:50:56,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:50:56,762 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc135e8df9359ff4898ae2/lava_callback.json.gz DEBUG: Setup failed for 69cc135e8df9359ff4898ae2. Transitting job node result: incomplete -> fail 2026-03-31 18:50:56,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:50:56,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc135e8df9359ff4898ae2 HTTP/1.1" 200 None 2026-03-31 18:50:56,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:50:56,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 18:50:56,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:50:56,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc135e8df9359ff4898ae2 HTTP/1.1" 200 None 2026-03-31 18:50:57,048 - lava_callback - INFO - Completed processing callback for node 69cc135e8df9359ff4898ae2 2026-03-31 18:50:59,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:50:59,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:50314 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:51:12,358 - lava_callback - INFO - Received callback for node 69cc15e58df9359ff4898c11 INFO: 172.19.0.1:47828 - "POST /node/69cc15e58df9359ff4898c11 HTTP/1.0" 202 Accepted 2026-03-31 18:51:12,366 - lava_callback - INFO - Starting async job submit for node 69cc15e58df9359ff4898c11 2026-03-31 18:51:12,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:51:12,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc15e58df9359ff4898c11 HTTP/1.1" 200 None 2026-03-31 18:51:12,438 - lava_callback - DEBUG - Processing node 69cc15e58df9359ff4898c11: baseline-arm64 2026-03-31 18:51:12,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:51:13,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:51:13,009 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc15e58df9359ff4898c11/log.txt.gz 2026-03-31 18:51:13,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:51:13,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:51:13,433 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc15e58df9359ff4898c11/lava-logs.yaml 2026-03-31 18:51:13,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:51:13,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:51:13,612 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cc15e58df9359ff4898c11/lava_callback.json.gz DEBUG: Setup failed for 69cc15e58df9359ff4898c11. Transitting job node result: incomplete -> fail 2026-03-31 18:51:13,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:51:13,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc15e58df9359ff4898c11 HTTP/1.1" 200 None 2026-03-31 18:51:13,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:51:13,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:51:13,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:51:13,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc15e58df9359ff4898c11 HTTP/1.1" 200 None 2026-03-31 18:51:13,823 - lava_callback - INFO - Completed processing callback for node 69cc15e58df9359ff4898c11 INFO: 172.19.0.1:47834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57592 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:51:29,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:51:29,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:57544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48772 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:51:52,400 - lava_callback - INFO - Received callback for node 69c974b4f2866272ea51ea56 INFO: 172.19.0.1:37420 - "POST /node/69c974b4f2866272ea51ea56 HTTP/1.0" 202 Accepted 2026-03-31 18:51:52,406 - lava_callback - INFO - Starting async job submit for node 69c974b4f2866272ea51ea56 2026-03-31 18:51:52,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:51:52,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c974b4f2866272ea51ea56 HTTP/1.1" 200 None 2026-03-31 18:51:52,422 - lava_callback - DEBUG - Processing node 69c974b4f2866272ea51ea56: baseline-arm64-clabbe 2026-03-31 18:51:52,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c974b4f2866272ea51ea56 2026-03-31 18:51:52,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:51:52,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c974b4f2866272ea51ea56 HTTP/1.1" 200 None 2026-03-31 18:51:52,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:51:52,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:51:52,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:51:52,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c974b4f2866272ea51ea56 HTTP/1.1" 200 None 2026-03-31 18:51:52,479 - lava_callback - ERROR - Error processing callback for node 69c974b4f2866272ea51ea56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52972 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:52:39,528 - lava_callback - INFO - Received callback for node 69c974f5f2866272ea51ea60 INFO: 172.19.0.1:52982 - "POST /node/69c974f5f2866272ea51ea60 HTTP/1.0" 202 Accepted 2026-03-31 18:52:39,533 - lava_callback - INFO - Starting async job submit for node 69c974f5f2866272ea51ea60 2026-03-31 18:52:39,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:52:39,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c974f5f2866272ea51ea60 HTTP/1.1" 200 None 2026-03-31 18:52:39,550 - lava_callback - DEBUG - Processing node 69c974f5f2866272ea51ea60: baseline-arm-baylibre 2026-03-31 18:52:39,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c974f5f2866272ea51ea60 2026-03-31 18:52:39,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:52:39,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c974f5f2866272ea51ea60 HTTP/1.1" 200 None 2026-03-31 18:52:39,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:52:39,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 18:52:39,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:52:39,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c974f5f2866272ea51ea60 HTTP/1.1" 200 None 2026-03-31 18:52:39,608 - lava_callback - ERROR - Error processing callback for node 69c974f5f2866272ea51ea60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36030 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:53:52,480 - lava_callback - INFO - Received callback for node 69c97536f2866272ea51ea71 INFO: 172.19.0.1:40886 - "POST /node/69c97536f2866272ea51ea71 HTTP/1.0" 202 Accepted 2026-03-31 18:53:52,489 - lava_callback - INFO - Starting async job submit for node 69c97536f2866272ea51ea71 2026-03-31 18:53:52,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:53:52,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97536f2866272ea51ea71 HTTP/1.1" 200 None 2026-03-31 18:53:52,507 - lava_callback - DEBUG - Processing node 69c97536f2866272ea51ea71: baseline-arm64-clabbe 2026-03-31 18:53:52,507 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c97536f2866272ea51ea71 2026-03-31 18:53:52,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:53:52,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97536f2866272ea51ea71 HTTP/1.1" 200 None 2026-03-31 18:53:52,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:53:52,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:53:52,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:53:52,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c97536f2866272ea51ea71 HTTP/1.1" 200 None 2026-03-31 18:53:52,571 - lava_callback - ERROR - Error processing callback for node 69c97536f2866272ea51ea71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49370 - "GET /node/69878589e31b58f834f1b9b8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:49380 - "GET /node/693f1b9324745582e399b633%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:49390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46298 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:56:32,624 - lava_callback - INFO - Received callback for node 69c975cef2866272ea51ea97 INFO: 172.19.0.1:56758 - "POST /node/69c975cef2866272ea51ea97 HTTP/1.0" 202 Accepted 2026-03-31 18:56:32,629 - lava_callback - INFO - Starting async job submit for node 69c975cef2866272ea51ea97 2026-03-31 18:56:32,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:56:32,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c975cef2866272ea51ea97 HTTP/1.1" 200 None 2026-03-31 18:56:32,645 - lava_callback - DEBUG - Processing node 69c975cef2866272ea51ea97: baseline-arm64-clabbe 2026-03-31 18:56:32,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c975cef2866272ea51ea97 2026-03-31 18:56:32,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:56:32,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c975cef2866272ea51ea97 HTTP/1.1" 200 None 2026-03-31 18:56:32,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:56:32,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-31 18:56:32,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:56:32,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c975cef2866272ea51ea97 HTTP/1.1" 200 None 2026-03-31 18:56:32,707 - lava_callback - ERROR - Error processing callback for node 69c975cef2866272ea51ea97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46286 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:57:32,665 - lava_callback - INFO - Received callback for node 69c9760ff2866272ea51eaa3 INFO: 172.19.0.1:57412 - "POST /node/69c9760ff2866272ea51eaa3 HTTP/1.0" 202 Accepted 2026-03-31 18:57:32,671 - lava_callback - INFO - Starting async job submit for node 69c9760ff2866272ea51eaa3 2026-03-31 18:57:32,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:57:32,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9760ff2866272ea51eaa3 HTTP/1.1" 200 None 2026-03-31 18:57:32,696 - lava_callback - DEBUG - Processing node 69c9760ff2866272ea51eaa3: baseline-arm-clabbe 2026-03-31 18:57:32,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9760ff2866272ea51eaa3 2026-03-31 18:57:32,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:57:32,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9760ff2866272ea51eaa3 HTTP/1.1" 200 None 2026-03-31 18:57:32,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:57:32,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 18:57:32,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:57:32,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9760ff2866272ea51eaa3 HTTP/1.1" 200 None 2026-03-31 18:57:32,775 - lava_callback - ERROR - Error processing callback for node 69c9760ff2866272ea51eaa3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58820 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:58:12,629 - lava_callback - INFO - Received callback for node 69c97637f2866272ea51eab5 INFO: 172.19.0.1:54034 - "POST /node/69c97637f2866272ea51eab5 HTTP/1.0" 202 Accepted 2026-03-31 18:58:12,634 - lava_callback - INFO - Starting async job submit for node 69c97637f2866272ea51eab5 2026-03-31 18:58:12,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:58:12,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97637f2866272ea51eab5 HTTP/1.1" 200 None 2026-03-31 18:58:12,652 - lava_callback - DEBUG - Processing node 69c97637f2866272ea51eab5: baseline-arm-clabbe 2026-03-31 18:58:12,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c97637f2866272ea51eab5 2026-03-31 18:58:12,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:58:12,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97637f2866272ea51eab5 HTTP/1.1" 200 None 2026-03-31 18:58:12,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:58:12,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 18:58:12,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:58:12,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c97637f2866272ea51eab5 HTTP/1.1" 200 None 2026-03-31 18:58:12,719 - lava_callback - ERROR - Error processing callback for node 69c97637f2866272ea51eab5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33356 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:58:52,672 - lava_callback - INFO - Received callback for node 69c9765df2866272ea51eabf INFO: 172.19.0.1:43520 - "POST /node/69c9765df2866272ea51eabf HTTP/1.0" 202 Accepted 2026-03-31 18:58:52,677 - lava_callback - INFO - Starting async job submit for node 69c9765df2866272ea51eabf 2026-03-31 18:58:52,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:58:52,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9765df2866272ea51eabf HTTP/1.1" 200 None 2026-03-31 18:58:52,694 - lava_callback - DEBUG - Processing node 69c9765df2866272ea51eabf: baseline-arm-clabbe 2026-03-31 18:58:52,694 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9765df2866272ea51eabf 2026-03-31 18:58:52,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:58:52,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9765df2866272ea51eabf HTTP/1.1" 200 None 2026-03-31 18:58:52,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:58:52,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 18:58:52,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:58:52,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9765df2866272ea51eabf HTTP/1.1" 200 None 2026-03-31 18:58:52,745 - lava_callback - ERROR - Error processing callback for node 69c9765df2866272ea51eabf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44500 - "GET /node/694ac6746c96ed51d316f2e1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:46146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50430 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:59:32,656 - lava_callback - INFO - Received callback for node 69c97685f2866272ea51eaca INFO: 172.19.0.1:38898 - "POST /node/69c97685f2866272ea51eaca HTTP/1.0" 202 Accepted 2026-03-31 18:59:32,661 - lava_callback - INFO - Starting async job submit for node 69c97685f2866272ea51eaca 2026-03-31 18:59:32,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:59:32,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97685f2866272ea51eaca HTTP/1.1" 200 None 2026-03-31 18:59:32,679 - lava_callback - DEBUG - Processing node 69c97685f2866272ea51eaca: baseline-arm-clabbe 2026-03-31 18:59:32,679 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c97685f2866272ea51eaca 2026-03-31 18:59:32,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:59:32,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c97685f2866272ea51eaca HTTP/1.1" 200 None 2026-03-31 18:59:32,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:59:32,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 18:59:32,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:59:32,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c97685f2866272ea51eaca HTTP/1.1" 200 None 2026-03-31 18:59:32,764 - lava_callback - ERROR - Error processing callback for node 69c97685f2866272ea51eaca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38906 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:59:41,394 - lava_callback - INFO - Received callback for node 69cc14e88df9359ff4898bd0 INFO: 172.19.0.1:53762 - "POST /node/69cc14e88df9359ff4898bd0 HTTP/1.0" 202 Accepted 2026-03-31 18:59:41,403 - lava_callback - INFO - Starting async job submit for node 69cc14e88df9359ff4898bd0 2026-03-31 18:59:41,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:59:41,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc14e88df9359ff4898bd0 HTTP/1.1" 200 None 2026-03-31 18:59:41,462 - lava_callback - DEBUG - Processing node 69cc14e88df9359ff4898bd0: baseline-nfs-arm64 2026-03-31 18:59:42,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:59:43,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:59:43,006 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc14e88df9359ff4898bd0/log.txt.gz 2026-03-31 18:59:43,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:59:43,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:59:43,902 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc14e88df9359ff4898bd0/lava-logs.yaml 2026-03-31 18:59:43,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 18:59:44,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 18:59:44,127 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc14e88df9359ff4898bd0/lava_callback.json.gz DEBUG: Setup failed for 69cc14e88df9359ff4898bd0. Transitting job node result: incomplete -> fail 2026-03-31 18:59:44,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:59:44,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc14e88df9359ff4898bd0 HTTP/1.1" 200 None 2026-03-31 18:59:44,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:59:44,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 18:59:44,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:59:44,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc14e88df9359ff4898bd0 HTTP/1.1" 200 None 2026-03-31 18:59:44,367 - lava_callback - INFO - Completed processing callback for node 69cc14e88df9359ff4898bd0 INFO: 172.19.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32906 - "GET / HTTP/1.0" 200 OK 2026-03-31 18:59:59,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 18:59:59,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:43102 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:00:12,916 - lava_callback - INFO - Received callback for node 69c976abf2866272ea51ead6 INFO: 172.19.0.1:47946 - "POST /node/69c976abf2866272ea51ead6 HTTP/1.0" 202 Accepted 2026-03-31 19:00:12,921 - lava_callback - INFO - Starting async job submit for node 69c976abf2866272ea51ead6 2026-03-31 19:00:12,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:00:12,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c976abf2866272ea51ead6 HTTP/1.1" 200 None 2026-03-31 19:00:12,937 - lava_callback - DEBUG - Processing node 69c976abf2866272ea51ead6: baseline-arm-clabbe 2026-03-31 19:00:12,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c976abf2866272ea51ead6 2026-03-31 19:00:12,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:00:12,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c976abf2866272ea51ead6 HTTP/1.1" 200 None 2026-03-31 19:00:12,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:00:12,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-31 19:00:12,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:00:12,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c976abf2866272ea51ead6 HTTP/1.1" 200 None 2026-03-31 19:00:12,998 - lava_callback - ERROR - Error processing callback for node 69c976abf2866272ea51ead6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58764 - "GET /node/6990b662c45febfe445235fd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57154 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.19.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57808 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:05:30,320 - lava_callback - INFO - Received callback for node 69cc17918df9359ff4898c41 INFO: 172.19.0.1:55272 - "POST /node/69cc17918df9359ff4898c41 HTTP/1.0" 202 Accepted 2026-03-31 19:05:30,331 - lava_callback - INFO - Starting async job submit for node 69cc17918df9359ff4898c41 2026-03-31 19:05:30,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:05:30,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc17918df9359ff4898c41 HTTP/1.1" 200 None 2026-03-31 19:05:30,397 - lava_callback - DEBUG - Processing node 69cc17918df9359ff4898c41: baseline-arm 2026-03-31 19:05:30,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:05:30,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:05:30,986 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cc17918df9359ff4898c41/log.txt.gz 2026-03-31 19:05:31,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:05:31,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:05:31,457 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cc17918df9359ff4898c41/lava-logs.yaml 2026-03-31 19:05:31,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:05:31,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:05:31,612 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cc17918df9359ff4898c41/lava_callback.json.gz DEBUG: Setup failed for 69cc17918df9359ff4898c41. Transitting job node result: incomplete -> fail 2026-03-31 19:05:31,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:05:31,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc17918df9359ff4898c41 HTTP/1.1" 200 None 2026-03-31 19:05:31,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:05:31,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:05:31,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:05:31,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc17918df9359ff4898c41 HTTP/1.1" 200 None 2026-03-31 19:05:31,915 - lava_callback - INFO - Completed processing callback for node 69cc17918df9359ff4898c41 INFO: 172.19.0.1:55282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47754 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:05:59,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:05:59,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43680 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52518 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:07:22,876 - lava_callback - INFO - Received callback for node 69cbfbe28df9359ff48982b1 INFO: 172.19.0.1:37518 - "POST /node/69cbfbe28df9359ff48982b1 HTTP/1.0" 202 Accepted 2026-03-31 19:07:22,882 - lava_callback - INFO - Starting async job submit for node 69cbfbe28df9359ff48982b1 2026-03-31 19:07:22,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:07:22,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbe28df9359ff48982b1 HTTP/1.1" 200 None 2026-03-31 19:07:22,924 - lava_callback - DEBUG - Processing node 69cbfbe28df9359ff48982b1: ltp-fs_bind 2026-03-31 19:07:22,979 - lava_callback - INFO - Received callback for node 69cbfbe58df9359ff48982b4 INFO: 172.19.0.1:37530 - "POST /node/69cbfbe58df9359ff48982b4 HTTP/1.0" 202 Accepted 2026-03-31 19:07:23,001 - lava_callback - INFO - Starting async job submit for node 69cbfbe58df9359ff48982b4 2026-03-31 19:07:23,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:07:23,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbe58df9359ff48982b4 HTTP/1.1" 200 None 2026-03-31 19:07:23,176 - lava_callback - DEBUG - Processing node 69cbfbe58df9359ff48982b4: ltp-fsx 2026-03-31 19:07:23,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:07:23,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:07:23,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:07:23,844 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cbfbe28df9359ff48982b1/log.txt.gz 2026-03-31 19:07:23,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:07:23,919 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69cbfbe58df9359ff48982b4/log.txt.gz 2026-03-31 19:07:24,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:07:24,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:07:24,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:07:24,549 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69cbfbe58df9359ff48982b4/lava-logs.yaml 2026-03-31 19:07:24,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:07:24,553 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cbfbe28df9359ff48982b1/lava-logs.yaml 2026-03-31 19:07:24,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:07:24,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:07:24,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:07:24,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69cbfbe58df9359ff48982b4/lava_callback.json.gz 2026-03-31 19:07:24,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:07:24,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbe58df9359ff48982b4 HTTP/1.1" 200 None 2026-03-31 19:07:24,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:07:24,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:07:24,766 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cbfbe28df9359ff48982b1/lava_callback.json.gz 2026-03-31 19:07:24,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:07:24,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:07:24,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:07:24,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbe28df9359ff48982b1 HTTP/1.1" 200 None 2026-03-31 19:07:24,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:07:24,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbe58df9359ff48982b4 HTTP/1.1" 200 None 2026-03-31 19:07:24,848 - lava_callback - INFO - Completed processing callback for node 69cbfbe58df9359ff48982b4 2026-03-31 19:07:24,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:07:24,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:07:24,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbe28df9359ff48982b1 HTTP/1.1" 200 None 2026-03-31 19:07:24,907 - lava_callback - INFO - Completed processing callback for node 69cbfbe28df9359ff48982b1 INFO: 172.19.0.1:37542 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:07:29,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:07:29,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49128 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:08:03,716 - lava_callback - INFO - Received callback for node 69cc06e38df9359ff4898861 2026-03-31 19:08:03,739 - lava_callback - INFO - Received callback for node 69cbfbe98df9359ff48982c0 INFO: 172.19.0.1:47566 - "POST /node/69cbfbe98df9359ff48982c0 HTTP/1.0" 202 Accepted 2026-03-31 19:08:03,746 - lava_callback - INFO - Starting async job submit for node 69cbfbe98df9359ff48982c0 INFO: 172.19.0.1:47558 - "POST /node/69cc06e38df9359ff4898861 HTTP/1.0" 202 Accepted 2026-03-31 19:08:03,779 - lava_callback - INFO - Starting async job submit for node 69cc06e38df9359ff4898861 2026-03-31 19:08:03,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:03,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:03,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbe98df9359ff48982c0 HTTP/1.1" 200 None 2026-03-31 19:08:03,824 - lava_callback - DEBUG - Processing node 69cbfbe98df9359ff48982c0: ltp-ipc 2026-03-31 19:08:03,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc06e38df9359ff4898861 HTTP/1.1" 200 None 2026-03-31 19:08:03,889 - lava_callback - DEBUG - Processing node 69cc06e38df9359ff4898861: baseline-arm-pengutronix 2026-03-31 19:08:04,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:08:04,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:08:04,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:08:04,373 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc06e38df9359ff4898861/log.txt.gz 2026-03-31 19:08:04,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:08:04,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:08:04,460 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69cbfbe98df9359ff48982c0/log.txt.gz 2026-03-31 19:08:04,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:08:04,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:08:04,737 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc06e38df9359ff4898861/lava-logs.yaml 2026-03-31 19:08:04,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:08:05,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:08:05,005 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69cbfbe98df9359ff48982c0/lava-logs.yaml 2026-03-31 19:08:05,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:08:05,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:08:05,049 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc06e38df9359ff4898861/lava_callback.json.gz DEBUG: Setup failed for 69cc06e38df9359ff4898861. Transitting job node result: incomplete -> fail 2026-03-31 19:08:05,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:05,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc06e38df9359ff4898861 HTTP/1.1" 200 None 2026-03-31 19:08:05,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:05,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:08:05,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:05,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc06e38df9359ff4898861 HTTP/1.1" 200 None 2026-03-31 19:08:05,230 - lava_callback - INFO - Completed processing callback for node 69cc06e38df9359ff4898861 2026-03-31 19:08:05,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:08:05,264 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69cbfbe98df9359ff48982c0/lava_callback.json.gz 2026-03-31 19:08:05,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:05,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbe98df9359ff48982c0 HTTP/1.1" 200 None 2026-03-31 19:08:05,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:05,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:08:05,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:05,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbe98df9359ff48982c0 HTTP/1.1" 200 None 2026-03-31 19:08:05,370 - lava_callback - INFO - Completed processing callback for node 69cbfbe98df9359ff48982c0 INFO: 172.19.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39012 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:08:23,294 - lava_callback - INFO - Received callback for node 69cbfbe78df9359ff48982b7 INFO: 172.19.0.1:47636 - "POST /node/69cbfbe78df9359ff48982b7 HTTP/1.0" 202 Accepted 2026-03-31 19:08:23,301 - lava_callback - INFO - Starting async job submit for node 69cbfbe78df9359ff48982b7 2026-03-31 19:08:23,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:23,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbe78df9359ff48982b7 HTTP/1.1" 200 None 2026-03-31 19:08:23,371 - lava_callback - DEBUG - Processing node 69cbfbe78df9359ff48982b7: ltp-input 2026-03-31 19:08:23,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:08:23,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:08:23,923 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69cbfbe78df9359ff48982b7/log.txt.gz 2026-03-31 19:08:24,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:08:24,362 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:08:24,364 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69cbfbe78df9359ff48982b7/lava-logs.yaml 2026-03-31 19:08:24,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:08:24,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:08:24,521 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69cbfbe78df9359ff48982b7/lava_callback.json.gz 2026-03-31 19:08:24,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:24,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbe78df9359ff48982b7 HTTP/1.1" 200 None 2026-03-31 19:08:24,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:24,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:08:24,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:24,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbe78df9359ff48982b7 HTTP/1.1" 200 None 2026-03-31 19:08:24,602 - lava_callback - INFO - Completed processing callback for node 69cbfbe78df9359ff48982b7 INFO: 172.19.0.1:47646 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:08:29,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:08:29,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:34478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60284 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:11:22,861 - lava_callback - INFO - Received callback for node 69cbfbed8df9359ff48982da INFO: 172.19.0.1:59476 - "POST /node/69cbfbed8df9359ff48982da HTTP/1.0" 202 Accepted 2026-03-31 19:11:22,867 - lava_callback - INFO - Starting async job submit for node 69cbfbed8df9359ff48982da 2026-03-31 19:11:22,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:11:22,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbed8df9359ff48982da HTTP/1.1" 200 None 2026-03-31 19:11:22,911 - lava_callback - DEBUG - Processing node 69cbfbed8df9359ff48982da: ltp-sched 2026-03-31 19:11:23,100 - lava_callback - INFO - Received callback for node 69cbfbeb8df9359ff48982c6 INFO: 172.19.0.1:59480 - "POST /node/69cbfbeb8df9359ff48982c6 HTTP/1.0" 202 Accepted 2026-03-31 19:11:23,133 - lava_callback - INFO - Starting async job submit for node 69cbfbeb8df9359ff48982c6 2026-03-31 19:11:23,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:11:23,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbeb8df9359ff48982c6 HTTP/1.1" 200 None 2026-03-31 19:11:23,285 - lava_callback - DEBUG - Processing node 69cbfbeb8df9359ff48982c6: ltp-pty 2026-03-31 19:11:23,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:11:23,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:11:23,675 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69cbfbed8df9359ff48982da/log.txt.gz 2026-03-31 19:11:23,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:11:23,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:11:23,939 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:11:23,941 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69cbfbeb8df9359ff48982c6/log.txt.gz 2026-03-31 19:11:24,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:11:24,157 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69cbfbed8df9359ff48982da/lava-logs.yaml 2026-03-31 19:11:24,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:11:24,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:11:24,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:11:24,366 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69cbfbed8df9359ff48982da/lava_callback.json.gz 2026-03-31 19:11:24,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:11:24,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbed8df9359ff48982da HTTP/1.1" 200 None 2026-03-31 19:11:24,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:11:24,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:11:24,415 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69cbfbeb8df9359ff48982c6/lava-logs.yaml 2026-03-31 19:11:24,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:11:24,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:11:24,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:11:24,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbed8df9359ff48982da HTTP/1.1" 200 None 2026-03-31 19:11:24,462 - lava_callback - INFO - Completed processing callback for node 69cbfbed8df9359ff48982da 2026-03-31 19:11:24,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:11:24,654 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69cbfbeb8df9359ff48982c6/lava_callback.json.gz 2026-03-31 19:11:24,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:11:24,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbeb8df9359ff48982c6 HTTP/1.1" 200 None 2026-03-31 19:11:24,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:11:24,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:11:24,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:11:24,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbeb8df9359ff48982c6 HTTP/1.1" 200 None 2026-03-31 19:11:24,764 - lava_callback - INFO - Completed processing callback for node 69cbfbeb8df9359ff48982c6 INFO: 172.19.0.1:59490 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:11:29,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:11:29,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:35672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40278 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:12:03,386 - lava_callback - INFO - Received callback for node 69cbfbee8df9359ff48982dd INFO: 172.19.0.1:57940 - "POST /node/69cbfbee8df9359ff48982dd HTTP/1.0" 202 Accepted 2026-03-31 19:12:03,392 - lava_callback - INFO - Starting async job submit for node 69cbfbee8df9359ff48982dd 2026-03-31 19:12:03,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:12:03,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbee8df9359ff48982dd HTTP/1.1" 200 None 2026-03-31 19:12:03,450 - lava_callback - DEBUG - Processing node 69cbfbee8df9359ff48982dd: ltp-smoke 2026-03-31 19:12:03,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:12:03,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:12:03,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cbfbee8df9359ff48982dd/log.txt.gz 2026-03-31 19:12:04,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:12:04,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:12:04,327 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cbfbee8df9359ff48982dd/lava-logs.yaml 2026-03-31 19:12:04,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:12:04,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:12:04,487 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cbfbee8df9359ff48982dd/lava_callback.json.gz 2026-03-31 19:12:04,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:12:04,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbee8df9359ff48982dd HTTP/1.1" 200 None 2026-03-31 19:12:04,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:12:04,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:12:04,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:12:04,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbee8df9359ff48982dd HTTP/1.1" 200 None 2026-03-31 19:12:04,604 - lava_callback - INFO - Completed processing callback for node 69cbfbee8df9359ff48982dd INFO: 172.19.0.1:57956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42842 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:12:29,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:12:29,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:53562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57096 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:13:03,273 - lava_callback - INFO - Received callback for node 69cbfbf28df9359ff48982e3 INFO: 172.19.0.1:58746 - "POST /node/69cbfbf28df9359ff48982e3 HTTP/1.0" 202 Accepted 2026-03-31 19:13:03,279 - lava_callback - INFO - Starting async job submit for node 69cbfbf28df9359ff48982e3 2026-03-31 19:13:03,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:13:03,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbf28df9359ff48982e3 HTTP/1.1" 200 None 2026-03-31 19:13:03,320 - lava_callback - DEBUG - Processing node 69cbfbf28df9359ff48982e3: ltp-syscalls-ipc 2026-03-31 19:13:03,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:13:03,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:13:03,871 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cbfbf28df9359ff48982e3/log.txt.gz 2026-03-31 19:13:04,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:13:04,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:13:04,324 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cbfbf28df9359ff48982e3/lava-logs.yaml 2026-03-31 19:13:04,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:13:04,499 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:13:04,500 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cbfbf28df9359ff48982e3/lava_callback.json.gz 2026-03-31 19:13:04,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:13:04,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbf28df9359ff48982e3 HTTP/1.1" 200 None 2026-03-31 19:13:04,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:13:04,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:13:04,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:13:04,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbf28df9359ff48982e3 HTTP/1.1" 200 None 2026-03-31 19:13:04,604 - lava_callback - INFO - Completed processing callback for node 69cbfbf28df9359ff48982e3 INFO: 172.19.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45444 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:13:29,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:13:30,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:58276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59680 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:15:23,846 - lava_callback - INFO - Received callback for node 69cbfbf68df9359ff48982e9 INFO: 172.19.0.1:58114 - "POST /node/69cbfbf68df9359ff48982e9 HTTP/1.0" 202 Accepted 2026-03-31 19:15:23,852 - lava_callback - INFO - Starting async job submit for node 69cbfbf68df9359ff48982e9 2026-03-31 19:15:23,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:15:23,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbf68df9359ff48982e9 HTTP/1.1" 200 None 2026-03-31 19:15:23,892 - lava_callback - DEBUG - Processing node 69cbfbf68df9359ff48982e9: ltp-watchqueue 2026-03-31 19:15:24,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:15:24,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:15:24,400 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cbfbf68df9359ff48982e9/log.txt.gz 2026-03-31 19:15:24,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:15:24,767 - lava_callback - INFO - Received callback for node 69cbfbf48df9359ff48982e6 INFO: 172.19.0.1:58122 - "POST /node/69cbfbf48df9359ff48982e6 HTTP/1.0" 202 Accepted 2026-03-31 19:15:24,774 - lava_callback - INFO - Starting async job submit for node 69cbfbf48df9359ff48982e6 2026-03-31 19:15:24,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:15:24,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbf48df9359ff48982e6 HTTP/1.1" 200 None 2026-03-31 19:15:24,819 - lava_callback - DEBUG - Processing node 69cbfbf48df9359ff48982e6: ltp-timers 2026-03-31 19:15:24,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:15:24,887 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cbfbf68df9359ff48982e9/lava-logs.yaml 2026-03-31 19:15:24,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:15:25,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:15:25,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:15:25,188 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cbfbf68df9359ff48982e9/lava_callback.json.gz 2026-03-31 19:15:25,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:15:25,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbf68df9359ff48982e9 HTTP/1.1" 200 None 2026-03-31 19:15:25,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:15:25,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:15:25,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:15:25,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbf68df9359ff48982e9 HTTP/1.1" 200 None 2026-03-31 19:15:25,287 - lava_callback - INFO - Completed processing callback for node 69cbfbf68df9359ff48982e9 2026-03-31 19:15:25,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:15:25,388 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69cbfbf48df9359ff48982e6/log.txt.gz 2026-03-31 19:15:25,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:15:25,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:15:25,841 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69cbfbf48df9359ff48982e6/lava-logs.yaml 2026-03-31 19:15:25,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:15:26,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:15:26,034 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69cbfbf48df9359ff48982e6/lava_callback.json.gz 2026-03-31 19:15:26,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:15:26,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbfbf48df9359ff48982e6 HTTP/1.1" 200 None 2026-03-31 19:15:26,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:15:26,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:15:26,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:15:26,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cbfbf48df9359ff48982e6 HTTP/1.1" 200 None 2026-03-31 19:15:26,124 - lava_callback - INFO - Completed processing callback for node 69cbfbf48df9359ff48982e6 INFO: 172.19.0.1:58132 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:15:30,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:15:30,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44536 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:19:57,839 - lava_callback - INFO - Received callback for node 69cc19a98df9359ff4898c7a INFO: 172.19.0.1:44544 - "POST /node/69cc19a98df9359ff4898c7a HTTP/1.0" 202 Accepted 2026-03-31 19:19:57,847 - lava_callback - INFO - Starting async job submit for node 69cc19a98df9359ff4898c7a 2026-03-31 19:19:57,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:19:57,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc19a98df9359ff4898c7a HTTP/1.1" 200 None 2026-03-31 19:19:57,908 - lava_callback - DEBUG - Processing node 69cc19a98df9359ff4898c7a: baseline-nfs-arm64 2026-03-31 19:19:59,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:19:59,562 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:19:59,564 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc19a98df9359ff4898c7a/log.txt.gz 2026-03-31 19:20:00,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:20:00,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:20:00,657 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc19a98df9359ff4898c7a/lava-logs.yaml 2026-03-31 19:20:00,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:20:00,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:20:00,849 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cc19a98df9359ff4898c7a/lava_callback.json.gz DEBUG: Setup failed for 69cc19a98df9359ff4898c7a. Transitting job node result: incomplete -> fail 2026-03-31 19:20:00,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:20:00,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc19a98df9359ff4898c7a HTTP/1.1" 200 None 2026-03-31 19:20:00,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:20:00,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-03-31 19:20:00,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:20:00,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc19a98df9359ff4898c7a HTTP/1.1" 200 None 2026-03-31 19:20:01,085 - lava_callback - INFO - Completed processing callback for node 69cc19a98df9359ff4898c7a INFO: 172.19.0.1:34938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36906 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36910 - "GET /node/69876cbce31b58f834f19a05%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:36916 - "GET /node/694acbba6c96ed51d316f796%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-31 19:20:30,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:20:30,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:38448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45130 - "GET /node/6992ed6b31166481c8dd9cc2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:45146 - "GET /node/69884c6ae408e9ea60fad270%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45562 - "GET /node/694ac6726c96ed51d316f2d2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:44480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36668 - "GET /node/69280ea7b1b5e06bf6c04e70%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45764 - "GET /node/694aad4c83c67c43d1ede685%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:45770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44088 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46860 - "GET /node/6984d50e3ff18e47df612911%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45960 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33846 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60062 - "GET /node/694aae6783c67c43d1ede740%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:51292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35032 - "GET /node/694991cbd45a0a86fbe25e6c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:41974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45728 - "GET /node/6928cbd63a49357e496c8ee5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:45734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47276 - "GET /node/69bd9bf40e93e231472418fc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:47278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44816 - "GET /node/6988c6b89fc35f969e1eee4a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:44822 - "GET /node/69af6c16275c8d3b7939a57b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54442 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:46:45,627 - lava_callback - INFO - Received callback for node 69cc1bdd8df9359ff4898ced INFO: 172.19.0.1:47188 - "POST /node/69cc1bdd8df9359ff4898ced HTTP/1.0" 202 Accepted 2026-03-31 19:46:45,673 - lava_callback - INFO - Starting async job submit for node 69cc1bdd8df9359ff4898ced 2026-03-31 19:46:45,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:46:45,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc1bdd8df9359ff4898ced HTTP/1.1" 200 None 2026-03-31 19:46:45,712 - lava_callback - DEBUG - Processing node 69cc1bdd8df9359ff4898ced: baseline-arm-pengutronix 2026-03-31 19:46:45,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:46:46,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:46:46,017 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc1bdd8df9359ff4898ced/log.txt.gz 2026-03-31 19:46:46,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:46:46,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:46:46,260 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc1bdd8df9359ff4898ced/lava-logs.yaml 2026-03-31 19:46:46,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:46:46,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:46:46,452 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc1bdd8df9359ff4898ced/lava_callback.json.gz DEBUG: Setup failed for 69cc1bdd8df9359ff4898ced. Transitting job node result: incomplete -> fail 2026-03-31 19:46:46,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:47190 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:46:46,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc1bdd8df9359ff4898ced HTTP/1.1" 200 None 2026-03-31 19:46:46,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:46:46,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 19:46:46,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:46:46,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc1bdd8df9359ff4898ced HTTP/1.1" 200 None 2026-03-31 19:46:46,622 - lava_callback - INFO - Completed processing callback for node 69cc1bdd8df9359ff4898ced INFO: 172.19.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56624 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:47:00,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:47:00,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:41890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55212 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39050 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:50:02,660 - lava_callback - INFO - Received callback for node 69c98263f2866272ea51eb5a INFO: 172.19.0.1:52340 - "POST /node/69c98263f2866272ea51eb5a HTTP/1.0" 202 Accepted 2026-03-31 19:50:02,665 - lava_callback - INFO - Starting async job submit for node 69c98263f2866272ea51eb5a 2026-03-31 19:50:02,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:50:02,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98263f2866272ea51eb5a HTTP/1.1" 200 None 2026-03-31 19:50:02,682 - lava_callback - DEBUG - Processing node 69c98263f2866272ea51eb5a: baseline-arm64-baylibre 2026-03-31 19:50:02,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98263f2866272ea51eb5a 2026-03-31 19:50:02,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:50:02,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98263f2866272ea51eb5a HTTP/1.1" 200 None 2026-03-31 19:50:02,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:50:02,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 19:50:02,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:50:02,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98263f2866272ea51eb5a HTTP/1.1" 200 None 2026-03-31 19:50:02,750 - lava_callback - ERROR - Error processing callback for node 69c98263f2866272ea51eb5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57554 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:50:29,787 - lava_callback - INFO - Received callback for node 69cc0c048df9359ff4898938 INFO: 172.19.0.1:53444 - "POST /node/69cc0c048df9359ff4898938 HTTP/1.0" 202 Accepted 2026-03-31 19:50:29,798 - lava_callback - INFO - Starting async job submit for node 69cc0c048df9359ff4898938 2026-03-31 19:50:29,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:50:29,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc0c048df9359ff4898938 HTTP/1.1" 200 None 2026-03-31 19:50:29,910 - lava_callback - DEBUG - Processing node 69cc0c048df9359ff4898938: ltp-smoke 2026-03-31 19:50:30,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:50:30,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:50:30,722 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc0c048df9359ff4898938/log.txt.gz 2026-03-31 19:50:31,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:50:31,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:50:31,271 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc0c048df9359ff4898938/lava-logs.yaml 2026-03-31 19:50:31,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 19:50:31,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 19:50:31,431 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc0c048df9359ff4898938/lava_callback.json.gz 2026-03-31 19:50:31,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:50:31,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc0c048df9359ff4898938 HTTP/1.1" 200 None 2026-03-31 19:50:31,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:50:31,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4198df9359ff4897d2e HTTP/1.1" 200 None 2026-03-31 19:50:31,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:50:31,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc0c048df9359ff4898938 HTTP/1.1" 200 None 2026-03-31 19:50:31,728 - lava_callback - INFO - Completed processing callback for node 69cc0c048df9359ff4898938 INFO: 172.19.0.1:53460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45340 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:51:00,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:51:00,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:41620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43970 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33204 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:51:35,240 - lava_callback - INFO - Received callback for node 69c982baf2866272ea51eb65 INFO: 172.19.0.1:40642 - "POST /node/69c982baf2866272ea51eb65 HTTP/1.0" 202 Accepted 2026-03-31 19:51:35,246 - lava_callback - INFO - Starting async job submit for node 69c982baf2866272ea51eb65 2026-03-31 19:51:35,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:51:35,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c982baf2866272ea51eb65 HTTP/1.1" 200 None 2026-03-31 19:51:35,267 - lava_callback - DEBUG - Processing node 69c982baf2866272ea51eb65: baseline-arm64-clabbe 2026-03-31 19:51:35,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c982baf2866272ea51eb65 2026-03-31 19:51:35,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:51:35,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c982baf2866272ea51eb65 HTTP/1.1" 200 None 2026-03-31 19:51:35,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:51:35,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 19:51:35,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:51:35,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c982baf2866272ea51eb65 HTTP/1.1" 200 None 2026-03-31 19:51:35,329 - lava_callback - ERROR - Error processing callback for node 69c982baf2866272ea51eb65: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37652 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:52:55,022 - lava_callback - INFO - Received callback for node 69c9830cf2866272ea51eb72 INFO: 172.19.0.1:59316 - "POST /node/69c9830cf2866272ea51eb72 HTTP/1.0" 202 Accepted 2026-03-31 19:52:55,028 - lava_callback - INFO - Starting async job submit for node 69c9830cf2866272ea51eb72 2026-03-31 19:52:55,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:52:55,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9830cf2866272ea51eb72 HTTP/1.1" 200 None 2026-03-31 19:52:55,043 - lava_callback - DEBUG - Processing node 69c9830cf2866272ea51eb72: baseline-arm64-clabbe 2026-03-31 19:52:55,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9830cf2866272ea51eb72 2026-03-31 19:52:55,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:52:55,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9830cf2866272ea51eb72 HTTP/1.1" 200 None 2026-03-31 19:52:55,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:52:55,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 19:52:55,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:52:55,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9830cf2866272ea51eb72 HTTP/1.1" 200 None 2026-03-31 19:52:55,108 - lava_callback - ERROR - Error processing callback for node 69c9830cf2866272ea51eb72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58712 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:54:15,055 - lava_callback - INFO - Received callback for node 69c9835df2866272ea51eb7e INFO: 172.19.0.1:34296 - "POST /node/69c9835df2866272ea51eb7e HTTP/1.0" 202 Accepted 2026-03-31 19:54:15,061 - lava_callback - INFO - Starting async job submit for node 69c9835df2866272ea51eb7e 2026-03-31 19:54:15,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:54:15,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9835df2866272ea51eb7e HTTP/1.1" 200 None 2026-03-31 19:54:15,083 - lava_callback - DEBUG - Processing node 69c9835df2866272ea51eb7e: baseline-arm64-clabbe 2026-03-31 19:54:15,083 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9835df2866272ea51eb7e 2026-03-31 19:54:15,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:54:15,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9835df2866272ea51eb7e HTTP/1.1" 200 None 2026-03-31 19:54:15,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:54:15,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 19:54:15,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:54:15,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9835df2866272ea51eb7e HTTP/1.1" 200 None 2026-03-31 19:54:15,151 - lava_callback - ERROR - Error processing callback for node 69c9835df2866272ea51eb7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44712 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:55:55,116 - lava_callback - INFO - Received callback for node 69c983bcf2866272ea51eb89 INFO: 172.19.0.1:45368 - "POST /node/69c983bcf2866272ea51eb89 HTTP/1.0" 202 Accepted 2026-03-31 19:55:55,121 - lava_callback - INFO - Starting async job submit for node 69c983bcf2866272ea51eb89 2026-03-31 19:55:55,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:55:55,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c983bcf2866272ea51eb89 HTTP/1.1" 200 None 2026-03-31 19:55:55,138 - lava_callback - DEBUG - Processing node 69c983bcf2866272ea51eb89: baseline-arm64-clabbe 2026-03-31 19:55:55,138 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c983bcf2866272ea51eb89 2026-03-31 19:55:55,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:55:55,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c983bcf2866272ea51eb89 HTTP/1.1" 200 None 2026-03-31 19:55:55,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:55:55,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 19:55:55,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:55:55,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c983bcf2866272ea51eb89 HTTP/1.1" 200 None 2026-03-31 19:55:55,224 - lava_callback - ERROR - Error processing callback for node 69c983bcf2866272ea51eb89: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35132 - "GET / HTTP/1.0" 200 OK 2026-03-31 19:57:55,152 - lava_callback - INFO - Received callback for node 69c98432f2866272ea51eb94 INFO: 172.19.0.1:52240 - "POST /node/69c98432f2866272ea51eb94 HTTP/1.0" 202 Accepted 2026-03-31 19:57:55,157 - lava_callback - INFO - Starting async job submit for node 69c98432f2866272ea51eb94 2026-03-31 19:57:55,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:57:55,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98432f2866272ea51eb94 HTTP/1.1" 200 None 2026-03-31 19:57:55,173 - lava_callback - DEBUG - Processing node 69c98432f2866272ea51eb94: baseline-arm64-clabbe 2026-03-31 19:57:55,173 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98432f2866272ea51eb94 2026-03-31 19:57:55,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:57:55,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98432f2866272ea51eb94 HTTP/1.1" 200 None 2026-03-31 19:57:55,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:57:55,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 19:57:55,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:57:55,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98432f2866272ea51eb94 HTTP/1.1" 200 None 2026-03-31 19:57:55,228 - lava_callback - ERROR - Error processing callback for node 69c98432f2866272ea51eb94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54536 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 19:59:15,252 - lava_callback - INFO - Received callback for node 69c98483f2866272ea51eb9c INFO: 172.19.0.1:54538 - "POST /node/69c98483f2866272ea51eb9c HTTP/1.0" 202 Accepted 2026-03-31 19:59:15,257 - lava_callback - INFO - Starting async job submit for node 69c98483f2866272ea51eb9c 2026-03-31 19:59:15,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:59:15,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98483f2866272ea51eb9c HTTP/1.1" 200 None 2026-03-31 19:59:15,282 - lava_callback - DEBUG - Processing node 69c98483f2866272ea51eb9c: baseline-arm64-clabbe 2026-03-31 19:59:15,282 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98483f2866272ea51eb9c 2026-03-31 19:59:15,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:59:15,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98483f2866272ea51eb9c HTTP/1.1" 200 None 2026-03-31 19:59:15,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:59:15,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-31 19:59:15,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 19:59:15,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98483f2866272ea51eb9c HTTP/1.1" 200 None 2026-03-31 19:59:15,339 - lava_callback - ERROR - Error processing callback for node 69c98483f2866272ea51eb9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42710 - "GET /node/69bda4f50e93e23147241dbf%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:37046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40868 - "GET /node/69a2b9db8020dd22eabee76a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:40876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36550 - "GET /node/693af179fee8383e92b69dbc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:36560 - "GET /node/693fb0150d8ce4018da9c0f8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:52754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35634 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38030 - "GET /node/694acae76c96ed51d316f5bf%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:47950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50044 - "GET /node/69bd7c860e93e2314723e913%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38824 - "GET /node/698b33fd3b3beb19e761a1c7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53960 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45432 - "GET /node/698f61673013abef95cfa38c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:42062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43976 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36018 - "GET /node/693f75200d8ce4018da93f15%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:36028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53350 - "GET /node/6986ffdf6e077e4126a7e12d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:53356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39760 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43750 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:26:06,892 - lava_callback - INFO - Received callback for node 69cc25898df9359ff4898f9a INFO: 172.19.0.1:43760 - "POST /node/69cc25898df9359ff4898f9a HTTP/1.0" 202 Accepted 2026-03-31 20:26:06,899 - lava_callback - INFO - Starting async job submit for node 69cc25898df9359ff4898f9a 2026-03-31 20:26:06,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:26:06,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc25898df9359ff4898f9a HTTP/1.1" 200 None 2026-03-31 20:26:06,990 - lava_callback - DEBUG - Processing node 69cc25898df9359ff4898f9a: ltp-smoke 2026-03-31 20:26:07,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 20:26:07,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 20:26:07,666 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc25898df9359ff4898f9a/log.txt.gz 2026-03-31 20:26:07,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 20:26:08,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 20:26:08,144 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc25898df9359ff4898f9a/lava-logs.yaml 2026-03-31 20:26:08,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 20:26:08,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 20:26:08,303 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cc25898df9359ff4898f9a/lava_callback.json.gz 2026-03-31 20:26:08,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:26:08,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc25898df9359ff4898f9a HTTP/1.1" 200 None 2026-03-31 20:26:08,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:26:08,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4198df9359ff4897d2e HTTP/1.1" 200 None 2026-03-31 20:26:08,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:26:08,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc25898df9359ff4898f9a HTTP/1.1" 200 None 2026-03-31 20:26:08,591 - lava_callback - INFO - Completed processing callback for node 69cc25898df9359ff4898f9a INFO: 172.19.0.1:53016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40384 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:26:30,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:26:30,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:58594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35352 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35018 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34044 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36304 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:39:05,468 - lava_callback - INFO - Received callback for node 69c98de4f2866272ea51ebd3 INFO: 172.19.0.1:55026 - "POST /node/69c98de4f2866272ea51ebd3 HTTP/1.0" 202 Accepted 2026-03-31 20:39:05,475 - lava_callback - INFO - Starting async job submit for node 69c98de4f2866272ea51ebd3 2026-03-31 20:39:05,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:39:05,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98de4f2866272ea51ebd3 HTTP/1.1" 200 None 2026-03-31 20:39:05,497 - lava_callback - DEBUG - Processing node 69c98de4f2866272ea51ebd3: baseline-arm-baylibre 2026-03-31 20:39:05,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98de4f2866272ea51ebd3 2026-03-31 20:39:05,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:39:05,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98de4f2866272ea51ebd3 HTTP/1.1" 200 None 2026-03-31 20:39:05,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:39:05,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 20:39:05,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:39:05,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98de4f2866272ea51ebd3 HTTP/1.1" 200 None 2026-03-31 20:39:05,554 - lava_callback - ERROR - Error processing callback for node 69c98de4f2866272ea51ebd3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58202 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:39:45,368 - lava_callback - INFO - Received callback for node 69c98e09f2866272ea51ebe0 INFO: 172.19.0.1:43706 - "POST /node/69c98e09f2866272ea51ebe0 HTTP/1.0" 202 Accepted 2026-03-31 20:39:45,374 - lava_callback - INFO - Starting async job submit for node 69c98e09f2866272ea51ebe0 2026-03-31 20:39:45,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:39:45,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98e09f2866272ea51ebe0 HTTP/1.1" 200 None 2026-03-31 20:39:45,390 - lava_callback - DEBUG - Processing node 69c98e09f2866272ea51ebe0: baseline-arm-baylibre 2026-03-31 20:39:45,390 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98e09f2866272ea51ebe0 2026-03-31 20:39:45,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:39:45,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98e09f2866272ea51ebe0 HTTP/1.1" 200 None 2026-03-31 20:39:45,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:39:45,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 20:39:45,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:39:45,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98e09f2866272ea51ebe0 HTTP/1.1" 200 None 2026-03-31 20:39:45,453 - lava_callback - ERROR - Error processing callback for node 69c98e09f2866272ea51ebe0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54742 - "GET /node/69b7d39400b80c8ffdcb8b7d%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-31 20:40:25,477 - lava_callback - INFO - Received callback for node 69c98e2ff2866272ea51ebea INFO: 172.19.0.1:54758 - "POST /node/69c98e2ff2866272ea51ebea HTTP/1.0" 202 Accepted 2026-03-31 20:40:25,483 - lava_callback - INFO - Starting async job submit for node 69c98e2ff2866272ea51ebea 2026-03-31 20:40:25,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:40:25,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98e2ff2866272ea51ebea HTTP/1.1" 200 None 2026-03-31 20:40:25,502 - lava_callback - DEBUG - Processing node 69c98e2ff2866272ea51ebea: baseline-arm-baylibre 2026-03-31 20:40:25,502 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98e2ff2866272ea51ebea 2026-03-31 20:40:25,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:40:25,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98e2ff2866272ea51ebea HTTP/1.1" 200 None 2026-03-31 20:40:25,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:40:25,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 20:40:25,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:40:25,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98e2ff2866272ea51ebea HTTP/1.1" 200 None 2026-03-31 20:40:25,557 - lava_callback - ERROR - Error processing callback for node 69c98e2ff2866272ea51ebea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44386 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:41:05,428 - lava_callback - INFO - Received callback for node 69c98e55f2866272ea51ebfa INFO: 172.19.0.1:34400 - "POST /node/69c98e55f2866272ea51ebfa HTTP/1.0" 202 Accepted 2026-03-31 20:41:05,433 - lava_callback - INFO - Starting async job submit for node 69c98e55f2866272ea51ebfa 2026-03-31 20:41:05,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:41:05,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98e55f2866272ea51ebfa HTTP/1.1" 200 None 2026-03-31 20:41:05,449 - lava_callback - DEBUG - Processing node 69c98e55f2866272ea51ebfa: baseline-arm-baylibre 2026-03-31 20:41:05,449 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98e55f2866272ea51ebfa 2026-03-31 20:41:05,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:41:05,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98e55f2866272ea51ebfa HTTP/1.1" 200 None 2026-03-31 20:41:05,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:41:05,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 20:41:05,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:41:05,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98e55f2866272ea51ebfa HTTP/1.1" 200 None 2026-03-31 20:41:05,501 - lava_callback - ERROR - Error processing callback for node 69c98e55f2866272ea51ebfa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36750 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:41:45,579 - lava_callback - INFO - Received callback for node 69c98e78f2866272ea51ec08 INFO: 172.19.0.1:53766 - "POST /node/69c98e78f2866272ea51ec08 HTTP/1.0" 202 Accepted 2026-03-31 20:41:45,585 - lava_callback - INFO - Starting async job submit for node 69c98e78f2866272ea51ec08 2026-03-31 20:41:45,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:41:45,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98e78f2866272ea51ec08 HTTP/1.1" 200 None 2026-03-31 20:41:45,604 - lava_callback - DEBUG - Processing node 69c98e78f2866272ea51ec08: baseline-arm-baylibre 2026-03-31 20:41:45,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98e78f2866272ea51ec08 2026-03-31 20:41:45,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:41:45,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98e78f2866272ea51ec08 HTTP/1.1" 200 None 2026-03-31 20:41:45,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:41:45,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:41:45,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:41:45,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98e78f2866272ea51ec08 HTTP/1.1" 200 None 2026-03-31 20:41:45,664 - lava_callback - ERROR - Error processing callback for node 69c98e78f2866272ea51ec08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59162 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:42:25,553 - lava_callback - INFO - Received callback for node 69c98e9ff2866272ea51ec13 INFO: 172.19.0.1:59394 - "POST /node/69c98e9ff2866272ea51ec13 HTTP/1.0" 202 Accepted 2026-03-31 20:42:25,559 - lava_callback - INFO - Starting async job submit for node 69c98e9ff2866272ea51ec13 2026-03-31 20:42:25,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:42:25,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98e9ff2866272ea51ec13 HTTP/1.1" 200 None 2026-03-31 20:42:25,579 - lava_callback - DEBUG - Processing node 69c98e9ff2866272ea51ec13: baseline-arm-baylibre 2026-03-31 20:42:25,579 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98e9ff2866272ea51ec13 2026-03-31 20:42:25,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:42:25,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98e9ff2866272ea51ec13 HTTP/1.1" 200 None 2026-03-31 20:42:25,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:42:25,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:42:25,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:42:25,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98e9ff2866272ea51ec13 HTTP/1.1" 200 None 2026-03-31 20:42:25,691 - lava_callback - ERROR - Error processing callback for node 69c98e9ff2866272ea51ec13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48666 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:43:05,648 - lava_callback - INFO - Received callback for node 69c98ec6f2866272ea51ec21 INFO: 172.19.0.1:47844 - "POST /node/69c98ec6f2866272ea51ec21 HTTP/1.0" 202 Accepted 2026-03-31 20:43:05,654 - lava_callback - INFO - Starting async job submit for node 69c98ec6f2866272ea51ec21 2026-03-31 20:43:05,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:43:05,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98ec6f2866272ea51ec21 HTTP/1.1" 200 None 2026-03-31 20:43:05,675 - lava_callback - DEBUG - Processing node 69c98ec6f2866272ea51ec21: baseline-arm-baylibre 2026-03-31 20:43:05,676 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98ec6f2866272ea51ec21 2026-03-31 20:43:05,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:43:05,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98ec6f2866272ea51ec21 HTTP/1.1" 200 None 2026-03-31 20:43:05,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:43:05,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 20:43:05,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:43:05,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98ec6f2866272ea51ec21 HTTP/1.1" 200 None 2026-03-31 20:43:05,763 - lava_callback - ERROR - Error processing callback for node 69c98ec6f2866272ea51ec21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38076 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:44:05,696 - lava_callback - INFO - Received callback for node 69c98f0ef2866272ea51ec3a INFO: 172.19.0.1:52370 - "POST /node/69c98f0ef2866272ea51ec3a HTTP/1.0" 202 Accepted 2026-03-31 20:44:05,702 - lava_callback - INFO - Starting async job submit for node 69c98f0ef2866272ea51ec3a 2026-03-31 20:44:05,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:44:05,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98f0ef2866272ea51ec3a HTTP/1.1" 200 None 2026-03-31 20:44:05,723 - lava_callback - DEBUG - Processing node 69c98f0ef2866272ea51ec3a: baseline-arm-baylibre 2026-03-31 20:44:05,723 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98f0ef2866272ea51ec3a 2026-03-31 20:44:05,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:44:05,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98f0ef2866272ea51ec3a HTTP/1.1" 200 None 2026-03-31 20:44:05,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:44:05,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:44:05,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:44:05,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98f0ef2866272ea51ec3a HTTP/1.1" 200 None 2026-03-31 20:44:05,784 - lava_callback - ERROR - Error processing callback for node 69c98f0ef2866272ea51ec3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58174 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58632 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:45:37,389 - lava_callback - INFO - Received callback for node 69c98f5cf2866272ea51ec58 INFO: 172.19.0.1:58644 - "POST /node/69c98f5cf2866272ea51ec58 HTTP/1.0" 202 Accepted 2026-03-31 20:45:37,396 - lava_callback - INFO - Starting async job submit for node 69c98f5cf2866272ea51ec58 2026-03-31 20:45:37,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:45:37,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98f5cf2866272ea51ec58 HTTP/1.1" 200 None 2026-03-31 20:45:37,418 - lava_callback - DEBUG - Processing node 69c98f5cf2866272ea51ec58: baseline-arm-clabbe 2026-03-31 20:45:37,418 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98f5cf2866272ea51ec58 2026-03-31 20:45:37,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:45:37,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98f5cf2866272ea51ec58 HTTP/1.1" 200 None 2026-03-31 20:45:37,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:45:37,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 20:45:37,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:45:37,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98f5cf2866272ea51ec58 HTTP/1.1" 200 None 2026-03-31 20:45:37,506 - lava_callback - ERROR - Error processing callback for node 69c98f5cf2866272ea51ec58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50174 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:46:17,407 - lava_callback - INFO - Received callback for node 69c98f85f2866272ea51ec66 INFO: 172.19.0.1:50182 - "POST /node/69c98f85f2866272ea51ec66 HTTP/1.0" 202 Accepted 2026-03-31 20:46:17,412 - lava_callback - INFO - Starting async job submit for node 69c98f85f2866272ea51ec66 2026-03-31 20:46:17,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:46:17,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98f85f2866272ea51ec66 HTTP/1.1" 200 None 2026-03-31 20:46:17,433 - lava_callback - DEBUG - Processing node 69c98f85f2866272ea51ec66: baseline-arm-clabbe 2026-03-31 20:46:17,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98f85f2866272ea51ec66 2026-03-31 20:46:17,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:46:17,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98f85f2866272ea51ec66 HTTP/1.1" 200 None 2026-03-31 20:46:17,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:46:17,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 20:46:17,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:46:17,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98f85f2866272ea51ec66 HTTP/1.1" 200 None 2026-03-31 20:46:17,491 - lava_callback - ERROR - Error processing callback for node 69c98f85f2866272ea51ec66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37690 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:46:45,807 - lava_callback - INFO - Received callback for node 69c98fa6f2866272ea51ec77 INFO: 172.19.0.1:55290 - "POST /node/69c98fa6f2866272ea51ec77 HTTP/1.0" 202 Accepted 2026-03-31 20:46:45,813 - lava_callback - INFO - Starting async job submit for node 69c98fa6f2866272ea51ec77 2026-03-31 20:46:45,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:46:45,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98fa6f2866272ea51ec77 HTTP/1.1" 200 None 2026-03-31 20:46:45,833 - lava_callback - DEBUG - Processing node 69c98fa6f2866272ea51ec77: baseline-arm-baylibre 2026-03-31 20:46:45,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98fa6f2866272ea51ec77 2026-03-31 20:46:45,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:46:45,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98fa6f2866272ea51ec77 HTTP/1.1" 200 None 2026-03-31 20:46:45,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:46:45,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:46:45,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:46:45,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98fa6f2866272ea51ec77 HTTP/1.1" 200 None 2026-03-31 20:46:45,883 - lava_callback - ERROR - Error processing callback for node 69c98fa6f2866272ea51ec77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54516 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:47:17,463 - lava_callback - INFO - Received callback for node 69c98fcef2866272ea51ec7f INFO: 172.19.0.1:54528 - "POST /node/69c98fcef2866272ea51ec7f HTTP/1.0" 202 Accepted 2026-03-31 20:47:17,469 - lava_callback - INFO - Starting async job submit for node 69c98fcef2866272ea51ec7f 2026-03-31 20:47:17,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:47:17,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98fcef2866272ea51ec7f HTTP/1.1" 200 None 2026-03-31 20:47:17,657 - lava_callback - DEBUG - Processing node 69c98fcef2866272ea51ec7f: baseline-arm-clabbe 2026-03-31 20:47:17,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98fcef2866272ea51ec7f 2026-03-31 20:47:17,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:47:17,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98fcef2866272ea51ec7f HTTP/1.1" 200 None 2026-03-31 20:47:17,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:47:17,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 20:47:17,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:47:17,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98fcef2866272ea51ec7f HTTP/1.1" 200 None 2026-03-31 20:47:17,737 - lava_callback - ERROR - Error processing callback for node 69c98fcef2866272ea51ec7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36242 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:47:57,504 - lava_callback - INFO - Received callback for node 69c98ff4f2866272ea51ec8e INFO: 172.19.0.1:36254 - "POST /node/69c98ff4f2866272ea51ec8e HTTP/1.0" 202 Accepted 2026-03-31 20:47:57,509 - lava_callback - INFO - Starting async job submit for node 69c98ff4f2866272ea51ec8e 2026-03-31 20:47:57,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:47:57,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98ff4f2866272ea51ec8e HTTP/1.1" 200 None 2026-03-31 20:47:57,525 - lava_callback - DEBUG - Processing node 69c98ff4f2866272ea51ec8e: baseline-arm-clabbe 2026-03-31 20:47:57,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c98ff4f2866272ea51ec8e 2026-03-31 20:47:57,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:47:57,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c98ff4f2866272ea51ec8e HTTP/1.1" 200 None 2026-03-31 20:47:57,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:47:57,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 20:47:57,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:47:57,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c98ff4f2866272ea51ec8e HTTP/1.1" 200 None 2026-03-31 20:47:57,577 - lava_callback - ERROR - Error processing callback for node 69c98ff4f2866272ea51ec8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58148 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55264 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:48:25,936 - lava_callback - INFO - Received callback for node 69c99016f2866272ea51ec9b INFO: 172.19.0.1:34926 - "POST /node/69c99016f2866272ea51ec9b HTTP/1.0" 202 Accepted 2026-03-31 20:48:25,941 - lava_callback - INFO - Starting async job submit for node 69c99016f2866272ea51ec9b 2026-03-31 20:48:25,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:48:25,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99016f2866272ea51ec9b HTTP/1.1" 200 None 2026-03-31 20:48:25,960 - lava_callback - DEBUG - Processing node 69c99016f2866272ea51ec9b: baseline-arm-baylibre 2026-03-31 20:48:25,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99016f2866272ea51ec9b 2026-03-31 20:48:25,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:48:25,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99016f2866272ea51ec9b HTTP/1.1" 200 None 2026-03-31 20:48:25,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:48:25,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:48:26,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:48:26,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99016f2866272ea51ec9b HTTP/1.1" 200 None 2026-03-31 20:48:26,032 - lava_callback - ERROR - Error processing callback for node 69c99016f2866272ea51ec9b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50102 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:49:17,530 - lava_callback - INFO - Received callback for node 69c9903cf2866272ea51eca9 INFO: 172.19.0.1:50108 - "POST /node/69c9903cf2866272ea51eca9 HTTP/1.0" 202 Accepted 2026-03-31 20:49:17,536 - lava_callback - INFO - Starting async job submit for node 69c9903cf2866272ea51eca9 2026-03-31 20:49:17,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:49:17,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9903cf2866272ea51eca9 HTTP/1.1" 200 None 2026-03-31 20:49:17,551 - lava_callback - DEBUG - Processing node 69c9903cf2866272ea51eca9: baseline-arm-clabbe 2026-03-31 20:49:17,551 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9903cf2866272ea51eca9 2026-03-31 20:49:17,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:49:17,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9903cf2866272ea51eca9 HTTP/1.1" 200 None 2026-03-31 20:49:17,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:49:17,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dfeb6a4c026e16347d36 HTTP/1.1" 200 None 2026-03-31 20:49:17,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:49:17,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9903cf2866272ea51eca9 HTTP/1.1" 200 None 2026-03-31 20:49:17,632 - lava_callback - ERROR - Error processing callback for node 69c9903cf2866272ea51eca9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47592 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:49:46,162 - lava_callback - INFO - Received callback for node 69c99060f2866272ea51ecba INFO: 172.19.0.1:57548 - "POST /node/69c99060f2866272ea51ecba HTTP/1.0" 202 Accepted 2026-03-31 20:49:46,168 - lava_callback - INFO - Starting async job submit for node 69c99060f2866272ea51ecba 2026-03-31 20:49:46,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:49:46,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99060f2866272ea51ecba HTTP/1.1" 200 None 2026-03-31 20:49:46,192 - lava_callback - DEBUG - Processing node 69c99060f2866272ea51ecba: baseline-arm-baylibre 2026-03-31 20:49:46,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99060f2866272ea51ecba 2026-03-31 20:49:46,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:49:46,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99060f2866272ea51ecba HTTP/1.1" 200 None 2026-03-31 20:49:46,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:49:46,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:49:46,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:49:46,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99060f2866272ea51ecba HTTP/1.1" 200 None 2026-03-31 20:49:46,249 - lava_callback - ERROR - Error processing callback for node 69c99060f2866272ea51ecba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47702 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:50:25,973 - lava_callback - INFO - Received callback for node 69c99080f2866272ea51ecc3 INFO: 172.19.0.1:52444 - "POST /node/69c99080f2866272ea51ecc3 HTTP/1.0" 202 Accepted 2026-03-31 20:50:25,979 - lava_callback - INFO - Starting async job submit for node 69c99080f2866272ea51ecc3 2026-03-31 20:50:25,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:50:25,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99080f2866272ea51ecc3 HTTP/1.1" 200 None 2026-03-31 20:50:25,995 - lava_callback - DEBUG - Processing node 69c99080f2866272ea51ecc3: baseline-arm-baylibre 2026-03-31 20:50:25,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99080f2866272ea51ecc3 2026-03-31 20:50:25,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:50:26,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99080f2866272ea51ecc3 HTTP/1.1" 200 None 2026-03-31 20:50:26,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:50:26,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:50:26,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:50:26,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99080f2866272ea51ecc3 HTTP/1.1" 200 None 2026-03-31 20:50:26,046 - lava_callback - ERROR - Error processing callback for node 69c99080f2866272ea51ecc3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52452 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44370 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:51:06,040 - lava_callback - INFO - Received callback for node 69c990a5f2866272ea51ecd4 INFO: 172.19.0.1:46742 - "POST /node/69c990a5f2866272ea51ecd4 HTTP/1.0" 202 Accepted 2026-03-31 20:51:06,046 - lava_callback - INFO - Starting async job submit for node 69c990a5f2866272ea51ecd4 2026-03-31 20:51:06,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:51:06,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c990a5f2866272ea51ecd4 HTTP/1.1" 200 None 2026-03-31 20:51:06,062 - lava_callback - DEBUG - Processing node 69c990a5f2866272ea51ecd4: baseline-arm-baylibre 2026-03-31 20:51:06,062 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c990a5f2866272ea51ecd4 2026-03-31 20:51:06,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:51:06,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c990a5f2866272ea51ecd4 HTTP/1.1" 200 None 2026-03-31 20:51:06,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:51:06,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:51:06,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:51:06,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c990a5f2866272ea51ecd4 HTTP/1.1" 200 None 2026-03-31 20:51:06,135 - lava_callback - ERROR - Error processing callback for node 69c990a5f2866272ea51ecd4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39660 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:51:26,019 - lava_callback - INFO - Received callback for node 69c990caf2866272ea51ece2 INFO: 172.19.0.1:51420 - "POST /node/69c990caf2866272ea51ece2 HTTP/1.0" 202 Accepted 2026-03-31 20:51:26,024 - lava_callback - INFO - Starting async job submit for node 69c990caf2866272ea51ece2 2026-03-31 20:51:26,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:51:26,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c990caf2866272ea51ece2 HTTP/1.1" 200 None 2026-03-31 20:51:26,041 - lava_callback - DEBUG - Processing node 69c990caf2866272ea51ece2: baseline-arm-baylibre 2026-03-31 20:51:26,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c990caf2866272ea51ece2 2026-03-31 20:51:26,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:51:26,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c990caf2866272ea51ece2 HTTP/1.1" 200 None 2026-03-31 20:51:26,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:51:26,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:51:26,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:51:26,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c990caf2866272ea51ece2 HTTP/1.1" 200 None 2026-03-31 20:51:26,113 - lava_callback - ERROR - Error processing callback for node 69c990caf2866272ea51ece2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48240 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:52:06,045 - lava_callback - INFO - Received callback for node 69c990eef2866272ea51ecec INFO: 172.19.0.1:47776 - "POST /node/69c990eef2866272ea51ecec HTTP/1.0" 202 Accepted 2026-03-31 20:52:06,050 - lava_callback - INFO - Starting async job submit for node 69c990eef2866272ea51ecec 2026-03-31 20:52:06,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:52:06,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c990eef2866272ea51ecec HTTP/1.1" 200 None 2026-03-31 20:52:06,073 - lava_callback - DEBUG - Processing node 69c990eef2866272ea51ecec: baseline-arm-baylibre 2026-03-31 20:52:06,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c990eef2866272ea51ecec 2026-03-31 20:52:06,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:52:06,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c990eef2866272ea51ecec HTTP/1.1" 200 None 2026-03-31 20:52:06,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:52:06,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:52:06,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:52:06,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c990eef2866272ea51ecec HTTP/1.1" 200 None 2026-03-31 20:52:06,137 - lava_callback - ERROR - Error processing callback for node 69c990eef2866272ea51ecec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42766 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:52:57,703 - lava_callback - INFO - Received callback for node 69c99115f2866272ea51ecff INFO: 172.19.0.1:42772 - "POST /node/69c99115f2866272ea51ecff HTTP/1.0" 202 Accepted 2026-03-31 20:52:57,709 - lava_callback - INFO - Starting async job submit for node 69c99115f2866272ea51ecff 2026-03-31 20:52:57,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:52:57,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99115f2866272ea51ecff HTTP/1.1" 200 None 2026-03-31 20:52:57,725 - lava_callback - DEBUG - Processing node 69c99115f2866272ea51ecff: baseline-arm-clabbe 2026-03-31 20:52:57,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99115f2866272ea51ecff 2026-03-31 20:52:57,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:52:57,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99115f2866272ea51ecff HTTP/1.1" 200 None 2026-03-31 20:52:57,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:52:57,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:52:57,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:52:57,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99115f2866272ea51ecff HTTP/1.1" 200 None 2026-03-31 20:52:57,790 - lava_callback - ERROR - Error processing callback for node 69c99115f2866272ea51ecff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52124 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:53:26,252 - lava_callback - INFO - Received callback for node 69c99138f2866272ea51ed07 INFO: 172.19.0.1:56750 - "POST /node/69c99138f2866272ea51ed07 HTTP/1.0" 202 Accepted 2026-03-31 20:53:26,261 - lava_callback - INFO - Starting async job submit for node 69c99138f2866272ea51ed07 2026-03-31 20:53:26,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:53:26,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99138f2866272ea51ed07 HTTP/1.1" 200 None 2026-03-31 20:53:26,284 - lava_callback - DEBUG - Processing node 69c99138f2866272ea51ed07: baseline-arm-baylibre 2026-03-31 20:53:26,284 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99138f2866272ea51ed07 2026-03-31 20:53:26,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:53:26,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99138f2866272ea51ed07 HTTP/1.1" 200 None 2026-03-31 20:53:26,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:53:26,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:53:26,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:53:26,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99138f2866272ea51ed07 HTTP/1.1" 200 None 2026-03-31 20:53:26,368 - lava_callback - ERROR - Error processing callback for node 69c99138f2866272ea51ed07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34878 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53922 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:53:57,771 - lava_callback - INFO - Received callback for node 69c9915ef2866272ea51ed17 INFO: 172.19.0.1:53926 - "POST /node/69c9915ef2866272ea51ed17 HTTP/1.0" 202 Accepted 2026-03-31 20:53:57,777 - lava_callback - INFO - Starting async job submit for node 69c9915ef2866272ea51ed17 2026-03-31 20:53:57,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:53:57,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9915ef2866272ea51ed17 HTTP/1.1" 200 None 2026-03-31 20:53:57,794 - lava_callback - DEBUG - Processing node 69c9915ef2866272ea51ed17: baseline-arm-clabbe 2026-03-31 20:53:57,794 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9915ef2866272ea51ed17 2026-03-31 20:53:57,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:53:57,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9915ef2866272ea51ed17 HTTP/1.1" 200 None 2026-03-31 20:53:57,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:53:57,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:53:57,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:53:57,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9915ef2866272ea51ed17 HTTP/1.1" 200 None 2026-03-31 20:53:57,848 - lava_callback - ERROR - Error processing callback for node 69c9915ef2866272ea51ed17: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44748 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:54:46,217 - lava_callback - INFO - Received callback for node 69c99181f2866272ea51ed22 INFO: 172.19.0.1:47460 - "POST /node/69c99181f2866272ea51ed22 HTTP/1.0" 202 Accepted 2026-03-31 20:54:46,223 - lava_callback - INFO - Starting async job submit for node 69c99181f2866272ea51ed22 2026-03-31 20:54:46,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:54:46,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99181f2866272ea51ed22 HTTP/1.1" 200 None 2026-03-31 20:54:46,245 - lava_callback - DEBUG - Processing node 69c99181f2866272ea51ed22: baseline-arm-baylibre 2026-03-31 20:54:46,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99181f2866272ea51ed22 2026-03-31 20:54:46,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:54:46,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99181f2866272ea51ed22 HTTP/1.1" 200 None 2026-03-31 20:54:46,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:54:46,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:54:46,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:54:46,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99181f2866272ea51ed22 HTTP/1.1" 200 None 2026-03-31 20:54:46,308 - lava_callback - ERROR - Error processing callback for node 69c99181f2866272ea51ed22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39030 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53250 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:55:17,725 - lava_callback - INFO - Received callback for node 69c991a7f2866272ea51ed33 INFO: 172.19.0.1:53260 - "POST /node/69c991a7f2866272ea51ed33 HTTP/1.0" 202 Accepted 2026-03-31 20:55:17,731 - lava_callback - INFO - Starting async job submit for node 69c991a7f2866272ea51ed33 2026-03-31 20:55:17,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:55:17,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c991a7f2866272ea51ed33 HTTP/1.1" 200 None 2026-03-31 20:55:17,748 - lava_callback - DEBUG - Processing node 69c991a7f2866272ea51ed33: baseline-arm-clabbe 2026-03-31 20:55:17,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c991a7f2866272ea51ed33 2026-03-31 20:55:17,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:55:17,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c991a7f2866272ea51ed33 HTTP/1.1" 200 None 2026-03-31 20:55:17,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:55:17,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:55:17,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:55:17,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c991a7f2866272ea51ed33 HTTP/1.1" 200 None 2026-03-31 20:55:17,801 - lava_callback - ERROR - Error processing callback for node 69c991a7f2866272ea51ed33: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53786 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:55:57,763 - lava_callback - INFO - Received callback for node 69c991cbf2866272ea51ed3f INFO: 172.19.0.1:53802 - "POST /node/69c991cbf2866272ea51ed3f HTTP/1.0" 202 Accepted 2026-03-31 20:55:57,769 - lava_callback - INFO - Starting async job submit for node 69c991cbf2866272ea51ed3f 2026-03-31 20:55:57,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:55:57,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c991cbf2866272ea51ed3f HTTP/1.1" 200 None 2026-03-31 20:55:57,786 - lava_callback - DEBUG - Processing node 69c991cbf2866272ea51ed3f: baseline-arm-clabbe 2026-03-31 20:55:57,786 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c991cbf2866272ea51ed3f 2026-03-31 20:55:57,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:55:57,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c991cbf2866272ea51ed3f HTTP/1.1" 200 None 2026-03-31 20:55:57,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:55:57,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:55:57,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:55:57,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c991cbf2866272ea51ed3f HTTP/1.1" 200 None 2026-03-31 20:55:57,858 - lava_callback - ERROR - Error processing callback for node 69c991cbf2866272ea51ed3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:32916 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:56:26,342 - lava_callback - INFO - Received callback for node 69c991eaf2866272ea51ed49 INFO: 172.19.0.1:43490 - "POST /node/69c991eaf2866272ea51ed49 HTTP/1.0" 202 Accepted 2026-03-31 20:56:26,350 - lava_callback - INFO - Starting async job submit for node 69c991eaf2866272ea51ed49 2026-03-31 20:56:26,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:56:26,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c991eaf2866272ea51ed49 HTTP/1.1" 200 None 2026-03-31 20:56:26,369 - lava_callback - DEBUG - Processing node 69c991eaf2866272ea51ed49: baseline-arm-baylibre 2026-03-31 20:56:26,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c991eaf2866272ea51ed49 2026-03-31 20:56:26,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:56:26,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c991eaf2866272ea51ed49 HTTP/1.1" 200 None 2026-03-31 20:56:26,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:56:26,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:56:26,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:56:26,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c991eaf2866272ea51ed49 HTTP/1.1" 200 None 2026-03-31 20:56:26,433 - lava_callback - ERROR - Error processing callback for node 69c991eaf2866272ea51ed49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35722 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:56:57,993 - lava_callback - INFO - Received callback for node 69c99214f2866272ea51ed58 INFO: 172.19.0.1:35732 - "POST /node/69c99214f2866272ea51ed58 HTTP/1.0" 202 Accepted 2026-03-31 20:56:58,002 - lava_callback - INFO - Starting async job submit for node 69c99214f2866272ea51ed58 2026-03-31 20:56:58,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:35744 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:56:58,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99214f2866272ea51ed58 HTTP/1.1" 200 None 2026-03-31 20:56:58,018 - lava_callback - DEBUG - Processing node 69c99214f2866272ea51ed58: baseline-arm-clabbe 2026-03-31 20:56:58,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99214f2866272ea51ed58 2026-03-31 20:56:58,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:56:58,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99214f2866272ea51ed58 HTTP/1.1" 200 None 2026-03-31 20:56:58,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:56:58,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:56:58,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:56:58,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99214f2866272ea51ed58 HTTP/1.1" 200 None 2026-03-31 20:56:58,092 - lava_callback - ERROR - Error processing callback for node 69c99214f2866272ea51ed58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42926 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:57:37,818 - lava_callback - INFO - Received callback for node 69c99238f2866272ea51ed63 INFO: 172.19.0.1:42930 - "POST /node/69c99238f2866272ea51ed63 HTTP/1.0" 202 Accepted 2026-03-31 20:57:37,823 - lava_callback - INFO - Starting async job submit for node 69c99238f2866272ea51ed63 2026-03-31 20:57:37,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:57:37,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99238f2866272ea51ed63 HTTP/1.1" 200 None 2026-03-31 20:57:37,840 - lava_callback - DEBUG - Processing node 69c99238f2866272ea51ed63: baseline-arm-clabbe 2026-03-31 20:57:37,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99238f2866272ea51ed63 2026-03-31 20:57:37,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:57:37,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99238f2866272ea51ed63 HTTP/1.1" 200 None 2026-03-31 20:57:37,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:57:37,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:57:37,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:57:37,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99238f2866272ea51ed63 HTTP/1.1" 200 None 2026-03-31 20:57:37,893 - lava_callback - ERROR - Error processing callback for node 69c99238f2866272ea51ed63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56592 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:58:06,521 - lava_callback - INFO - Received callback for node 69c99259f2866272ea51ed6e INFO: 172.19.0.1:56608 - "POST /node/69c99259f2866272ea51ed6e HTTP/1.0" 202 Accepted 2026-03-31 20:58:06,527 - lava_callback - INFO - Starting async job submit for node 69c99259f2866272ea51ed6e 2026-03-31 20:58:06,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:58:06,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99259f2866272ea51ed6e HTTP/1.1" 200 None 2026-03-31 20:58:06,543 - lava_callback - DEBUG - Processing node 69c99259f2866272ea51ed6e: baseline-arm-baylibre 2026-03-31 20:58:06,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99259f2866272ea51ed6e 2026-03-31 20:58:06,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:58:06,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99259f2866272ea51ed6e HTTP/1.1" 200 None 2026-03-31 20:58:06,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:58:06,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:58:06,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:58:06,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99259f2866272ea51ed6e HTTP/1.1" 200 None 2026-03-31 20:58:06,599 - lava_callback - ERROR - Error processing callback for node 69c99259f2866272ea51ed6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54854 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:58:46,414 - lava_callback - INFO - Received callback for node 69c9927ef2866272ea51ed7b INFO: 172.19.0.1:51490 - "POST /node/69c9927ef2866272ea51ed7b HTTP/1.0" 202 Accepted 2026-03-31 20:58:46,419 - lava_callback - INFO - Starting async job submit for node 69c9927ef2866272ea51ed7b 2026-03-31 20:58:46,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:58:46,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9927ef2866272ea51ed7b HTTP/1.1" 200 None 2026-03-31 20:58:46,436 - lava_callback - DEBUG - Processing node 69c9927ef2866272ea51ed7b: baseline-arm-baylibre 2026-03-31 20:58:46,436 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9927ef2866272ea51ed7b 2026-03-31 20:58:46,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:58:46,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9927ef2866272ea51ed7b HTTP/1.1" 200 None 2026-03-31 20:58:46,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:58:46,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:58:46,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:58:46,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9927ef2866272ea51ed7b HTTP/1.1" 200 None 2026-03-31 20:58:46,487 - lava_callback - ERROR - Error processing callback for node 69c9927ef2866272ea51ed7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56388 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:59:37,971 - lava_callback - INFO - Received callback for node 69c992a6f2866272ea51ed84 INFO: 172.19.0.1:56404 - "POST /node/69c992a6f2866272ea51ed84 HTTP/1.0" 202 Accepted 2026-03-31 20:59:37,977 - lava_callback - INFO - Starting async job submit for node 69c992a6f2866272ea51ed84 2026-03-31 20:59:37,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:59:37,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c992a6f2866272ea51ed84 HTTP/1.1" 200 None 2026-03-31 20:59:37,992 - lava_callback - DEBUG - Processing node 69c992a6f2866272ea51ed84: baseline-arm-clabbe 2026-03-31 20:59:37,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c992a6f2866272ea51ed84 2026-03-31 20:59:37,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:59:38,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c992a6f2866272ea51ed84 HTTP/1.1" 200 None 2026-03-31 20:59:38,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:59:38,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:59:38,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:59:38,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c992a6f2866272ea51ed84 HTTP/1.1" 200 None 2026-03-31 20:59:38,072 - lava_callback - ERROR - Error processing callback for node 69c992a6f2866272ea51ed84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55076 - "GET / HTTP/1.0" 200 OK 2026-03-31 20:59:57,909 - lava_callback - INFO - Received callback for node 69c992caf2866272ea51ed90 INFO: 172.19.0.1:55092 - "POST /node/69c992caf2866272ea51ed90 HTTP/1.0" 202 Accepted 2026-03-31 20:59:57,915 - lava_callback - INFO - Starting async job submit for node 69c992caf2866272ea51ed90 2026-03-31 20:59:57,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:59:57,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c992caf2866272ea51ed90 HTTP/1.1" 200 None 2026-03-31 20:59:57,931 - lava_callback - DEBUG - Processing node 69c992caf2866272ea51ed90: baseline-arm-clabbe 2026-03-31 20:59:57,932 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c992caf2866272ea51ed90 2026-03-31 20:59:57,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:59:57,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c992caf2866272ea51ed90 HTTP/1.1" 200 None 2026-03-31 20:59:57,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:59:57,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 20:59:57,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 20:59:57,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c992caf2866272ea51ed90 HTTP/1.1" 200 None 2026-03-31 20:59:57,989 - lava_callback - ERROR - Error processing callback for node 69c992caf2866272ea51ed90: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54682 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:00:37,998 - lava_callback - INFO - Received callback for node 69c992eff2866272ea51ed9c INFO: 172.19.0.1:54686 - "POST /node/69c992eff2866272ea51ed9c HTTP/1.0" 202 Accepted 2026-03-31 21:00:38,005 - lava_callback - INFO - Starting async job submit for node 69c992eff2866272ea51ed9c 2026-03-31 21:00:38,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:00:38,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c992eff2866272ea51ed9c HTTP/1.1" 200 None 2026-03-31 21:00:38,021 - lava_callback - DEBUG - Processing node 69c992eff2866272ea51ed9c: baseline-arm-clabbe 2026-03-31 21:00:38,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c992eff2866272ea51ed9c 2026-03-31 21:00:38,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:00:38,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c992eff2866272ea51ed9c HTTP/1.1" 200 None 2026-03-31 21:00:38,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:00:38,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 21:00:38,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:00:38,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c992eff2866272ea51ed9c HTTP/1.1" 200 None 2026-03-31 21:00:38,538 - lava_callback - ERROR - Error processing callback for node 69c992eff2866272ea51ed9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41222 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48584 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:01:06,571 - lava_callback - INFO - Received callback for node 69c99310f2866272ea51eda5 INFO: 172.19.0.1:48594 - "POST /node/69c99310f2866272ea51eda5 HTTP/1.0" 202 Accepted 2026-03-31 21:01:06,576 - lava_callback - INFO - Starting async job submit for node 69c99310f2866272ea51eda5 2026-03-31 21:01:06,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:01:06,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99310f2866272ea51eda5 HTTP/1.1" 200 None 2026-03-31 21:01:06,601 - lava_callback - DEBUG - Processing node 69c99310f2866272ea51eda5: baseline-arm-baylibre 2026-03-31 21:01:06,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99310f2866272ea51eda5 2026-03-31 21:01:06,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:01:06,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99310f2866272ea51eda5 HTTP/1.1" 200 None 2026-03-31 21:01:06,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:01:06,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 21:01:06,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:01:06,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99310f2866272ea51eda5 HTTP/1.1" 200 None 2026-03-31 21:01:06,655 - lava_callback - ERROR - Error processing callback for node 69c99310f2866272ea51eda5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46642 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:01:58,130 - lava_callback - INFO - Received callback for node 69c99338f2866272ea51edb4 INFO: 172.19.0.1:46658 - "POST /node/69c99338f2866272ea51edb4 HTTP/1.0" 202 Accepted 2026-03-31 21:01:58,136 - lava_callback - INFO - Starting async job submit for node 69c99338f2866272ea51edb4 2026-03-31 21:01:58,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:01:58,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99338f2866272ea51edb4 HTTP/1.1" 200 None 2026-03-31 21:01:58,153 - lava_callback - DEBUG - Processing node 69c99338f2866272ea51edb4: baseline-arm-clabbe 2026-03-31 21:01:58,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99338f2866272ea51edb4 2026-03-31 21:01:58,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:01:58,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99338f2866272ea51edb4 HTTP/1.1" 200 None 2026-03-31 21:01:58,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:01:58,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 21:01:58,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:01:58,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99338f2866272ea51edb4 HTTP/1.1" 200 None 2026-03-31 21:01:58,218 - lava_callback - ERROR - Error processing callback for node 69c99338f2866272ea51edb4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41364 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40108 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:02:26,674 - lava_callback - INFO - Received callback for node 69c9935bf2866272ea51edbe INFO: 172.19.0.1:40110 - "POST /node/69c9935bf2866272ea51edbe HTTP/1.0" 202 Accepted 2026-03-31 21:02:26,680 - lava_callback - INFO - Starting async job submit for node 69c9935bf2866272ea51edbe 2026-03-31 21:02:26,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:02:26,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9935bf2866272ea51edbe HTTP/1.1" 200 None 2026-03-31 21:02:26,696 - lava_callback - DEBUG - Processing node 69c9935bf2866272ea51edbe: baseline-arm-baylibre 2026-03-31 21:02:26,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9935bf2866272ea51edbe 2026-03-31 21:02:26,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:02:26,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9935bf2866272ea51edbe HTTP/1.1" 200 None 2026-03-31 21:02:26,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:02:26,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 21:02:26,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:02:26,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9935bf2866272ea51edbe HTTP/1.1" 200 None 2026-03-31 21:02:26,761 - lava_callback - ERROR - Error processing callback for node 69c9935bf2866272ea51edbe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46830 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:03:06,676 - lava_callback - INFO - Received callback for node 69c99380f2866272ea51edca INFO: 172.19.0.1:46836 - "POST /node/69c99380f2866272ea51edca HTTP/1.0" 202 Accepted 2026-03-31 21:03:06,684 - lava_callback - INFO - Starting async job submit for node 69c99380f2866272ea51edca 2026-03-31 21:03:06,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:03:06,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99380f2866272ea51edca HTTP/1.1" 200 None 2026-03-31 21:03:06,707 - lava_callback - DEBUG - Processing node 69c99380f2866272ea51edca: baseline-arm-baylibre 2026-03-31 21:03:06,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99380f2866272ea51edca 2026-03-31 21:03:06,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:03:06,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99380f2866272ea51edca HTTP/1.1" 200 None 2026-03-31 21:03:06,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:03:06,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 21:03:06,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:03:06,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99380f2866272ea51edca HTTP/1.1" 200 None 2026-03-31 21:03:06,788 - lava_callback - ERROR - Error processing callback for node 69c99380f2866272ea51edca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37436 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:03:58,165 - lava_callback - INFO - Received callback for node 69c993a8f2866272ea51eddb INFO: 172.19.0.1:37440 - "POST /node/69c993a8f2866272ea51eddb HTTP/1.0" 202 Accepted 2026-03-31 21:03:58,171 - lava_callback - INFO - Starting async job submit for node 69c993a8f2866272ea51eddb 2026-03-31 21:03:58,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:03:58,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c993a8f2866272ea51eddb HTTP/1.1" 200 None 2026-03-31 21:03:58,187 - lava_callback - DEBUG - Processing node 69c993a8f2866272ea51eddb: baseline-arm-clabbe 2026-03-31 21:03:58,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c993a8f2866272ea51eddb 2026-03-31 21:03:58,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:03:58,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c993a8f2866272ea51eddb HTTP/1.1" 200 None 2026-03-31 21:03:58,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:03:58,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 21:03:58,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:03:58,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c993a8f2866272ea51eddb HTTP/1.1" 200 None 2026-03-31 21:03:58,243 - lava_callback - ERROR - Error processing callback for node 69c993a8f2866272ea51eddb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34510 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:04:38,111 - lava_callback - INFO - Received callback for node 69c993d2f2866272ea51ede4 INFO: 172.19.0.1:34524 - "POST /node/69c993d2f2866272ea51ede4 HTTP/1.0" 202 Accepted 2026-03-31 21:04:38,117 - lava_callback - INFO - Starting async job submit for node 69c993d2f2866272ea51ede4 2026-03-31 21:04:38,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:04:38,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c993d2f2866272ea51ede4 HTTP/1.1" 200 None 2026-03-31 21:04:38,133 - lava_callback - DEBUG - Processing node 69c993d2f2866272ea51ede4: baseline-arm-clabbe 2026-03-31 21:04:38,134 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c993d2f2866272ea51ede4 2026-03-31 21:04:38,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:04:38,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c993d2f2866272ea51ede4 HTTP/1.1" 200 None 2026-03-31 21:04:38,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:04:38,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 21:04:38,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:04:38,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c993d2f2866272ea51ede4 HTTP/1.1" 200 None 2026-03-31 21:04:38,191 - lava_callback - ERROR - Error processing callback for node 69c993d2f2866272ea51ede4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45710 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:05:18,210 - lava_callback - INFO - Received callback for node 69c993faf2866272ea51edf4 INFO: 172.19.0.1:45722 - "POST /node/69c993faf2866272ea51edf4 HTTP/1.0" 202 Accepted 2026-03-31 21:05:18,219 - lava_callback - INFO - Starting async job submit for node 69c993faf2866272ea51edf4 2026-03-31 21:05:18,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:05:18,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c993faf2866272ea51edf4 HTTP/1.1" 200 None 2026-03-31 21:05:18,243 - lava_callback - DEBUG - Processing node 69c993faf2866272ea51edf4: baseline-arm-clabbe 2026-03-31 21:05:18,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c993faf2866272ea51edf4 2026-03-31 21:05:18,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:05:18,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c993faf2866272ea51edf4 HTTP/1.1" 200 None 2026-03-31 21:05:18,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:05:18,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 21:05:18,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:05:18,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c993faf2866272ea51edf4 HTTP/1.1" 200 None 2026-03-31 21:05:18,320 - lava_callback - ERROR - Error processing callback for node 69c993faf2866272ea51edf4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43562 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:05:58,224 - lava_callback - INFO - Received callback for node 69c99423f2866272ea51edff INFO: 172.19.0.1:43564 - "POST /node/69c99423f2866272ea51edff HTTP/1.0" 202 Accepted 2026-03-31 21:05:58,230 - lava_callback - INFO - Starting async job submit for node 69c99423f2866272ea51edff 2026-03-31 21:05:58,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:05:58,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99423f2866272ea51edff HTTP/1.1" 200 None 2026-03-31 21:05:58,245 - lava_callback - DEBUG - Processing node 69c99423f2866272ea51edff: baseline-arm-clabbe 2026-03-31 21:05:58,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99423f2866272ea51edff 2026-03-31 21:05:58,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:05:58,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99423f2866272ea51edff HTTP/1.1" 200 None 2026-03-31 21:05:58,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:05:58,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 21:05:58,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:05:58,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99423f2866272ea51edff HTTP/1.1" 200 None 2026-03-31 21:05:58,304 - lava_callback - ERROR - Error processing callback for node 69c99423f2866272ea51edff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37106 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:06:38,316 - lava_callback - INFO - Received callback for node 69c9944bf2866272ea51ee09 INFO: 172.19.0.1:37112 - "POST /node/69c9944bf2866272ea51ee09 HTTP/1.0" 202 Accepted 2026-03-31 21:06:38,322 - lava_callback - INFO - Starting async job submit for node 69c9944bf2866272ea51ee09 2026-03-31 21:06:38,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:06:38,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9944bf2866272ea51ee09 HTTP/1.1" 200 None 2026-03-31 21:06:38,338 - lava_callback - DEBUG - Processing node 69c9944bf2866272ea51ee09: baseline-arm-clabbe 2026-03-31 21:06:38,338 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9944bf2866272ea51ee09 2026-03-31 21:06:38,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:06:38,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9944bf2866272ea51ee09 HTTP/1.1" 200 None 2026-03-31 21:06:38,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:06:38,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-31 21:06:38,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:06:38,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9944bf2866272ea51ee09 HTTP/1.1" 200 None 2026-03-31 21:06:38,399 - lava_callback - ERROR - Error processing callback for node 69c9944bf2866272ea51ee09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59612 - "GET /node/69c16cc62b2b51e9ad97a0ef%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:59628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42126 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38914 - "GET /node/6991163985f550420785e3fb%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:38928 - "GET /node/693f4ba28b4d8f471e43d194%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:40414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52090 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45830 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:12:47,257 - lava_callback - INFO - Received callback for node 69c995c0f2866272ea51ee4a INFO: 172.19.0.1:45840 - "POST /node/69c995c0f2866272ea51ee4a HTTP/1.0" 202 Accepted 2026-03-31 21:12:47,263 - lava_callback - INFO - Starting async job submit for node 69c995c0f2866272ea51ee4a 2026-03-31 21:12:47,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:12:47,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c995c0f2866272ea51ee4a HTTP/1.1" 200 None 2026-03-31 21:12:47,280 - lava_callback - DEBUG - Processing node 69c995c0f2866272ea51ee4a: baseline-arm64-baylibre 2026-03-31 21:12:47,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c995c0f2866272ea51ee4a 2026-03-31 21:12:47,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:12:47,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c995c0f2866272ea51ee4a HTTP/1.1" 200 None 2026-03-31 21:12:47,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:12:47,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:12:47,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:12:47,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c995c0f2866272ea51ee4a HTTP/1.1" 200 None 2026-03-31 21:12:47,342 - lava_callback - ERROR - Error processing callback for node 69c995c0f2866272ea51ee4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35078 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53980 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:14:07,334 - lava_callback - INFO - Received callback for node 69c99613f2866272ea51ee51 INFO: 172.19.0.1:53990 - "POST /node/69c99613f2866272ea51ee51 HTTP/1.0" 202 Accepted 2026-03-31 21:14:07,339 - lava_callback - INFO - Starting async job submit for node 69c99613f2866272ea51ee51 2026-03-31 21:14:07,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:14:07,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99613f2866272ea51ee51 HTTP/1.1" 200 None 2026-03-31 21:14:07,357 - lava_callback - DEBUG - Processing node 69c99613f2866272ea51ee51: baseline-arm64-baylibre 2026-03-31 21:14:07,357 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99613f2866272ea51ee51 2026-03-31 21:14:07,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:14:07,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99613f2866272ea51ee51 HTTP/1.1" 200 None 2026-03-31 21:14:07,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:14:07,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:14:07,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:14:07,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99613f2866272ea51ee51 HTTP/1.1" 200 None 2026-03-31 21:14:07,418 - lava_callback - ERROR - Error processing callback for node 69c99613f2866272ea51ee51: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39304 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:15:27,394 - lava_callback - INFO - Received callback for node 69c99664f2866272ea51ee5b INFO: 172.19.0.1:39312 - "POST /node/69c99664f2866272ea51ee5b HTTP/1.0" 202 Accepted 2026-03-31 21:15:27,400 - lava_callback - INFO - Starting async job submit for node 69c99664f2866272ea51ee5b 2026-03-31 21:15:27,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:15:27,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99664f2866272ea51ee5b HTTP/1.1" 200 None 2026-03-31 21:15:27,415 - lava_callback - DEBUG - Processing node 69c99664f2866272ea51ee5b: baseline-arm64-baylibre 2026-03-31 21:15:27,416 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99664f2866272ea51ee5b 2026-03-31 21:15:27,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:15:27,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99664f2866272ea51ee5b HTTP/1.1" 200 None 2026-03-31 21:15:27,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:15:27,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:15:27,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:15:27,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99664f2866272ea51ee5b HTTP/1.1" 200 None 2026-03-31 21:15:27,480 - lava_callback - ERROR - Error processing callback for node 69c99664f2866272ea51ee5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54252 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:16:47,465 - lava_callback - INFO - Received callback for node 69c996b6f2866272ea51ee6d INFO: 172.19.0.1:54258 - "POST /node/69c996b6f2866272ea51ee6d HTTP/1.0" 202 Accepted 2026-03-31 21:16:47,471 - lava_callback - INFO - Starting async job submit for node 69c996b6f2866272ea51ee6d 2026-03-31 21:16:47,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:16:47,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c996b6f2866272ea51ee6d HTTP/1.1" 200 None 2026-03-31 21:16:47,491 - lava_callback - DEBUG - Processing node 69c996b6f2866272ea51ee6d: baseline-arm64-baylibre 2026-03-31 21:16:47,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c996b6f2866272ea51ee6d 2026-03-31 21:16:47,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:16:47,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c996b6f2866272ea51ee6d HTTP/1.1" 200 None 2026-03-31 21:16:47,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:16:47,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:16:47,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:16:47,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c996b6f2866272ea51ee6d HTTP/1.1" 200 None 2026-03-31 21:16:47,547 - lava_callback - ERROR - Error processing callback for node 69c996b6f2866272ea51ee6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55188 - "GET /node/69aefa1fce46d8ec186abf59%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:44498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40220 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44572 - "GET /node/69a3fd266ebb1cc56c1def48%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:44574 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:18:07,644 - lava_callback - INFO - Received callback for node 69c9970af2866272ea51ee7f INFO: 172.19.0.1:44590 - "POST /node/69c9970af2866272ea51ee7f HTTP/1.0" 202 Accepted 2026-03-31 21:18:07,649 - lava_callback - INFO - Starting async job submit for node 69c9970af2866272ea51ee7f 2026-03-31 21:18:07,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:18:07,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9970af2866272ea51ee7f HTTP/1.1" 200 None 2026-03-31 21:18:07,665 - lava_callback - DEBUG - Processing node 69c9970af2866272ea51ee7f: baseline-arm64-baylibre 2026-03-31 21:18:07,666 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9970af2866272ea51ee7f 2026-03-31 21:18:07,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:18:07,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9970af2866272ea51ee7f HTTP/1.1" 200 None 2026-03-31 21:18:07,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:18:07,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:18:07,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:18:07,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9970af2866272ea51ee7f HTTP/1.1" 200 None 2026-03-31 21:18:07,718 - lava_callback - ERROR - Error processing callback for node 69c9970af2866272ea51ee7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51910 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52552 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:19:47,619 - lava_callback - INFO - Received callback for node 69c9975ff2866272ea51ee88 INFO: 172.19.0.1:52556 - "POST /node/69c9975ff2866272ea51ee88 HTTP/1.0" 202 Accepted 2026-03-31 21:19:47,624 - lava_callback - INFO - Starting async job submit for node 69c9975ff2866272ea51ee88 2026-03-31 21:19:47,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:19:47,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9975ff2866272ea51ee88 HTTP/1.1" 200 None 2026-03-31 21:19:47,640 - lava_callback - DEBUG - Processing node 69c9975ff2866272ea51ee88: baseline-arm64-baylibre 2026-03-31 21:19:47,640 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9975ff2866272ea51ee88 2026-03-31 21:19:47,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:19:47,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9975ff2866272ea51ee88 HTTP/1.1" 200 None 2026-03-31 21:19:47,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:19:47,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:19:47,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:19:47,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9975ff2866272ea51ee88 HTTP/1.1" 200 None 2026-03-31 21:19:47,700 - lava_callback - ERROR - Error processing callback for node 69c9975ff2866272ea51ee88: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47582 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51260 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58892 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:21:19,128 - lava_callback - INFO - Received callback for node 69c997bcf2866272ea51ee96 INFO: 172.19.0.1:58896 - "POST /node/69c997bcf2866272ea51ee96 HTTP/1.0" 202 Accepted 2026-03-31 21:21:19,134 - lava_callback - INFO - Starting async job submit for node 69c997bcf2866272ea51ee96 2026-03-31 21:21:19,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:21:19,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c997bcf2866272ea51ee96 HTTP/1.1" 200 None 2026-03-31 21:21:19,149 - lava_callback - DEBUG - Processing node 69c997bcf2866272ea51ee96: baseline-arm64-clabbe 2026-03-31 21:21:19,149 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c997bcf2866272ea51ee96 2026-03-31 21:21:19,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:21:19,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c997bcf2866272ea51ee96 HTTP/1.1" 200 None 2026-03-31 21:21:19,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:21:19,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:21:19,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:21:19,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c997bcf2866272ea51ee96 HTTP/1.1" 200 None 2026-03-31 21:21:19,217 - lava_callback - ERROR - Error processing callback for node 69c997bcf2866272ea51ee96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36154 - "GET /node/694ab00b83c67c43d1ede84a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:36158 - "GET /node/694bd897c19e09af58d497f5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53702 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:22:38,936 - lava_callback - INFO - Received callback for node 69c99818f2866272ea51eeaa INFO: 172.19.0.1:53704 - "POST /node/69c99818f2866272ea51eeaa HTTP/1.0" 202 Accepted 2026-03-31 21:22:38,941 - lava_callback - INFO - Starting async job submit for node 69c99818f2866272ea51eeaa 2026-03-31 21:22:38,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:22:38,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99818f2866272ea51eeaa HTTP/1.1" 200 None 2026-03-31 21:22:38,957 - lava_callback - DEBUG - Processing node 69c99818f2866272ea51eeaa: baseline-arm64-clabbe 2026-03-31 21:22:38,957 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99818f2866272ea51eeaa 2026-03-31 21:22:38,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:22:38,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99818f2866272ea51eeaa HTTP/1.1" 200 None 2026-03-31 21:22:38,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:22:38,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:22:38,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:22:39,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99818f2866272ea51eeaa HTTP/1.1" 200 None 2026-03-31 21:22:39,015 - lava_callback - ERROR - Error processing callback for node 69c99818f2866272ea51eeaa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38370 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38640 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41260 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:23:58,942 - lava_callback - INFO - Received callback for node 69c9986cf2866272ea51eeb0 INFO: 172.19.0.1:41264 - "POST /node/69c9986cf2866272ea51eeb0 HTTP/1.0" 202 Accepted 2026-03-31 21:23:58,948 - lava_callback - INFO - Starting async job submit for node 69c9986cf2866272ea51eeb0 2026-03-31 21:23:58,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:23:58,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9986cf2866272ea51eeb0 HTTP/1.1" 200 None 2026-03-31 21:23:58,970 - lava_callback - DEBUG - Processing node 69c9986cf2866272ea51eeb0: baseline-arm64-clabbe 2026-03-31 21:23:58,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9986cf2866272ea51eeb0 2026-03-31 21:23:58,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:23:58,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9986cf2866272ea51eeb0 HTTP/1.1" 200 None 2026-03-31 21:23:58,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:23:59,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:23:59,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:23:59,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9986cf2866272ea51eeb0 HTTP/1.1" 200 None 2026-03-31 21:23:59,049 - lava_callback - ERROR - Error processing callback for node 69c9986cf2866272ea51eeb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:43216 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56488 - "GET /node/69847c123a8748f2820874c2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:56496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47102 - "GET /node/69c7337229015d0f9599264c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47120 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55392 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:25:28,053 - lava_callback - INFO - Received callback for node 69c998b8f2866272ea51eebe INFO: 172.19.0.1:55398 - "POST /node/69c998b8f2866272ea51eebe HTTP/1.0" 202 Accepted 2026-03-31 21:25:28,058 - lava_callback - INFO - Starting async job submit for node 69c998b8f2866272ea51eebe 2026-03-31 21:25:28,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:25:28,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c998b8f2866272ea51eebe HTTP/1.1" 200 None 2026-03-31 21:25:28,085 - lava_callback - DEBUG - Processing node 69c998b8f2866272ea51eebe: baseline-arm64-baylibre 2026-03-31 21:25:28,085 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c998b8f2866272ea51eebe 2026-03-31 21:25:28,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:25:28,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c998b8f2866272ea51eebe HTTP/1.1" 200 None 2026-03-31 21:25:28,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:25:28,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:25:28,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:25:28,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c998b8f2866272ea51eebe HTTP/1.1" 200 None 2026-03-31 21:25:28,169 - lava_callback - ERROR - Error processing callback for node 69c998b8f2866272ea51eebe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33486 - "GET /node/693f80ee0d8ce4018da96771%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48122 - "GET /node/693fcf73575bc060ce14b3d9%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:48130 - "GET /node/69193e7fef36f328cfd42c1e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:48136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55562 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:26:59,119 - lava_callback - INFO - Received callback for node 69c99914f2866272ea51eece INFO: 172.19.0.1:55568 - "POST /node/69c99914f2866272ea51eece HTTP/1.0" 202 Accepted 2026-03-31 21:26:59,124 - lava_callback - INFO - Starting async job submit for node 69c99914f2866272ea51eece 2026-03-31 21:26:59,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:26:59,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99914f2866272ea51eece HTTP/1.1" 200 None 2026-03-31 21:26:59,140 - lava_callback - DEBUG - Processing node 69c99914f2866272ea51eece: baseline-arm64-clabbe 2026-03-31 21:26:59,141 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99914f2866272ea51eece 2026-03-31 21:26:59,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:26:59,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99914f2866272ea51eece HTTP/1.1" 200 None 2026-03-31 21:26:59,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:26:59,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:26:59,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:26:59,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99914f2866272ea51eece HTTP/1.1" 200 None 2026-03-31 21:26:59,191 - lava_callback - ERROR - Error processing callback for node 69c99914f2866272ea51eece: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41054 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45974 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42036 - "GET /node/69bdf35bd54b5122738c6970%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:42048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42052 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:28:19,176 - lava_callback - INFO - Received callback for node 69c99967f2866272ea51eede INFO: 172.19.0.1:42064 - "POST /node/69c99967f2866272ea51eede HTTP/1.0" 202 Accepted 2026-03-31 21:28:19,181 - lava_callback - INFO - Starting async job submit for node 69c99967f2866272ea51eede 2026-03-31 21:28:19,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:28:19,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99967f2866272ea51eede HTTP/1.1" 200 None 2026-03-31 21:28:19,197 - lava_callback - DEBUG - Processing node 69c99967f2866272ea51eede: baseline-arm64-clabbe 2026-03-31 21:28:19,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99967f2866272ea51eede 2026-03-31 21:28:19,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:28:19,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99967f2866272ea51eede HTTP/1.1" 200 None 2026-03-31 21:28:19,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:28:19,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:28:19,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:28:19,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99967f2866272ea51eede HTTP/1.1" 200 None 2026-03-31 21:28:19,249 - lava_callback - ERROR - Error processing callback for node 69c99967f2866272ea51eede: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57512 - "GET /node/69a7ef177c78d2166ff85a6f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:57520 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:29:28,198 - lava_callback - INFO - Received callback for node 69c999b2f2866272ea51eee7 INFO: 172.19.0.1:57530 - "POST /node/69c999b2f2866272ea51eee7 HTTP/1.0" 202 Accepted 2026-03-31 21:29:28,204 - lava_callback - INFO - Starting async job submit for node 69c999b2f2866272ea51eee7 2026-03-31 21:29:28,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:29:28,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c999b2f2866272ea51eee7 HTTP/1.1" 200 None 2026-03-31 21:29:28,220 - lava_callback - DEBUG - Processing node 69c999b2f2866272ea51eee7: baseline-arm64-baylibre 2026-03-31 21:29:28,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c999b2f2866272ea51eee7 2026-03-31 21:29:28,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:29:28,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c999b2f2866272ea51eee7 HTTP/1.1" 200 None 2026-03-31 21:29:28,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:29:28,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:29:28,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:29:28,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c999b2f2866272ea51eee7 HTTP/1.1" 200 None 2026-03-31 21:29:28,285 - lava_callback - ERROR - Error processing callback for node 69c999b2f2866272ea51eee7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44286 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:30:59,295 - lava_callback - INFO - Received callback for node 69c99a0ef2866272ea51eef7 INFO: 172.19.0.1:44296 - "POST /node/69c99a0ef2866272ea51eef7 HTTP/1.0" 202 Accepted 2026-03-31 21:30:59,301 - lava_callback - INFO - Starting async job submit for node 69c99a0ef2866272ea51eef7 2026-03-31 21:30:59,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:30:59,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99a0ef2866272ea51eef7 HTTP/1.1" 200 None 2026-03-31 21:30:59,330 - lava_callback - DEBUG - Processing node 69c99a0ef2866272ea51eef7: baseline-arm64-clabbe 2026-03-31 21:30:59,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99a0ef2866272ea51eef7 2026-03-31 21:30:59,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:30:59,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99a0ef2866272ea51eef7 HTTP/1.1" 200 None 2026-03-31 21:30:59,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:30:59,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8dff86a4c026e16347d45 HTTP/1.1" 200 None 2026-03-31 21:30:59,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:30:59,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99a0ef2866272ea51eef7 HTTP/1.1" 200 None 2026-03-31 21:30:59,389 - lava_callback - ERROR - Error processing callback for node 69c99a0ef2866272ea51eef7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55284 - "GET /node/699448545ae459b9df3dad4d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:43644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48296 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:32:28,366 - lava_callback - INFO - Received callback for node 69c99a60f2866272ea51ef05 INFO: 172.19.0.1:48312 - "POST /node/69c99a60f2866272ea51ef05 HTTP/1.0" 202 Accepted 2026-03-31 21:32:28,372 - lava_callback - INFO - Starting async job submit for node 69c99a60f2866272ea51ef05 2026-03-31 21:32:28,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:32:28,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99a60f2866272ea51ef05 HTTP/1.1" 200 None 2026-03-31 21:32:28,392 - lava_callback - DEBUG - Processing node 69c99a60f2866272ea51ef05: baseline-arm64-baylibre 2026-03-31 21:32:28,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99a60f2866272ea51ef05 2026-03-31 21:32:28,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:32:28,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99a60f2866272ea51ef05 HTTP/1.1" 200 None 2026-03-31 21:32:28,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:32:28,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:32:28,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:32:28,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99a60f2866272ea51ef05 HTTP/1.1" 200 None 2026-03-31 21:32:28,448 - lava_callback - ERROR - Error processing callback for node 69c99a60f2866272ea51ef05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43252 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50476 - "GET /node/694aac7083c67c43d1ede3fd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:50488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33508 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33810 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:33:48,418 - lava_callback - INFO - Received callback for node 69c99ab5f2866272ea51ef16 INFO: 172.19.0.1:33814 - "POST /node/69c99ab5f2866272ea51ef16 HTTP/1.0" 202 Accepted 2026-03-31 21:33:48,424 - lava_callback - INFO - Starting async job submit for node 69c99ab5f2866272ea51ef16 2026-03-31 21:33:48,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:33:48,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99ab5f2866272ea51ef16 HTTP/1.1" 200 None 2026-03-31 21:33:48,447 - lava_callback - DEBUG - Processing node 69c99ab5f2866272ea51ef16: baseline-arm64-baylibre 2026-03-31 21:33:48,447 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99ab5f2866272ea51ef16 2026-03-31 21:33:48,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:33:48,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99ab5f2866272ea51ef16 HTTP/1.1" 200 None 2026-03-31 21:33:48,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:33:48,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:33:48,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:33:48,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99ab5f2866272ea51ef16 HTTP/1.1" 200 None 2026-03-31 21:33:48,502 - lava_callback - ERROR - Error processing callback for node 69c99ab5f2866272ea51ef16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53356 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34278 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54442 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:35:19,462 - lava_callback - INFO - Received callback for node 69c99b10f2866272ea51ef25 INFO: 172.19.0.1:54454 - "POST /node/69c99b10f2866272ea51ef25 HTTP/1.0" 202 Accepted 2026-03-31 21:35:19,468 - lava_callback - INFO - Starting async job submit for node 69c99b10f2866272ea51ef25 2026-03-31 21:35:19,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:35:19,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99b10f2866272ea51ef25 HTTP/1.1" 200 None 2026-03-31 21:35:19,484 - lava_callback - DEBUG - Processing node 69c99b10f2866272ea51ef25: baseline-arm64-clabbe 2026-03-31 21:35:19,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99b10f2866272ea51ef25 2026-03-31 21:35:19,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:35:19,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99b10f2866272ea51ef25 HTTP/1.1" 200 None 2026-03-31 21:35:19,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:35:19,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:35:19,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:35:19,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99b10f2866272ea51ef25 HTTP/1.1" 200 None 2026-03-31 21:35:19,535 - lava_callback - ERROR - Error processing callback for node 69c99b10f2866272ea51ef25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44410 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36882 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:36:59,514 - lava_callback - INFO - Received callback for node 69c99b66f2866272ea51ef36 INFO: 172.19.0.1:36896 - "POST /node/69c99b66f2866272ea51ef36 HTTP/1.0" 202 Accepted 2026-03-31 21:36:59,519 - lava_callback - INFO - Starting async job submit for node 69c99b66f2866272ea51ef36 2026-03-31 21:36:59,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:36:59,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99b66f2866272ea51ef36 HTTP/1.1" 200 None 2026-03-31 21:36:59,535 - lava_callback - DEBUG - Processing node 69c99b66f2866272ea51ef36: baseline-arm64-clabbe 2026-03-31 21:36:59,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99b66f2866272ea51ef36 2026-03-31 21:36:59,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:36:59,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99b66f2866272ea51ef36 HTTP/1.1" 200 None 2026-03-31 21:36:59,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:36:59,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:36:59,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:36:59,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99b66f2866272ea51ef36 HTTP/1.1" 200 None 2026-03-31 21:36:59,592 - lava_callback - ERROR - Error processing callback for node 69c99b66f2866272ea51ef36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41280 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52194 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:38:19,612 - lava_callback - INFO - Received callback for node 69c99bbaf2866272ea51ef44 INFO: 172.19.0.1:52210 - "POST /node/69c99bbaf2866272ea51ef44 HTTP/1.0" 202 Accepted 2026-03-31 21:38:19,617 - lava_callback - INFO - Starting async job submit for node 69c99bbaf2866272ea51ef44 2026-03-31 21:38:19,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:38:19,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99bbaf2866272ea51ef44 HTTP/1.1" 200 None 2026-03-31 21:38:19,634 - lava_callback - DEBUG - Processing node 69c99bbaf2866272ea51ef44: baseline-arm64-clabbe 2026-03-31 21:38:19,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99bbaf2866272ea51ef44 2026-03-31 21:38:19,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:38:19,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99bbaf2866272ea51ef44 HTTP/1.1" 200 None 2026-03-31 21:38:19,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:38:19,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:38:19,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:38:19,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99bbaf2866272ea51ef44 HTTP/1.1" 200 None 2026-03-31 21:38:19,720 - lava_callback - ERROR - Error processing callback for node 69c99bbaf2866272ea51ef44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60700 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:41:19,943 - lava_callback - INFO - Received callback for node 69c99c7af2866272ea51ef5b INFO: 172.19.0.1:47740 - "POST /node/69c99c7af2866272ea51ef5b HTTP/1.0" 202 Accepted 2026-03-31 21:41:19,948 - lava_callback - INFO - Starting async job submit for node 69c99c7af2866272ea51ef5b 2026-03-31 21:41:19,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:41:19,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99c7af2866272ea51ef5b HTTP/1.1" 200 None 2026-03-31 21:41:19,965 - lava_callback - DEBUG - Processing node 69c99c7af2866272ea51ef5b: baseline-arm64-clabbe 2026-03-31 21:41:19,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99c7af2866272ea51ef5b 2026-03-31 21:41:19,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:41:19,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99c7af2866272ea51ef5b HTTP/1.1" 200 None 2026-03-31 21:41:19,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:41:19,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:41:19,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:41:20,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99c7af2866272ea51ef5b HTTP/1.1" 200 None 2026-03-31 21:41:20,029 - lava_callback - ERROR - Error processing callback for node 69c99c7af2866272ea51ef5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47744 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48412 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51086 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:42:48,912 - lava_callback - INFO - Received callback for node 69c99cc7f2866272ea51ef67 INFO: 172.19.0.1:51088 - "POST /node/69c99cc7f2866272ea51ef67 HTTP/1.0" 202 Accepted 2026-03-31 21:42:48,918 - lava_callback - INFO - Starting async job submit for node 69c99cc7f2866272ea51ef67 2026-03-31 21:42:48,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:42:48,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99cc7f2866272ea51ef67 HTTP/1.1" 200 None 2026-03-31 21:42:48,935 - lava_callback - DEBUG - Processing node 69c99cc7f2866272ea51ef67: baseline-arm64-baylibre 2026-03-31 21:42:48,935 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99cc7f2866272ea51ef67 2026-03-31 21:42:48,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:42:48,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99cc7f2866272ea51ef67 HTTP/1.1" 200 None 2026-03-31 21:42:48,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:42:48,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:42:48,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:42:48,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99cc7f2866272ea51ef67 HTTP/1.1" 200 None 2026-03-31 21:42:49,003 - lava_callback - ERROR - Error processing callback for node 69c99cc7f2866272ea51ef67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44880 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53018 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:43:31,844 - lava_callback - INFO - Received callback for node 69cc24b38df9359ff4898f85 INFO: 172.19.0.1:56712 - "POST /node/69cc24b38df9359ff4898f85 HTTP/1.0" 202 Accepted 2026-03-31 21:43:31,893 - lava_callback - INFO - Starting async job submit for node 69cc24b38df9359ff4898f85 2026-03-31 21:43:31,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:43:31,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc24b38df9359ff4898f85 HTTP/1.1" 200 None 2026-03-31 21:43:31,933 - lava_callback - DEBUG - Processing node 69cc24b38df9359ff4898f85: baseline-arm-pengutronix 2026-03-31 21:43:32,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 21:43:32,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 21:43:32,199 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc24b38df9359ff4898f85/log.txt.gz 2026-03-31 21:43:32,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 21:43:32,487 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 21:43:32,489 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc24b38df9359ff4898f85/lava-logs.yaml 2026-03-31 21:43:32,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-31 21:43:32,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-31 21:43:32,637 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cc24b38df9359ff4898f85/lava_callback.json.gz DEBUG: Setup failed for 69cc24b38df9359ff4898f85. Transitting job node result: incomplete -> fail 2026-03-31 21:43:32,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:43:32,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc24b38df9359ff4898f85 HTTP/1.1" 200 None 2026-03-31 21:43:32,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:43:32,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-03-31 21:43:32,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:43:32,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc24b38df9359ff4898f85 HTTP/1.1" 200 None 2026-03-31 21:43:32,825 - lava_callback - INFO - Completed processing callback for node 69cc24b38df9359ff4898f85 INFO: 172.19.0.1:56722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54182 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:44:00,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:44:00,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.19.0.1:49944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33100 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:44:49,119 - lava_callback - INFO - Received callback for node 69c99d3ef2866272ea51ef75 INFO: 172.19.0.1:33108 - "POST /node/69c99d3ef2866272ea51ef75 HTTP/1.0" 202 Accepted 2026-03-31 21:44:49,125 - lava_callback - INFO - Starting async job submit for node 69c99d3ef2866272ea51ef75 2026-03-31 21:44:49,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:44:49,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99d3ef2866272ea51ef75 HTTP/1.1" 200 None 2026-03-31 21:44:49,141 - lava_callback - DEBUG - Processing node 69c99d3ef2866272ea51ef75: baseline-arm64-baylibre 2026-03-31 21:44:49,141 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99d3ef2866272ea51ef75 2026-03-31 21:44:49,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:44:49,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99d3ef2866272ea51ef75 HTTP/1.1" 200 None 2026-03-31 21:44:49,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:44:49,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:44:49,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:44:49,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99d3ef2866272ea51ef75 HTTP/1.1" 200 None 2026-03-31 21:44:49,195 - lava_callback - ERROR - Error processing callback for node 69c99d3ef2866272ea51ef75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59656 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34722 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:46:19,940 - lava_callback - INFO - Received callback for node 69c99d9af2866272ea51ef85 INFO: 172.19.0.1:32906 - "POST /node/69c99d9af2866272ea51ef85 HTTP/1.0" 202 Accepted 2026-03-31 21:46:19,945 - lava_callback - INFO - Starting async job submit for node 69c99d9af2866272ea51ef85 2026-03-31 21:46:19,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:46:19,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99d9af2866272ea51ef85 HTTP/1.1" 200 None 2026-03-31 21:46:19,988 - lava_callback - DEBUG - Processing node 69c99d9af2866272ea51ef85: baseline-arm64-clabbe 2026-03-31 21:46:19,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99d9af2866272ea51ef85 2026-03-31 21:46:19,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:46:20,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99d9af2866272ea51ef85 HTTP/1.1" 200 None 2026-03-31 21:46:20,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:46:20,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:46:20,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:46:20,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99d9af2866272ea51ef85 HTTP/1.1" 200 None 2026-03-31 21:46:20,052 - lava_callback - ERROR - Error processing callback for node 69c99d9af2866272ea51ef85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:32908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46954 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40906 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:47:29,272 - lava_callback - INFO - Received callback for node 69c99de8f2866272ea51ef8e INFO: 172.19.0.1:40912 - "POST /node/69c99de8f2866272ea51ef8e HTTP/1.0" 202 Accepted 2026-03-31 21:47:29,277 - lava_callback - INFO - Starting async job submit for node 69c99de8f2866272ea51ef8e 2026-03-31 21:47:29,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:47:29,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99de8f2866272ea51ef8e HTTP/1.1" 200 None 2026-03-31 21:47:29,293 - lava_callback - DEBUG - Processing node 69c99de8f2866272ea51ef8e: baseline-arm64-baylibre 2026-03-31 21:47:29,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99de8f2866272ea51ef8e 2026-03-31 21:47:29,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:47:29,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99de8f2866272ea51ef8e HTTP/1.1" 200 None 2026-03-31 21:47:29,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:47:29,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:47:29,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:47:29,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99de8f2866272ea51ef8e HTTP/1.1" 200 None 2026-03-31 21:47:29,346 - lava_callback - ERROR - Error processing callback for node 69c99de8f2866272ea51ef8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56652 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:48:59,968 - lava_callback - INFO - Received callback for node 69c99e3ff2866272ea51ef9b INFO: 172.19.0.1:52856 - "POST /node/69c99e3ff2866272ea51ef9b HTTP/1.0" 202 Accepted 2026-03-31 21:48:59,976 - lava_callback - INFO - Starting async job submit for node 69c99e3ff2866272ea51ef9b 2026-03-31 21:48:59,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:48:59,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99e3ff2866272ea51ef9b HTTP/1.1" 200 None 2026-03-31 21:49:00,000 - lava_callback - DEBUG - Processing node 69c99e3ff2866272ea51ef9b: baseline-arm64-clabbe 2026-03-31 21:49:00,000 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99e3ff2866272ea51ef9b 2026-03-31 21:49:00,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:49:00,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99e3ff2866272ea51ef9b HTTP/1.1" 200 None 2026-03-31 21:49:00,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:49:00,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:49:00,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:49:00,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99e3ff2866272ea51ef9b HTTP/1.1" 200 None 2026-03-31 21:49:00,072 - lava_callback - ERROR - Error processing callback for node 69c99e3ff2866272ea51ef9b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38678 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34100 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:50:09,555 - lava_callback - INFO - Received callback for node 69c99e87f2866272ea51efa6 INFO: 172.19.0.1:34112 - "POST /node/69c99e87f2866272ea51efa6 HTTP/1.0" 202 Accepted 2026-03-31 21:50:09,564 - lava_callback - INFO - Starting async job submit for node 69c99e87f2866272ea51efa6 2026-03-31 21:50:09,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:50:09,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99e87f2866272ea51efa6 HTTP/1.1" 200 None 2026-03-31 21:50:09,580 - lava_callback - DEBUG - Processing node 69c99e87f2866272ea51efa6: baseline-arm64-baylibre 2026-03-31 21:50:09,581 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99e87f2866272ea51efa6 2026-03-31 21:50:09,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:50:09,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99e87f2866272ea51efa6 HTTP/1.1" 200 None 2026-03-31 21:50:09,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:50:09,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:50:09,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:50:09,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99e87f2866272ea51efa6 HTTP/1.1" 200 None 2026-03-31 21:50:09,645 - lava_callback - ERROR - Error processing callback for node 69c99e87f2866272ea51efa6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59746 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60768 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:51:40,142 - lava_callback - INFO - Received callback for node 69c99ee1f2866272ea51efb5 INFO: 172.19.0.1:56002 - "POST /node/69c99ee1f2866272ea51efb5 HTTP/1.0" 202 Accepted 2026-03-31 21:51:40,148 - lava_callback - INFO - Starting async job submit for node 69c99ee1f2866272ea51efb5 2026-03-31 21:51:40,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:51:40,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99ee1f2866272ea51efb5 HTTP/1.1" 200 None 2026-03-31 21:51:40,164 - lava_callback - DEBUG - Processing node 69c99ee1f2866272ea51efb5: baseline-arm64-clabbe 2026-03-31 21:51:40,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99ee1f2866272ea51efb5 2026-03-31 21:51:40,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:51:40,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99ee1f2866272ea51efb5 HTTP/1.1" 200 None 2026-03-31 21:51:40,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:51:40,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:51:40,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:51:40,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99ee1f2866272ea51efb5 HTTP/1.1" 200 None 2026-03-31 21:51:40,249 - lava_callback - ERROR - Error processing callback for node 69c99ee1f2866272ea51efb5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56704 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54818 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:53:00,209 - lava_callback - INFO - Received callback for node 69c99f35f2866272ea51efc2 INFO: 172.19.0.1:44530 - "POST /node/69c99f35f2866272ea51efc2 HTTP/1.0" 202 Accepted 2026-03-31 21:53:00,215 - lava_callback - INFO - Starting async job submit for node 69c99f35f2866272ea51efc2 2026-03-31 21:53:00,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:53:00,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99f35f2866272ea51efc2 HTTP/1.1" 200 None 2026-03-31 21:53:00,243 - lava_callback - DEBUG - Processing node 69c99f35f2866272ea51efc2: baseline-arm64-clabbe 2026-03-31 21:53:00,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99f35f2866272ea51efc2 2026-03-31 21:53:00,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:53:00,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99f35f2866272ea51efc2 HTTP/1.1" 200 None 2026-03-31 21:53:00,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:53:00,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:53:00,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:53:00,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99f35f2866272ea51efc2 HTTP/1.1" 200 None 2026-03-31 21:53:00,302 - lava_callback - ERROR - Error processing callback for node 69c99f35f2866272ea51efc2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37706 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50936 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35170 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38864 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:55:00,375 - lava_callback - INFO - Received callback for node 69c99fabf2866272ea51efce INFO: 172.19.0.1:34950 - "POST /node/69c99fabf2866272ea51efce HTTP/1.0" 202 Accepted 2026-03-31 21:55:00,381 - lava_callback - INFO - Starting async job submit for node 69c99fabf2866272ea51efce 2026-03-31 21:55:00,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:55:00,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99fabf2866272ea51efce HTTP/1.1" 200 None 2026-03-31 21:55:00,396 - lava_callback - DEBUG - Processing node 69c99fabf2866272ea51efce: baseline-arm64-clabbe 2026-03-31 21:55:00,396 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c99fabf2866272ea51efce 2026-03-31 21:55:00,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:55:00,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c99fabf2866272ea51efce HTTP/1.1" 200 None 2026-03-31 21:55:00,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:55:00,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:55:00,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:55:00,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c99fabf2866272ea51efce HTTP/1.1" 200 None 2026-03-31 21:55:00,459 - lava_callback - ERROR - Error processing callback for node 69c99fabf2866272ea51efce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57874 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49942 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57676 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42474 - "GET / HTTP/1.0" 200 OK 2026-03-31 21:58:00,347 - lava_callback - INFO - Received callback for node 69c9a060f2866272ea51efdc INFO: 172.19.0.1:35206 - "POST /node/69c9a060f2866272ea51efdc HTTP/1.0" 202 Accepted 2026-03-31 21:58:00,352 - lava_callback - INFO - Starting async job submit for node 69c9a060f2866272ea51efdc 2026-03-31 21:58:00,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:58:00,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a060f2866272ea51efdc HTTP/1.1" 200 None 2026-03-31 21:58:00,391 - lava_callback - DEBUG - Processing node 69c9a060f2866272ea51efdc: baseline-arm64-clabbe 2026-03-31 21:58:00,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9a060f2866272ea51efdc 2026-03-31 21:58:00,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:58:00,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a060f2866272ea51efdc HTTP/1.1" 200 None 2026-03-31 21:58:00,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:58:00,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 21:58:00,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 21:58:00,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9a060f2866272ea51efdc HTTP/1.1" 200 None 2026-03-31 21:58:00,453 - lava_callback - ERROR - Error processing callback for node 69c9a060f2866272ea51efdc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48610 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35604 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:00:01,459 - lava_callback - INFO - Received callback for node 69c9a0d8f2866272ea51efea INFO: 172.19.0.1:43506 - "POST /node/69c9a0d8f2866272ea51efea HTTP/1.0" 202 Accepted 2026-03-31 22:00:01,465 - lava_callback - INFO - Starting async job submit for node 69c9a0d8f2866272ea51efea 2026-03-31 22:00:01,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:00:01,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a0d8f2866272ea51efea HTTP/1.1" 200 None 2026-03-31 22:00:01,486 - lava_callback - DEBUG - Processing node 69c9a0d8f2866272ea51efea: baseline-arm64-clabbe 2026-03-31 22:00:01,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9a0d8f2866272ea51efea 2026-03-31 22:00:01,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:00:01,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a0d8f2866272ea51efea HTTP/1.1" 200 None 2026-03-31 22:00:01,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:00:01,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 22:00:01,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:00:01,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9a0d8f2866272ea51efea HTTP/1.1" 200 None 2026-03-31 22:00:01,553 - lava_callback - ERROR - Error processing callback for node 69c9a0d8f2866272ea51efea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60840 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:01:09,785 - lava_callback - INFO - Received callback for node 69c9a120f2866272ea51effa INFO: 172.19.0.1:55666 - "POST /node/69c9a120f2866272ea51effa HTTP/1.0" 202 Accepted 2026-03-31 22:01:09,790 - lava_callback - INFO - Starting async job submit for node 69c9a120f2866272ea51effa 2026-03-31 22:01:09,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:01:09,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a120f2866272ea51effa HTTP/1.1" 200 None 2026-03-31 22:01:09,805 - lava_callback - DEBUG - Processing node 69c9a120f2866272ea51effa: baseline-arm64-baylibre 2026-03-31 22:01:09,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9a120f2866272ea51effa 2026-03-31 22:01:09,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:01:09,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a120f2866272ea51effa HTTP/1.1" 200 None 2026-03-31 22:01:09,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:01:09,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 22:01:09,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:01:09,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9a120f2866272ea51effa HTTP/1.1" 200 None 2026-03-31 22:01:09,895 - lava_callback - ERROR - Error processing callback for node 69c9a120f2866272ea51effa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54344 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:03:00,533 - lava_callback - INFO - Received callback for node 69c9a18cf2866272ea51f004 INFO: 172.19.0.1:57830 - "POST /node/69c9a18cf2866272ea51f004 HTTP/1.0" 202 Accepted 2026-03-31 22:03:00,538 - lava_callback - INFO - Starting async job submit for node 69c9a18cf2866272ea51f004 2026-03-31 22:03:00,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:03:00,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a18cf2866272ea51f004 HTTP/1.1" 200 None 2026-03-31 22:03:00,554 - lava_callback - DEBUG - Processing node 69c9a18cf2866272ea51f004: baseline-arm64-clabbe 2026-03-31 22:03:00,555 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9a18cf2866272ea51f004 2026-03-31 22:03:00,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:03:00,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a18cf2866272ea51f004 HTTP/1.1" 200 None 2026-03-31 22:03:00,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:03:00,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 22:03:00,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:03:00,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9a18cf2866272ea51f004 HTTP/1.1" 200 None 2026-03-31 22:03:00,610 - lava_callback - ERROR - Error processing callback for node 69c9a18cf2866272ea51f004: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57832 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57326 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:04:29,971 - lava_callback - INFO - Received callback for node 69c9a1d8f2866272ea51f014 INFO: 172.19.0.1:36274 - "POST /node/69c9a1d8f2866272ea51f014 HTTP/1.0" 202 Accepted 2026-03-31 22:04:29,976 - lava_callback - INFO - Starting async job submit for node 69c9a1d8f2866272ea51f014 2026-03-31 22:04:29,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:04:29,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a1d8f2866272ea51f014 HTTP/1.1" 200 None 2026-03-31 22:04:29,994 - lava_callback - DEBUG - Processing node 69c9a1d8f2866272ea51f014: baseline-arm64-baylibre 2026-03-31 22:04:29,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9a1d8f2866272ea51f014 2026-03-31 22:04:29,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:04:30,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a1d8f2866272ea51f014 HTTP/1.1" 200 None 2026-03-31 22:04:30,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:04:30,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 22:04:30,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:04:30,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9a1d8f2866272ea51f014 HTTP/1.1" 200 None 2026-03-31 22:04:30,054 - lava_callback - ERROR - Error processing callback for node 69c9a1d8f2866272ea51f014: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41116 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36658 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44028 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58318 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:05:50,028 - lava_callback - INFO - Received callback for node 69c9a22af2866272ea51f01e INFO: 172.19.0.1:48750 - "POST /node/69c9a22af2866272ea51f01e HTTP/1.0" 202 Accepted 2026-03-31 22:05:50,033 - lava_callback - INFO - Starting async job submit for node 69c9a22af2866272ea51f01e 2026-03-31 22:05:50,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:05:50,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a22af2866272ea51f01e HTTP/1.1" 200 None 2026-03-31 22:05:50,069 - lava_callback - DEBUG - Processing node 69c9a22af2866272ea51f01e: baseline-arm64-baylibre 2026-03-31 22:05:50,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9a22af2866272ea51f01e 2026-03-31 22:05:50,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:05:50,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a22af2866272ea51f01e HTTP/1.1" 200 None 2026-03-31 22:05:50,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:05:50,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 22:05:50,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:05:50,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9a22af2866272ea51f01e HTTP/1.1" 200 None 2026-03-31 22:05:50,123 - lava_callback - ERROR - Error processing callback for node 69c9a22af2866272ea51f01e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49984 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33778 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:09:00,822 - lava_callback - INFO - Received callback for node 69c9a2f4f2866272ea51f02f INFO: 172.19.0.1:43122 - "POST /node/69c9a2f4f2866272ea51f02f HTTP/1.0" 202 Accepted 2026-03-31 22:09:00,827 - lava_callback - INFO - Starting async job submit for node 69c9a2f4f2866272ea51f02f 2026-03-31 22:09:00,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:09:00,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a2f4f2866272ea51f02f HTTP/1.1" 200 None 2026-03-31 22:09:00,843 - lava_callback - DEBUG - Processing node 69c9a2f4f2866272ea51f02f: baseline-arm64-clabbe 2026-03-31 22:09:00,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9a2f4f2866272ea51f02f 2026-03-31 22:09:00,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:09:00,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a2f4f2866272ea51f02f HTTP/1.1" 200 None 2026-03-31 22:09:00,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:09:00,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 22:09:00,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:09:00,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9a2f4f2866272ea51f02f HTTP/1.1" 200 None 2026-03-31 22:09:00,900 - lava_callback - ERROR - Error processing callback for node 69c9a2f4f2866272ea51f02f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45644 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41038 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58428 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57960 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59556 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:12:00,951 - lava_callback - INFO - Received callback for node 69c9a3a8f2866272ea51f03e INFO: 172.19.0.1:43862 - "POST /node/69c9a3a8f2866272ea51f03e HTTP/1.0" 202 Accepted 2026-03-31 22:12:00,956 - lava_callback - INFO - Starting async job submit for node 69c9a3a8f2866272ea51f03e 2026-03-31 22:12:00,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:12:00,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a3a8f2866272ea51f03e HTTP/1.1" 200 None 2026-03-31 22:12:00,972 - lava_callback - DEBUG - Processing node 69c9a3a8f2866272ea51f03e: baseline-arm64-clabbe 2026-03-31 22:12:00,972 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9a3a8f2866272ea51f03e 2026-03-31 22:12:00,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:12:00,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a3a8f2866272ea51f03e HTTP/1.1" 200 None 2026-03-31 22:12:00,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:12:01,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 22:12:01,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:12:01,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9a3a8f2866272ea51f03e HTTP/1.1" 200 None 2026-03-31 22:12:01,024 - lava_callback - ERROR - Error processing callback for node 69c9a3a8f2866272ea51f03e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49426 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42790 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57390 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:13:20,972 - lava_callback - INFO - Received callback for node 69c9a3faf2866272ea51f04e INFO: 172.19.0.1:57636 - "POST /node/69c9a3faf2866272ea51f04e HTTP/1.0" 202 Accepted 2026-03-31 22:13:20,978 - lava_callback - INFO - Starting async job submit for node 69c9a3faf2866272ea51f04e 2026-03-31 22:13:20,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:13:20,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a3faf2866272ea51f04e HTTP/1.1" 200 None 2026-03-31 22:13:20,994 - lava_callback - DEBUG - Processing node 69c9a3faf2866272ea51f04e: baseline-arm64-clabbe 2026-03-31 22:13:20,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9a3faf2866272ea51f04e 2026-03-31 22:13:20,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:13:21,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9a3faf2866272ea51f04e HTTP/1.1" 200 None 2026-03-31 22:13:21,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:13:21,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-31 22:13:21,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:13:21,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9a3faf2866272ea51f04e HTTP/1.1" 200 None 2026-03-31 22:13:21,048 - lava_callback - ERROR - Error processing callback for node 69c9a3faf2866272ea51f04e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34372 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41418 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49778 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55556 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50382 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37732 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55114 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39716 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48244 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60702 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57910 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:32916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58490 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48816 - "GET /node/69bd0ff005e4bbd6ba635a67%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:48822 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51456 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48512 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35728 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38106 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59390 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41094 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55150 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55862 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42086 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50126 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47004 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44928 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39154 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49628 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60864 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49570 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44574 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36452 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:43:52,235 - lava_callback - INFO - Received callback for node 69c9ab24f2866272ea51f0a6 INFO: 172.19.0.1:59324 - "POST /node/69c9ab24f2866272ea51f0a6 HTTP/1.0" 202 Accepted 2026-03-31 22:43:52,241 - lava_callback - INFO - Starting async job submit for node 69c9ab24f2866272ea51f0a6 2026-03-31 22:43:52,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:43:52,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ab24f2866272ea51f0a6 HTTP/1.1" 200 None 2026-03-31 22:43:52,265 - lava_callback - DEBUG - Processing node 69c9ab24f2866272ea51f0a6: baseline-arm-baylibre 2026-03-31 22:43:52,265 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ab24f2866272ea51f0a6 2026-03-31 22:43:52,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:43:52,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ab24f2866272ea51f0a6 HTTP/1.1" 200 None 2026-03-31 22:43:52,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:43:52,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 22:43:52,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:43:52,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ab24f2866272ea51f0a6 HTTP/1.1" 200 None 2026-03-31 22:43:52,323 - lava_callback - ERROR - Error processing callback for node 69c9ab24f2866272ea51f0a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49888 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54150 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:44:32,117 - lava_callback - INFO - Received callback for node 69c9ab48f2866272ea51f0b3 INFO: 172.19.0.1:52542 - "POST /node/69c9ab48f2866272ea51f0b3 HTTP/1.0" 202 Accepted 2026-03-31 22:44:32,126 - lava_callback - INFO - Starting async job submit for node 69c9ab48f2866272ea51f0b3 2026-03-31 22:44:32,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:44:32,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ab48f2866272ea51f0b3 HTTP/1.1" 200 None 2026-03-31 22:44:32,143 - lava_callback - DEBUG - Processing node 69c9ab48f2866272ea51f0b3: baseline-arm-baylibre 2026-03-31 22:44:32,143 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ab48f2866272ea51f0b3 2026-03-31 22:44:32,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:44:32,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ab48f2866272ea51f0b3 HTTP/1.1" 200 None 2026-03-31 22:44:32,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:44:32,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 22:44:32,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:44:32,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ab48f2866272ea51f0b3 HTTP/1.1" 200 None 2026-03-31 22:44:32,209 - lava_callback - ERROR - Error processing callback for node 69c9ab48f2866272ea51f0b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47514 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39660 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40434 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:45:12,435 - lava_callback - INFO - Received callback for node 69c9ab6df2866272ea51f0bd INFO: 172.19.0.1:37546 - "POST /node/69c9ab6df2866272ea51f0bd HTTP/1.0" 202 Accepted 2026-03-31 22:45:12,441 - lava_callback - INFO - Starting async job submit for node 69c9ab6df2866272ea51f0bd 2026-03-31 22:45:12,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:45:12,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ab6df2866272ea51f0bd HTTP/1.1" 200 None 2026-03-31 22:45:12,458 - lava_callback - DEBUG - Processing node 69c9ab6df2866272ea51f0bd: baseline-arm-baylibre 2026-03-31 22:45:12,458 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ab6df2866272ea51f0bd 2026-03-31 22:45:12,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:45:12,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ab6df2866272ea51f0bd HTTP/1.1" 200 None 2026-03-31 22:45:12,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:45:12,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 22:45:12,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:45:12,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ab6df2866272ea51f0bd HTTP/1.1" 200 None 2026-03-31 22:45:12,521 - lava_callback - ERROR - Error processing callback for node 69c9ab6df2866272ea51f0bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48936 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:45:52,184 - lava_callback - INFO - Received callback for node 69c9ab91f2866272ea51f0c9 INFO: 172.19.0.1:34502 - "POST /node/69c9ab91f2866272ea51f0c9 HTTP/1.0" 202 Accepted 2026-03-31 22:45:52,190 - lava_callback - INFO - Starting async job submit for node 69c9ab91f2866272ea51f0c9 2026-03-31 22:45:52,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:45:52,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ab91f2866272ea51f0c9 HTTP/1.1" 200 None 2026-03-31 22:45:52,205 - lava_callback - DEBUG - Processing node 69c9ab91f2866272ea51f0c9: baseline-arm-baylibre 2026-03-31 22:45:52,206 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ab91f2866272ea51f0c9 2026-03-31 22:45:52,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:45:52,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ab91f2866272ea51f0c9 HTTP/1.1" 200 None 2026-03-31 22:45:52,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:45:52,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 22:45:52,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:45:52,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ab91f2866272ea51f0c9 HTTP/1.1" 200 None 2026-03-31 22:45:52,270 - lava_callback - ERROR - Error processing callback for node 69c9ab91f2866272ea51f0c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34528 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35754 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:46:32,252 - lava_callback - INFO - Received callback for node 69c9abbaf2866272ea51f0d4 INFO: 172.19.0.1:54342 - "POST /node/69c9abbaf2866272ea51f0d4 HTTP/1.0" 202 Accepted 2026-03-31 22:46:32,258 - lava_callback - INFO - Starting async job submit for node 69c9abbaf2866272ea51f0d4 2026-03-31 22:46:32,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:46:32,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9abbaf2866272ea51f0d4 HTTP/1.1" 200 None 2026-03-31 22:46:32,281 - lava_callback - DEBUG - Processing node 69c9abbaf2866272ea51f0d4: baseline-arm-baylibre 2026-03-31 22:46:32,281 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9abbaf2866272ea51f0d4 2026-03-31 22:46:32,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:46:32,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9abbaf2866272ea51f0d4 HTTP/1.1" 200 None 2026-03-31 22:46:32,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:46:32,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 22:46:32,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:46:32,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9abbaf2866272ea51f0d4 HTTP/1.1" 200 None 2026-03-31 22:46:32,350 - lava_callback - ERROR - Error processing callback for node 69c9abbaf2866272ea51f0d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59684 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47896 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51764 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56306 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46296 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:49:02,415 - lava_callback - INFO - Received callback for node 69c9ac48f2866272ea51f0ff INFO: 172.19.0.1:46306 - "POST /node/69c9ac48f2866272ea51f0ff HTTP/1.0" 202 Accepted 2026-03-31 22:49:02,420 - lava_callback - INFO - Starting async job submit for node 69c9ac48f2866272ea51f0ff 2026-03-31 22:49:02,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:49:02,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ac48f2866272ea51f0ff HTTP/1.1" 200 None 2026-03-31 22:49:02,446 - lava_callback - DEBUG - Processing node 69c9ac48f2866272ea51f0ff: baseline-arm-clabbe 2026-03-31 22:49:02,447 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ac48f2866272ea51f0ff 2026-03-31 22:49:02,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:49:02,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ac48f2866272ea51f0ff HTTP/1.1" 200 None 2026-03-31 22:49:02,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:49:02,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 22:49:02,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:49:02,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ac48f2866272ea51f0ff HTTP/1.1" 200 None 2026-03-31 22:49:02,505 - lava_callback - ERROR - Error processing callback for node 69c9ac48f2866272ea51f0ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:46320 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54536 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:49:22,358 - lava_callback - INFO - Received callback for node 69c9ac6df2866272ea51f10a INFO: 172.19.0.1:36266 - "POST /node/69c9ac6df2866272ea51f10a HTTP/1.0" 202 Accepted 2026-03-31 22:49:22,363 - lava_callback - INFO - Starting async job submit for node 69c9ac6df2866272ea51f10a 2026-03-31 22:49:22,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:49:22,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ac6df2866272ea51f10a HTTP/1.1" 200 None 2026-03-31 22:49:22,390 - lava_callback - DEBUG - Processing node 69c9ac6df2866272ea51f10a: baseline-arm-clabbe 2026-03-31 22:49:22,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ac6df2866272ea51f10a 2026-03-31 22:49:22,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:49:22,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ac6df2866272ea51f10a HTTP/1.1" 200 None 2026-03-31 22:49:22,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:49:22,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 22:49:22,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:49:22,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ac6df2866272ea51f10a HTTP/1.1" 200 None 2026-03-31 22:49:22,445 - lava_callback - ERROR - Error processing callback for node 69c9ac6df2866272ea51f10a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58208 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53284 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:50:02,395 - lava_callback - INFO - Received callback for node 69c9ac92f2866272ea51f117 INFO: 172.19.0.1:53298 - "POST /node/69c9ac92f2866272ea51f117 HTTP/1.0" 202 Accepted 2026-03-31 22:50:02,400 - lava_callback - INFO - Starting async job submit for node 69c9ac92f2866272ea51f117 2026-03-31 22:50:02,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:50:02,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ac92f2866272ea51f117 HTTP/1.1" 200 None 2026-03-31 22:50:02,416 - lava_callback - DEBUG - Processing node 69c9ac92f2866272ea51f117: baseline-arm-clabbe 2026-03-31 22:50:02,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ac92f2866272ea51f117 2026-03-31 22:50:02,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:50:02,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ac92f2866272ea51f117 HTTP/1.1" 200 None 2026-03-31 22:50:02,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:50:02,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 22:50:02,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:50:02,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ac92f2866272ea51f117 HTTP/1.1" 200 None 2026-03-31 22:50:02,493 - lava_callback - ERROR - Error processing callback for node 69c9ac92f2866272ea51f117: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49368 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50474 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:50:42,421 - lava_callback - INFO - Received callback for node 69c9acb8f2866272ea51f124 INFO: 172.19.0.1:41064 - "POST /node/69c9acb8f2866272ea51f124 HTTP/1.0" 202 Accepted 2026-03-31 22:50:42,429 - lava_callback - INFO - Starting async job submit for node 69c9acb8f2866272ea51f124 2026-03-31 22:50:42,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:50:42,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9acb8f2866272ea51f124 HTTP/1.1" 200 None 2026-03-31 22:50:42,446 - lava_callback - DEBUG - Processing node 69c9acb8f2866272ea51f124: baseline-arm-clabbe 2026-03-31 22:50:42,447 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9acb8f2866272ea51f124 2026-03-31 22:50:42,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:50:42,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9acb8f2866272ea51f124 HTTP/1.1" 200 None 2026-03-31 22:50:42,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:50:42,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 22:50:42,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:50:42,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9acb8f2866272ea51f124 HTTP/1.1" 200 None 2026-03-31 22:50:42,505 - lava_callback - ERROR - Error processing callback for node 69c9acb8f2866272ea51f124: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41074 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39108 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55562 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:51:22,480 - lava_callback - INFO - Received callback for node 69c9acdcf2866272ea51f135 INFO: 172.19.0.1:53156 - "POST /node/69c9acdcf2866272ea51f135 HTTP/1.0" 202 Accepted 2026-03-31 22:51:22,485 - lava_callback - INFO - Starting async job submit for node 69c9acdcf2866272ea51f135 2026-03-31 22:51:22,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:51:22,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9acdcf2866272ea51f135 HTTP/1.1" 200 None 2026-03-31 22:51:22,504 - lava_callback - DEBUG - Processing node 69c9acdcf2866272ea51f135: baseline-arm-clabbe 2026-03-31 22:51:22,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9acdcf2866272ea51f135 2026-03-31 22:51:22,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:51:22,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9acdcf2866272ea51f135 HTTP/1.1" 200 None 2026-03-31 22:51:22,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:51:22,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-03-31 22:51:22,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:51:22,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9acdcf2866272ea51f135 HTTP/1.1" 200 None 2026-03-31 22:51:22,578 - lava_callback - ERROR - Error processing callback for node 69c9acdcf2866272ea51f135: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59132 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:51:52,577 - lava_callback - INFO - Received callback for node 69c9acfcf2866272ea51f141 INFO: 172.19.0.1:60230 - "POST /node/69c9acfcf2866272ea51f141 HTTP/1.0" 202 Accepted 2026-03-31 22:51:52,582 - lava_callback - INFO - Starting async job submit for node 69c9acfcf2866272ea51f141 2026-03-31 22:51:52,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:51:52,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9acfcf2866272ea51f141 HTTP/1.1" 200 None 2026-03-31 22:51:52,602 - lava_callback - DEBUG - Processing node 69c9acfcf2866272ea51f141: baseline-arm-baylibre 2026-03-31 22:51:52,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9acfcf2866272ea51f141 2026-03-31 22:51:52,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:51:52,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9acfcf2866272ea51f141 HTTP/1.1" 200 None 2026-03-31 22:51:52,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:51:52,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 22:51:52,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:51:52,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9acfcf2866272ea51f141 HTTP/1.1" 200 None 2026-03-31 22:51:52,660 - lava_callback - ERROR - Error processing callback for node 69c9acfcf2866272ea51f141: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39968 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56606 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48546 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:52:32,510 - lava_callback - INFO - Received callback for node 69c9ad1ff2866272ea51f14c INFO: 172.19.0.1:48336 - "POST /node/69c9ad1ff2866272ea51f14c HTTP/1.0" 202 Accepted 2026-03-31 22:52:32,516 - lava_callback - INFO - Starting async job submit for node 69c9ad1ff2866272ea51f14c 2026-03-31 22:52:32,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:52:32,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ad1ff2866272ea51f14c HTTP/1.1" 200 None 2026-03-31 22:52:32,535 - lava_callback - DEBUG - Processing node 69c9ad1ff2866272ea51f14c: baseline-arm-baylibre 2026-03-31 22:52:32,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ad1ff2866272ea51f14c 2026-03-31 22:52:32,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:52:32,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ad1ff2866272ea51f14c HTTP/1.1" 200 None 2026-03-31 22:52:32,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:52:32,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 22:52:32,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:52:32,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ad1ff2866272ea51f14c HTTP/1.1" 200 None 2026-03-31 22:52:32,587 - lava_callback - ERROR - Error processing callback for node 69c9ad1ff2866272ea51f14c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54742 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58260 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:53:12,617 - lava_callback - INFO - Received callback for node 69c9ad43f2866272ea51f161 INFO: 172.19.0.1:36234 - "POST /node/69c9ad43f2866272ea51f161 HTTP/1.0" 202 Accepted 2026-03-31 22:53:12,623 - lava_callback - INFO - Starting async job submit for node 69c9ad43f2866272ea51f161 2026-03-31 22:53:12,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:53:12,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ad43f2866272ea51f161 HTTP/1.1" 200 None 2026-03-31 22:53:12,639 - lava_callback - DEBUG - Processing node 69c9ad43f2866272ea51f161: baseline-arm-baylibre 2026-03-31 22:53:12,639 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ad43f2866272ea51f161 2026-03-31 22:53:12,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:53:12,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ad43f2866272ea51f161 HTTP/1.1" 200 None 2026-03-31 22:53:12,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:53:12,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 22:53:12,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:53:12,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ad43f2866272ea51f161 HTTP/1.1" 200 None 2026-03-31 22:53:12,692 - lava_callback - ERROR - Error processing callback for node 69c9ad43f2866272ea51f161: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40792 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:53:32,583 - lava_callback - INFO - Received callback for node 69c9ad67f2866272ea51f171 INFO: 172.19.0.1:54280 - "POST /node/69c9ad67f2866272ea51f171 HTTP/1.0" 202 Accepted 2026-03-31 22:53:32,589 - lava_callback - INFO - Starting async job submit for node 69c9ad67f2866272ea51f171 2026-03-31 22:53:32,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:53:32,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ad67f2866272ea51f171 HTTP/1.1" 200 None 2026-03-31 22:53:32,605 - lava_callback - DEBUG - Processing node 69c9ad67f2866272ea51f171: baseline-arm-baylibre 2026-03-31 22:53:32,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ad67f2866272ea51f171 2026-03-31 22:53:32,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:53:32,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ad67f2866272ea51f171 HTTP/1.1" 200 None 2026-03-31 22:53:32,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:53:32,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 22:53:32,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:53:32,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ad67f2866272ea51f171 HTTP/1.1" 200 None 2026-03-31 22:53:32,700 - lava_callback - ERROR - Error processing callback for node 69c9ad67f2866272ea51f171: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52830 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47168 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:54:12,623 - lava_callback - INFO - Received callback for node 69c9ad89f2866272ea51f199 INFO: 172.19.0.1:50420 - "POST /node/69c9ad89f2866272ea51f199 HTTP/1.0" 202 Accepted 2026-03-31 22:54:12,628 - lava_callback - INFO - Starting async job submit for node 69c9ad89f2866272ea51f199 2026-03-31 22:54:12,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:54:12,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ad89f2866272ea51f199 HTTP/1.1" 200 None 2026-03-31 22:54:12,645 - lava_callback - DEBUG - Processing node 69c9ad89f2866272ea51f199: baseline-arm-baylibre 2026-03-31 22:54:12,645 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ad89f2866272ea51f199 2026-03-31 22:54:12,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:54:12,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ad89f2866272ea51f199 HTTP/1.1" 200 None 2026-03-31 22:54:12,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:54:12,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 22:54:12,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:54:12,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ad89f2866272ea51f199 HTTP/1.1" 200 None 2026-03-31 22:54:12,699 - lava_callback - ERROR - Error processing callback for node 69c9ad89f2866272ea51f199: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43012 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:54:52,658 - lava_callback - INFO - Received callback for node 69c9adaff2866272ea51f1a7 INFO: 172.19.0.1:56288 - "POST /node/69c9adaff2866272ea51f1a7 HTTP/1.0" 202 Accepted 2026-03-31 22:54:52,664 - lava_callback - INFO - Starting async job submit for node 69c9adaff2866272ea51f1a7 2026-03-31 22:54:52,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:54:52,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9adaff2866272ea51f1a7 HTTP/1.1" 200 None 2026-03-31 22:54:52,680 - lava_callback - DEBUG - Processing node 69c9adaff2866272ea51f1a7: baseline-arm-baylibre 2026-03-31 22:54:52,680 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9adaff2866272ea51f1a7 2026-03-31 22:54:52,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:54:52,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9adaff2866272ea51f1a7 HTTP/1.1" 200 None 2026-03-31 22:54:52,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:54:52,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 22:54:52,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:54:52,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9adaff2866272ea51f1a7 HTTP/1.1" 200 None 2026-03-31 22:54:52,732 - lava_callback - ERROR - Error processing callback for node 69c9adaff2866272ea51f1a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44334 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48536 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:55:32,699 - lava_callback - INFO - Received callback for node 69c9add1f2866272ea51f1b1 INFO: 172.19.0.1:40020 - "POST /node/69c9add1f2866272ea51f1b1 HTTP/1.0" 202 Accepted 2026-03-31 22:55:32,705 - lava_callback - INFO - Starting async job submit for node 69c9add1f2866272ea51f1b1 2026-03-31 22:55:32,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:55:32,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9add1f2866272ea51f1b1 HTTP/1.1" 200 None 2026-03-31 22:55:32,722 - lava_callback - DEBUG - Processing node 69c9add1f2866272ea51f1b1: baseline-arm-baylibre 2026-03-31 22:55:32,722 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9add1f2866272ea51f1b1 2026-03-31 22:55:32,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:55:32,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9add1f2866272ea51f1b1 HTTP/1.1" 200 None 2026-03-31 22:55:32,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:55:32,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 22:55:32,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:55:32,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9add1f2866272ea51f1b1 HTTP/1.1" 200 None 2026-03-31 22:55:32,774 - lava_callback - ERROR - Error processing callback for node 69c9add1f2866272ea51f1b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40026 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57274 - "GET / HTTP/1.0" 200 OK 2026-03-31 22:56:12,761 - lava_callback - INFO - Received callback for node 69c9adf8f2866272ea51f1be INFO: 172.19.0.1:48080 - "POST /node/69c9adf8f2866272ea51f1be HTTP/1.0" 202 Accepted 2026-03-31 22:56:12,766 - lava_callback - INFO - Starting async job submit for node 69c9adf8f2866272ea51f1be 2026-03-31 22:56:12,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:56:12,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9adf8f2866272ea51f1be HTTP/1.1" 200 None 2026-03-31 22:56:12,790 - lava_callback - DEBUG - Processing node 69c9adf8f2866272ea51f1be: baseline-arm-baylibre 2026-03-31 22:56:12,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9adf8f2866272ea51f1be 2026-03-31 22:56:12,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:56:12,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9adf8f2866272ea51f1be HTTP/1.1" 200 None 2026-03-31 22:56:12,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:56:12,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 22:56:12,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 22:56:12,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9adf8f2866272ea51f1be HTTP/1.1" 200 None 2026-03-31 22:56:12,840 - lava_callback - ERROR - Error processing callback for node 69c9adf8f2866272ea51f1be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:48092 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56422 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40062 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43476 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53808 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35346 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46388 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59682 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40576 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52640 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:01:53,546 - lava_callback - INFO - Received callback for node 69c9af56f2866272ea51f20d INFO: 172.19.0.1:49078 - "POST /node/69c9af56f2866272ea51f20d HTTP/1.0" 202 Accepted 2026-03-31 23:01:53,551 - lava_callback - INFO - Starting async job submit for node 69c9af56f2866272ea51f20d 2026-03-31 23:01:53,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:01:53,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9af56f2866272ea51f20d HTTP/1.1" 200 None 2026-03-31 23:01:53,571 - lava_callback - DEBUG - Processing node 69c9af56f2866272ea51f20d: baseline-arm-baylibre 2026-03-31 23:01:53,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9af56f2866272ea51f20d 2026-03-31 23:01:53,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:01:53,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9af56f2866272ea51f20d HTTP/1.1" 200 None 2026-03-31 23:01:53,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:01:53,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:01:53,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:01:53,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9af56f2866272ea51f20d HTTP/1.1" 200 None 2026-03-31 23:01:53,627 - lava_callback - ERROR - Error processing callback for node 69c9af56f2866272ea51f20d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60040 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60052 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:45654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40172 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40528 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:02:42,953 - lava_callback - INFO - Received callback for node 69c9af7ef2866272ea51f218 INFO: 172.19.0.1:57138 - "POST /node/69c9af7ef2866272ea51f218 HTTP/1.0" 202 Accepted 2026-03-31 23:02:42,958 - lava_callback - INFO - Starting async job submit for node 69c9af7ef2866272ea51f218 2026-03-31 23:02:42,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:02:42,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9af7ef2866272ea51f218 HTTP/1.1" 200 None 2026-03-31 23:02:42,974 - lava_callback - DEBUG - Processing node 69c9af7ef2866272ea51f218: baseline-arm-clabbe 2026-03-31 23:02:42,975 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9af7ef2866272ea51f218 2026-03-31 23:02:42,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:02:42,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9af7ef2866272ea51f218 HTTP/1.1" 200 None 2026-03-31 23:02:42,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:02:43,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:02:43,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:02:43,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9af7ef2866272ea51f218 HTTP/1.1" 200 None 2026-03-31 23:02:43,030 - lava_callback - ERROR - Error processing callback for node 69c9af7ef2866272ea51f218: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56476 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:03:13,145 - lava_callback - INFO - Received callback for node 69c9afa0f2866272ea51f222 INFO: 172.19.0.1:59706 - "POST /node/69c9afa0f2866272ea51f222 HTTP/1.0" 202 Accepted 2026-03-31 23:03:13,150 - lava_callback - INFO - Starting async job submit for node 69c9afa0f2866272ea51f222 2026-03-31 23:03:13,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:03:13,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9afa0f2866272ea51f222 HTTP/1.1" 200 None 2026-03-31 23:03:13,189 - lava_callback - DEBUG - Processing node 69c9afa0f2866272ea51f222: baseline-arm-baylibre 2026-03-31 23:03:13,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9afa0f2866272ea51f222 2026-03-31 23:03:13,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:03:13,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9afa0f2866272ea51f222 HTTP/1.1" 200 None 2026-03-31 23:03:13,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:03:13,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:03:13,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:03:13,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9afa0f2866272ea51f222 HTTP/1.1" 200 None 2026-03-31 23:03:13,242 - lava_callback - ERROR - Error processing callback for node 69c9afa0f2866272ea51f222: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59708 - "GET /node/69bd7c840e93e2314723e910%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:59714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40848 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:03:53,139 - lava_callback - INFO - Received callback for node 69c9afccf2866272ea51f22f INFO: 172.19.0.1:41242 - "POST /node/69c9afccf2866272ea51f22f HTTP/1.0" 202 Accepted 2026-03-31 23:03:53,145 - lava_callback - INFO - Starting async job submit for node 69c9afccf2866272ea51f22f 2026-03-31 23:03:53,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:03:53,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9afccf2866272ea51f22f HTTP/1.1" 200 None 2026-03-31 23:03:53,161 - lava_callback - DEBUG - Processing node 69c9afccf2866272ea51f22f: baseline-arm64-baylibre 2026-03-31 23:03:53,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9afccf2866272ea51f22f 2026-03-31 23:03:53,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:03:53,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9afccf2866272ea51f22f HTTP/1.1" 200 None 2026-03-31 23:03:53,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:03:53,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 23:03:53,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:03:53,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9afccf2866272ea51f22f HTTP/1.1" 200 None 2026-03-31 23:03:53,232 - lava_callback - ERROR - Error processing callback for node 69c9afccf2866272ea51f22f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48190 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48206 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44332 - "GET /node/694aa9f183c67c43d1ede04b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:44338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44344 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40946 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38338 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59386 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39234 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39236 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60542 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44712 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:06:33,322 - lava_callback - INFO - Received callback for node 69c9b06ff2866272ea51f240 INFO: 172.19.0.1:36480 - "POST /node/69c9b06ff2866272ea51f240 HTTP/1.0" 202 Accepted 2026-03-31 23:06:33,328 - lava_callback - INFO - Starting async job submit for node 69c9b06ff2866272ea51f240 2026-03-31 23:06:33,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:06:33,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b06ff2866272ea51f240 HTTP/1.1" 200 None 2026-03-31 23:06:33,344 - lava_callback - DEBUG - Processing node 69c9b06ff2866272ea51f240: baseline-arm64-baylibre 2026-03-31 23:06:33,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b06ff2866272ea51f240 2026-03-31 23:06:33,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:06:33,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b06ff2866272ea51f240 HTTP/1.1" 200 None 2026-03-31 23:06:33,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:06:33,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 23:06:33,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:06:33,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b06ff2866272ea51f240 HTTP/1.1" 200 None 2026-03-31 23:06:33,414 - lava_callback - ERROR - Error processing callback for node 69c9b06ff2866272ea51f240: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35854 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50952 - "GET /node/698e2a84952a14bf500e4120%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35238 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:07:53,383 - lava_callback - INFO - Received callback for node 69c9b0bef2866272ea51f252 INFO: 172.19.0.1:38096 - "POST /node/69c9b0bef2866272ea51f252 HTTP/1.0" 202 Accepted 2026-03-31 23:07:53,389 - lava_callback - INFO - Starting async job submit for node 69c9b0bef2866272ea51f252 2026-03-31 23:07:53,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:07:53,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b0bef2866272ea51f252 HTTP/1.1" 200 None 2026-03-31 23:07:53,421 - lava_callback - DEBUG - Processing node 69c9b0bef2866272ea51f252: baseline-arm64-baylibre 2026-03-31 23:07:53,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b0bef2866272ea51f252 2026-03-31 23:07:53,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:07:53,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b0bef2866272ea51f252 HTTP/1.1" 200 None 2026-03-31 23:07:53,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:07:53,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 23:07:53,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:07:53,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b0bef2866272ea51f252 HTTP/1.1" 200 None 2026-03-31 23:07:53,473 - lava_callback - ERROR - Error processing callback for node 69c9b0bef2866272ea51f252: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55922 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55932 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34800 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53312 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48388 - "GET /node/693b7b2141fb4c9509762cae%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:48396 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54892 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47710 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:09:23,330 - lava_callback - INFO - Received callback for node 69c9b115f2866272ea51f262 INFO: 172.19.0.1:53392 - "POST /node/69c9b115f2866272ea51f262 HTTP/1.0" 202 Accepted 2026-03-31 23:09:23,335 - lava_callback - INFO - Starting async job submit for node 69c9b115f2866272ea51f262 2026-03-31 23:09:23,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:09:23,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b115f2866272ea51f262 HTTP/1.1" 200 None 2026-03-31 23:09:23,352 - lava_callback - DEBUG - Processing node 69c9b115f2866272ea51f262: baseline-arm64-clabbe 2026-03-31 23:09:23,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b115f2866272ea51f262 2026-03-31 23:09:23,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:09:23,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b115f2866272ea51f262 HTTP/1.1" 200 None 2026-03-31 23:09:23,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:09:23,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 23:09:23,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:09:23,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b115f2866272ea51f262 HTTP/1.1" 200 None 2026-03-31 23:09:23,420 - lava_callback - ERROR - Error processing callback for node 69c9b115f2866272ea51f262: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57998 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56072 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58026 - "GET /node/6930450d009d5f8c767ee98c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:58042 - "GET /node/6992eec431166481c8dda0a3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:58046 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48984 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:10:43,236 - lava_callback - INFO - Received callback for node 69c9b166f2866272ea51f269 INFO: 172.19.0.1:34688 - "POST /node/69c9b166f2866272ea51f269 HTTP/1.0" 202 Accepted 2026-03-31 23:10:43,241 - lava_callback - INFO - Starting async job submit for node 69c9b166f2866272ea51f269 2026-03-31 23:10:43,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:10:43,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b166f2866272ea51f269 HTTP/1.1" 200 None 2026-03-31 23:10:43,266 - lava_callback - DEBUG - Processing node 69c9b166f2866272ea51f269: baseline-arm64-clabbe 2026-03-31 23:10:43,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b166f2866272ea51f269 2026-03-31 23:10:43,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:10:43,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b166f2866272ea51f269 HTTP/1.1" 200 None 2026-03-31 23:10:43,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:10:43,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 23:10:43,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:10:43,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b166f2866272ea51f269 HTTP/1.1" 200 None 2026-03-31 23:10:43,319 - lava_callback - ERROR - Error processing callback for node 69c9b166f2866272ea51f269: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34692 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56234 - "GET /node/6990b0dbc45febfe44522e74%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:56246 - "GET /node/6992d9e731166481c8dd41b7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:56262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54594 - "GET /node/69bdee80d54b5122738c67e5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:54610 - "GET /node/69afdb77ae557f2600f0f783%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:36560 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52842 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:12:03,295 - lava_callback - INFO - Received callback for node 69c9b1b7f2866272ea51f278 INFO: 172.19.0.1:52844 - "POST /node/69c9b1b7f2866272ea51f278 HTTP/1.0" 202 Accepted 2026-03-31 23:12:03,301 - lava_callback - INFO - Starting async job submit for node 69c9b1b7f2866272ea51f278 2026-03-31 23:12:03,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:12:03,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b1b7f2866272ea51f278 HTTP/1.1" 200 None 2026-03-31 23:12:03,317 - lava_callback - DEBUG - Processing node 69c9b1b7f2866272ea51f278: baseline-arm64-clabbe 2026-03-31 23:12:03,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b1b7f2866272ea51f278 2026-03-31 23:12:03,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:12:03,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b1b7f2866272ea51f278 HTTP/1.1" 200 None 2026-03-31 23:12:03,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:12:03,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 23:12:03,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:12:03,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b1b7f2866272ea51f278 HTTP/1.1" 200 None 2026-03-31 23:12:03,373 - lava_callback - ERROR - Error processing callback for node 69c9b1b7f2866272ea51f278: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32940 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48616 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59802 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33700 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33716 - "GET /node/6983ef2c969566dcb0d2a493%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:33728 - "GET /node/6983aa6e66dc8a56088a7061%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:33730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39342 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:13:23,338 - lava_callback - INFO - Received callback for node 69c9b206f2866272ea51f287 INFO: 172.19.0.1:34720 - "POST /node/69c9b206f2866272ea51f287 HTTP/1.0" 202 Accepted 2026-03-31 23:13:23,343 - lava_callback - INFO - Starting async job submit for node 69c9b206f2866272ea51f287 2026-03-31 23:13:23,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:13:23,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b206f2866272ea51f287 HTTP/1.1" 200 None 2026-03-31 23:13:23,360 - lava_callback - DEBUG - Processing node 69c9b206f2866272ea51f287: baseline-arm64-clabbe 2026-03-31 23:13:23,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b206f2866272ea51f287 2026-03-31 23:13:23,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:13:23,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b206f2866272ea51f287 HTTP/1.1" 200 None 2026-03-31 23:13:23,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:13:23,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 23:13:23,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:13:23,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b206f2866272ea51f287 HTTP/1.1" 200 None 2026-03-31 23:13:23,430 - lava_callback - ERROR - Error processing callback for node 69c9b206f2866272ea51f287: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49332 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53788 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38624 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:14:43,385 - lava_callback - INFO - Received callback for node 69c9b255f2866272ea51f291 INFO: 172.19.0.1:55830 - "POST /node/69c9b255f2866272ea51f291 HTTP/1.0" 202 Accepted 2026-03-31 23:14:43,391 - lava_callback - INFO - Starting async job submit for node 69c9b255f2866272ea51f291 2026-03-31 23:14:43,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:14:43,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b255f2866272ea51f291 HTTP/1.1" 200 None 2026-03-31 23:14:43,407 - lava_callback - DEBUG - Processing node 69c9b255f2866272ea51f291: baseline-arm64-clabbe 2026-03-31 23:14:43,407 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b255f2866272ea51f291 2026-03-31 23:14:43,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:14:43,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b255f2866272ea51f291 HTTP/1.1" 200 None 2026-03-31 23:14:43,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:14:43,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 23:14:43,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:14:43,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b255f2866272ea51f291 HTTP/1.1" 200 None 2026-03-31 23:14:43,459 - lava_callback - ERROR - Error processing callback for node 69c9b255f2866272ea51f291: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55836 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55846 - "GET /node/693fae700d8ce4018da9bc5a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:60122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55858 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59694 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35166 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:16:03,413 - lava_callback - INFO - Received callback for node 69c9b2a4f2866272ea51f2a1 INFO: 172.19.0.1:35178 - "POST /node/69c9b2a4f2866272ea51f2a1 HTTP/1.0" 202 Accepted 2026-03-31 23:16:03,419 - lava_callback - INFO - Starting async job submit for node 69c9b2a4f2866272ea51f2a1 2026-03-31 23:16:03,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:16:03,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b2a4f2866272ea51f2a1 HTTP/1.1" 200 None 2026-03-31 23:16:03,440 - lava_callback - DEBUG - Processing node 69c9b2a4f2866272ea51f2a1: baseline-arm64-clabbe 2026-03-31 23:16:03,441 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b2a4f2866272ea51f2a1 2026-03-31 23:16:03,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:16:03,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b2a4f2866272ea51f2a1 HTTP/1.1" 200 None 2026-03-31 23:16:03,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:16:03,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-03-31 23:16:03,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:16:03,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b2a4f2866272ea51f2a1 HTTP/1.1" 200 None 2026-03-31 23:16:03,498 - lava_callback - ERROR - Error processing callback for node 69c9b2a4f2866272ea51f2a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35192 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39620 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38474 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58248 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:17:03,442 - lava_callback - INFO - Received callback for node 69c9b2e4f2866272ea51f2ab INFO: 172.19.0.1:58258 - "POST /node/69c9b2e4f2866272ea51f2ab HTTP/1.0" 202 Accepted 2026-03-31 23:17:03,447 - lava_callback - INFO - Starting async job submit for node 69c9b2e4f2866272ea51f2ab 2026-03-31 23:17:03,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:17:03,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b2e4f2866272ea51f2ab HTTP/1.1" 200 None 2026-03-31 23:17:03,464 - lava_callback - DEBUG - Processing node 69c9b2e4f2866272ea51f2ab: baseline-arm-clabbe 2026-03-31 23:17:03,465 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b2e4f2866272ea51f2ab 2026-03-31 23:17:03,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:17:03,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b2e4f2866272ea51f2ab HTTP/1.1" 200 None 2026-03-31 23:17:03,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:17:03,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:17:03,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:17:03,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b2e4f2866272ea51f2ab HTTP/1.1" 200 None 2026-03-31 23:17:03,531 - lava_callback - ERROR - Error processing callback for node 69c9b2e4f2866272ea51f2ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:34210 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55266 - "GET /node/694aacd983c67c43d1ede4f2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:55272 - "GET /node/69a161b6a41e9451dcf4132d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:55286 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:17:34,011 - lava_callback - INFO - Received callback for node 69c9b306f2866272ea51f2b6 INFO: 172.19.0.1:48168 - "POST /node/69c9b306f2866272ea51f2b6 HTTP/1.0" 202 Accepted 2026-03-31 23:17:34,016 - lava_callback - INFO - Starting async job submit for node 69c9b306f2866272ea51f2b6 2026-03-31 23:17:34,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:17:34,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b306f2866272ea51f2b6 HTTP/1.1" 200 None 2026-03-31 23:17:34,032 - lava_callback - DEBUG - Processing node 69c9b306f2866272ea51f2b6: baseline-arm-baylibre 2026-03-31 23:17:34,032 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b306f2866272ea51f2b6 2026-03-31 23:17:34,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:17:34,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b306f2866272ea51f2b6 HTTP/1.1" 200 None 2026-03-31 23:17:34,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:17:34,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:17:34,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:17:34,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b306f2866272ea51f2b6 HTTP/1.1" 200 None 2026-03-31 23:17:34,085 - lava_callback - ERROR - Error processing callback for node 69c9b306f2866272ea51f2b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33512 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:18:13,915 - lava_callback - INFO - Received callback for node 69c9b32bf2866272ea51f2c3 INFO: 172.19.0.1:42412 - "POST /node/69c9b32bf2866272ea51f2c3 HTTP/1.0" 202 Accepted 2026-03-31 23:18:13,921 - lava_callback - INFO - Starting async job submit for node 69c9b32bf2866272ea51f2c3 2026-03-31 23:18:13,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:18:13,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b32bf2866272ea51f2c3 HTTP/1.1" 200 None 2026-03-31 23:18:13,936 - lava_callback - DEBUG - Processing node 69c9b32bf2866272ea51f2c3: baseline-arm-baylibre 2026-03-31 23:18:13,937 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b32bf2866272ea51f2c3 2026-03-31 23:18:13,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:18:13,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b32bf2866272ea51f2c3 HTTP/1.1" 200 None 2026-03-31 23:18:13,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:18:13,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:18:13,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:18:14,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b32bf2866272ea51f2c3 HTTP/1.1" 200 None 2026-03-31 23:18:14,006 - lava_callback - ERROR - Error processing callback for node 69c9b32bf2866272ea51f2c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:42422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47966 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:19:03,551 - lava_callback - INFO - Received callback for node 69c9b354f2866272ea51f2cf INFO: 172.19.0.1:47978 - "POST /node/69c9b354f2866272ea51f2cf HTTP/1.0" 202 Accepted 2026-03-31 23:19:03,557 - lava_callback - INFO - Starting async job submit for node 69c9b354f2866272ea51f2cf 2026-03-31 23:19:03,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:19:03,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b354f2866272ea51f2cf HTTP/1.1" 200 None 2026-03-31 23:19:03,573 - lava_callback - DEBUG - Processing node 69c9b354f2866272ea51f2cf: baseline-arm-clabbe 2026-03-31 23:19:03,573 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b354f2866272ea51f2cf 2026-03-31 23:19:03,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:19:03,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b354f2866272ea51f2cf HTTP/1.1" 200 None 2026-03-31 23:19:03,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:19:03,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:19:03,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:19:03,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b354f2866272ea51f2cf HTTP/1.1" 200 None 2026-03-31 23:19:03,624 - lava_callback - ERROR - Error processing callback for node 69c9b354f2866272ea51f2cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56126 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:19:34,049 - lava_callback - INFO - Received callback for node 69c9b376f2866272ea51f2da INFO: 172.19.0.1:58714 - "POST /node/69c9b376f2866272ea51f2da HTTP/1.0" 202 Accepted 2026-03-31 23:19:34,054 - lava_callback - INFO - Starting async job submit for node 69c9b376f2866272ea51f2da 2026-03-31 23:19:34,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:19:34,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b376f2866272ea51f2da HTTP/1.1" 200 None 2026-03-31 23:19:34,074 - lava_callback - DEBUG - Processing node 69c9b376f2866272ea51f2da: baseline-arm-baylibre 2026-03-31 23:19:34,075 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b376f2866272ea51f2da 2026-03-31 23:19:34,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:19:34,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b376f2866272ea51f2da HTTP/1.1" 200 None 2026-03-31 23:19:34,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:19:34,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:19:34,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:19:34,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b376f2866272ea51f2da HTTP/1.1" 200 None 2026-03-31 23:19:34,127 - lava_callback - ERROR - Error processing callback for node 69c9b376f2866272ea51f2da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58720 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45754 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52516 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:20:03,577 - lava_callback - INFO - Received callback for node 69c9b39cf2866272ea51f2e6 INFO: 172.19.0.1:52528 - "POST /node/69c9b39cf2866272ea51f2e6 HTTP/1.0" 202 Accepted 2026-03-31 23:20:03,583 - lava_callback - INFO - Starting async job submit for node 69c9b39cf2866272ea51f2e6 2026-03-31 23:20:03,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:20:03,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b39cf2866272ea51f2e6 HTTP/1.1" 200 None 2026-03-31 23:20:03,600 - lava_callback - DEBUG - Processing node 69c9b39cf2866272ea51f2e6: baseline-arm-clabbe 2026-03-31 23:20:03,600 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b39cf2866272ea51f2e6 2026-03-31 23:20:03,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:20:03,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b39cf2866272ea51f2e6 HTTP/1.1" 200 None 2026-03-31 23:20:03,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:20:03,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:20:03,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:20:03,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b39cf2866272ea51f2e6 HTTP/1.1" 200 None 2026-03-31 23:20:03,660 - lava_callback - ERROR - Error processing callback for node 69c9b39cf2866272ea51f2e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52550 - "GET /node/694ac6736c96ed51d316f2db%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:52566 - "GET /node/69281104b1b5e06bf6c05005%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:60152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:60164 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45318 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58366 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:21:23,629 - lava_callback - INFO - Received callback for node 69c9b3e5f2866272ea51f2fd INFO: 172.19.0.1:53870 - "POST /node/69c9b3e5f2866272ea51f2fd HTTP/1.0" 202 Accepted 2026-03-31 23:21:23,635 - lava_callback - INFO - Starting async job submit for node 69c9b3e5f2866272ea51f2fd 2026-03-31 23:21:23,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:21:23,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b3e5f2866272ea51f2fd HTTP/1.1" 200 None 2026-03-31 23:21:23,656 - lava_callback - DEBUG - Processing node 69c9b3e5f2866272ea51f2fd: baseline-arm-clabbe 2026-03-31 23:21:23,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b3e5f2866272ea51f2fd 2026-03-31 23:21:23,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:21:23,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b3e5f2866272ea51f2fd HTTP/1.1" 200 None 2026-03-31 23:21:23,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:21:23,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:21:23,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:21:23,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b3e5f2866272ea51f2fd HTTP/1.1" 200 None 2026-03-31 23:21:23,715 - lava_callback - ERROR - Error processing callback for node 69c9b3e5f2866272ea51f2fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53884 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50376 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48876 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:22:03,610 - lava_callback - INFO - Received callback for node 69c9b40bf2866272ea51f307 INFO: 172.19.0.1:48880 - "POST /node/69c9b40bf2866272ea51f307 HTTP/1.0" 202 Accepted 2026-03-31 23:22:03,616 - lava_callback - INFO - Starting async job submit for node 69c9b40bf2866272ea51f307 2026-03-31 23:22:03,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:22:03,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b40bf2866272ea51f307 HTTP/1.1" 200 None 2026-03-31 23:22:03,631 - lava_callback - DEBUG - Processing node 69c9b40bf2866272ea51f307: baseline-arm-clabbe 2026-03-31 23:22:03,631 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b40bf2866272ea51f307 2026-03-31 23:22:03,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:22:03,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b40bf2866272ea51f307 HTTP/1.1" 200 None 2026-03-31 23:22:03,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:22:03,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:22:03,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:22:03,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b40bf2866272ea51f307 HTTP/1.1" 200 None 2026-03-31 23:22:03,728 - lava_callback - ERROR - Error processing callback for node 69c9b40bf2866272ea51f307: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48886 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56498 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38414 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60722 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:22:43,711 - lava_callback - INFO - Received callback for node 69c9b42ff2866272ea51f314 INFO: 172.19.0.1:37720 - "POST /node/69c9b42ff2866272ea51f314 HTTP/1.0" 202 Accepted 2026-03-31 23:22:43,718 - lava_callback - INFO - Starting async job submit for node 69c9b42ff2866272ea51f314 2026-03-31 23:22:43,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:22:43,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b42ff2866272ea51f314 HTTP/1.1" 200 None 2026-03-31 23:22:43,735 - lava_callback - DEBUG - Processing node 69c9b42ff2866272ea51f314: baseline-arm-clabbe 2026-03-31 23:22:43,735 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b42ff2866272ea51f314 2026-03-31 23:22:43,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:22:43,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b42ff2866272ea51f314 HTTP/1.1" 200 None 2026-03-31 23:22:43,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:22:43,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:22:43,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:22:43,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b42ff2866272ea51f314 HTTP/1.1" 200 None 2026-03-31 23:22:43,805 - lava_callback - ERROR - Error processing callback for node 69c9b42ff2866272ea51f314: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37730 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60240 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:23:03,639 - lava_callback - INFO - Received callback for node 69c9b454f2866272ea51f321 INFO: 172.19.0.1:60256 - "POST /node/69c9b454f2866272ea51f321 HTTP/1.0" 202 Accepted 2026-03-31 23:23:03,648 - lava_callback - INFO - Starting async job submit for node 69c9b454f2866272ea51f321 2026-03-31 23:23:03,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:23:03,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b454f2866272ea51f321 HTTP/1.1" 200 None 2026-03-31 23:23:03,665 - lava_callback - DEBUG - Processing node 69c9b454f2866272ea51f321: baseline-arm-clabbe 2026-03-31 23:23:03,665 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b454f2866272ea51f321 2026-03-31 23:23:03,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:23:03,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b454f2866272ea51f321 HTTP/1.1" 200 None 2026-03-31 23:23:03,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:23:03,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:23:03,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:23:03,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b454f2866272ea51f321 HTTP/1.1" 200 None 2026-03-31 23:23:03,725 - lava_callback - ERROR - Error processing callback for node 69c9b454f2866272ea51f321: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60272 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36634 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:23:43,730 - lava_callback - INFO - Received callback for node 69c9b478f2866272ea51f32f INFO: 172.19.0.1:48898 - "POST /node/69c9b478f2866272ea51f32f HTTP/1.0" 202 Accepted 2026-03-31 23:23:43,738 - lava_callback - INFO - Starting async job submit for node 69c9b478f2866272ea51f32f 2026-03-31 23:23:43,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:23:43,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b478f2866272ea51f32f HTTP/1.1" 200 None 2026-03-31 23:23:43,769 - lava_callback - DEBUG - Processing node 69c9b478f2866272ea51f32f: baseline-arm-clabbe 2026-03-31 23:23:43,769 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b478f2866272ea51f32f 2026-03-31 23:23:43,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:23:43,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b478f2866272ea51f32f HTTP/1.1" 200 None 2026-03-31 23:23:43,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:23:43,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:23:43,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:23:43,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b478f2866272ea51f32f HTTP/1.1" 200 None 2026-03-31 23:23:43,833 - lava_callback - ERROR - Error processing callback for node 69c9b478f2866272ea51f32f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42366 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41004 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:24:23,708 - lava_callback - INFO - Received callback for node 69c9b49df2866272ea51f337 INFO: 172.19.0.1:50106 - "POST /node/69c9b49df2866272ea51f337 HTTP/1.0" 202 Accepted 2026-03-31 23:24:23,717 - lava_callback - INFO - Starting async job submit for node 69c9b49df2866272ea51f337 2026-03-31 23:24:23,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:24:23,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b49df2866272ea51f337 HTTP/1.1" 200 None 2026-03-31 23:24:23,740 - lava_callback - DEBUG - Processing node 69c9b49df2866272ea51f337: baseline-arm-clabbe 2026-03-31 23:24:23,740 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b49df2866272ea51f337 2026-03-31 23:24:23,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:24:23,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b49df2866272ea51f337 HTTP/1.1" 200 None 2026-03-31 23:24:23,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:24:23,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:24:23,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:24:23,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b49df2866272ea51f337 HTTP/1.1" 200 None 2026-03-31 23:24:23,803 - lava_callback - ERROR - Error processing callback for node 69c9b49df2866272ea51f337: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45922 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:25:03,813 - lava_callback - INFO - Received callback for node 69c9b4c4f2866272ea51f345 INFO: 172.19.0.1:45930 - "POST /node/69c9b4c4f2866272ea51f345 HTTP/1.0" 202 Accepted 2026-03-31 23:25:03,819 - lava_callback - INFO - Starting async job submit for node 69c9b4c4f2866272ea51f345 2026-03-31 23:25:03,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:25:03,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b4c4f2866272ea51f345 HTTP/1.1" 200 None 2026-03-31 23:25:03,842 - lava_callback - DEBUG - Processing node 69c9b4c4f2866272ea51f345: baseline-arm-clabbe 2026-03-31 23:25:03,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b4c4f2866272ea51f345 2026-03-31 23:25:03,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:25:03,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b4c4f2866272ea51f345 HTTP/1.1" 200 None 2026-03-31 23:25:03,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:25:03,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:25:03,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:25:03,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b4c4f2866272ea51f345 HTTP/1.1" 200 None 2026-03-31 23:25:03,907 - lava_callback - ERROR - Error processing callback for node 69c9b4c4f2866272ea51f345: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45944 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51810 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54750 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:25:43,766 - lava_callback - INFO - Received callback for node 69c9b4e9f2866272ea51f350 INFO: 172.19.0.1:40340 - "POST /node/69c9b4e9f2866272ea51f350 HTTP/1.0" 202 Accepted 2026-03-31 23:25:43,772 - lava_callback - INFO - Starting async job submit for node 69c9b4e9f2866272ea51f350 2026-03-31 23:25:43,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:25:43,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b4e9f2866272ea51f350 HTTP/1.1" 200 None 2026-03-31 23:25:43,789 - lava_callback - DEBUG - Processing node 69c9b4e9f2866272ea51f350: baseline-arm-clabbe 2026-03-31 23:25:43,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b4e9f2866272ea51f350 2026-03-31 23:25:43,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:25:43,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b4e9f2866272ea51f350 HTTP/1.1" 200 None 2026-03-31 23:25:43,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:25:43,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:25:43,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:25:43,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b4e9f2866272ea51f350 HTTP/1.1" 200 None 2026-03-31 23:25:43,842 - lava_callback - ERROR - Error processing callback for node 69c9b4e9f2866272ea51f350: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:40348 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37860 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50120 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:26:23,862 - lava_callback - INFO - Received callback for node 69c9b50df2866272ea51f361 INFO: 172.19.0.1:47240 - "POST /node/69c9b50df2866272ea51f361 HTTP/1.0" 202 Accepted 2026-03-31 23:26:23,868 - lava_callback - INFO - Starting async job submit for node 69c9b50df2866272ea51f361 2026-03-31 23:26:23,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:26:23,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b50df2866272ea51f361 HTTP/1.1" 200 None 2026-03-31 23:26:23,883 - lava_callback - DEBUG - Processing node 69c9b50df2866272ea51f361: baseline-arm-clabbe 2026-03-31 23:26:23,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b50df2866272ea51f361 2026-03-31 23:26:23,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:26:23,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b50df2866272ea51f361 HTTP/1.1" 200 None 2026-03-31 23:26:23,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:26:23,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:26:23,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:26:23,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b50df2866272ea51f361 HTTP/1.1" 200 None 2026-03-31 23:26:23,939 - lava_callback - ERROR - Error processing callback for node 69c9b50df2866272ea51f361: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51204 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38538 - "GET /node/69296736f7abe3da4c22aae1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.19.0.1:38548 - "GET /node/69bdf4d0d54b5122738c6a9c%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-31 23:26:54,452 - lava_callback - INFO - Received callback for node 69c9b536f2866272ea51f36d INFO: 172.19.0.1:33398 - "POST /node/69c9b536f2866272ea51f36d HTTP/1.0" 202 Accepted 2026-03-31 23:26:54,457 - lava_callback - INFO - Starting async job submit for node 69c9b536f2866272ea51f36d 2026-03-31 23:26:54,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:26:54,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b536f2866272ea51f36d HTTP/1.1" 200 None 2026-03-31 23:26:54,482 - lava_callback - DEBUG - Processing node 69c9b536f2866272ea51f36d: baseline-arm64-baylibre 2026-03-31 23:26:54,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b536f2866272ea51f36d 2026-03-31 23:26:54,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:26:54,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b536f2866272ea51f36d HTTP/1.1" 200 None 2026-03-31 23:26:54,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:26:54,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:26:54,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:26:54,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b536f2866272ea51f36d HTTP/1.1" 200 None 2026-03-31 23:26:54,553 - lava_callback - ERROR - Error processing callback for node 69c9b536f2866272ea51f36d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33408 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51284 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52918 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54236 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:28:14,520 - lava_callback - INFO - Received callback for node 69c9b582f2866272ea51f376 INFO: 172.19.0.1:33216 - "POST /node/69c9b582f2866272ea51f376 HTTP/1.0" 202 Accepted 2026-03-31 23:28:14,526 - lava_callback - INFO - Starting async job submit for node 69c9b582f2866272ea51f376 2026-03-31 23:28:14,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:28:14,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b582f2866272ea51f376 HTTP/1.1" 200 None 2026-03-31 23:28:14,542 - lava_callback - DEBUG - Processing node 69c9b582f2866272ea51f376: baseline-arm64-baylibre 2026-03-31 23:28:14,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b582f2866272ea51f376 2026-03-31 23:28:14,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:28:14,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b582f2866272ea51f376 HTTP/1.1" 200 None 2026-03-31 23:28:14,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:28:14,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:28:14,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:28:14,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b582f2866272ea51f376 HTTP/1.1" 200 None 2026-03-31 23:28:14,594 - lava_callback - ERROR - Error processing callback for node 69c9b582f2866272ea51f376: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:33228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33230 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37404 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38144 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:41664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52836 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:29:34,690 - lava_callback - INFO - Received callback for node 69c9b5d3f2866272ea51f384 INFO: 172.19.0.1:60282 - "POST /node/69c9b5d3f2866272ea51f384 HTTP/1.0" 202 Accepted 2026-03-31 23:29:34,696 - lava_callback - INFO - Starting async job submit for node 69c9b5d3f2866272ea51f384 2026-03-31 23:29:34,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:29:34,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b5d3f2866272ea51f384 HTTP/1.1" 200 None 2026-03-31 23:29:34,712 - lava_callback - DEBUG - Processing node 69c9b5d3f2866272ea51f384: baseline-arm64-baylibre 2026-03-31 23:29:34,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b5d3f2866272ea51f384 2026-03-31 23:29:34,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:29:34,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b5d3f2866272ea51f384 HTTP/1.1" 200 None 2026-03-31 23:29:34,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:29:34,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:29:34,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:29:34,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b5d3f2866272ea51f384 HTTP/1.1" 200 None 2026-03-31 23:29:34,769 - lava_callback - ERROR - Error processing callback for node 69c9b5d3f2866272ea51f384: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37814 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58592 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53966 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:30:54,779 - lava_callback - INFO - Received callback for node 69c9b621f2866272ea51f393 INFO: 172.19.0.1:57236 - "POST /node/69c9b621f2866272ea51f393 HTTP/1.0" 202 Accepted 2026-03-31 23:30:54,785 - lava_callback - INFO - Starting async job submit for node 69c9b621f2866272ea51f393 2026-03-31 23:30:54,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:30:54,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b621f2866272ea51f393 HTTP/1.1" 200 None 2026-03-31 23:30:54,823 - lava_callback - DEBUG - Processing node 69c9b621f2866272ea51f393: baseline-arm64-baylibre 2026-03-31 23:30:54,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b621f2866272ea51f393 2026-03-31 23:30:54,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:30:54,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b621f2866272ea51f393 HTTP/1.1" 200 None 2026-03-31 23:30:54,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:30:54,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:30:54,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:30:54,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b621f2866272ea51f393 HTTP/1.1" 200 None 2026-03-31 23:30:54,883 - lava_callback - ERROR - Error processing callback for node 69c9b621f2866272ea51f393: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47964 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35184 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55166 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45602 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:32:14,741 - lava_callback - INFO - Received callback for node 69c9b66df2866272ea51f39e INFO: 172.19.0.1:58530 - "POST /node/69c9b66df2866272ea51f39e HTTP/1.0" 202 Accepted 2026-03-31 23:32:14,747 - lava_callback - INFO - Starting async job submit for node 69c9b66df2866272ea51f39e 2026-03-31 23:32:14,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:32:14,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b66df2866272ea51f39e HTTP/1.1" 200 None 2026-03-31 23:32:14,772 - lava_callback - DEBUG - Processing node 69c9b66df2866272ea51f39e: baseline-arm64-baylibre 2026-03-31 23:32:14,772 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b66df2866272ea51f39e 2026-03-31 23:32:14,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:32:14,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b66df2866272ea51f39e HTTP/1.1" 200 None 2026-03-31 23:32:14,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:32:14,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:32:14,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:32:14,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b66df2866272ea51f39e HTTP/1.1" 200 None 2026-03-31 23:32:14,827 - lava_callback - ERROR - Error processing callback for node 69c9b66df2866272ea51f39e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:58546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58124 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44406 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53784 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:33:34,803 - lava_callback - INFO - Received callback for node 69c9b6bcf2866272ea51f3aa INFO: 172.19.0.1:52270 - "POST /node/69c9b6bcf2866272ea51f3aa HTTP/1.0" 202 Accepted 2026-03-31 23:33:34,809 - lava_callback - INFO - Starting async job submit for node 69c9b6bcf2866272ea51f3aa 2026-03-31 23:33:34,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:33:34,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b6bcf2866272ea51f3aa HTTP/1.1" 200 None 2026-03-31 23:33:34,848 - lava_callback - DEBUG - Processing node 69c9b6bcf2866272ea51f3aa: baseline-arm64-baylibre 2026-03-31 23:33:34,849 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b6bcf2866272ea51f3aa 2026-03-31 23:33:34,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:33:34,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b6bcf2866272ea51f3aa HTTP/1.1" 200 None 2026-03-31 23:33:34,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:33:34,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:33:34,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:33:34,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b6bcf2866272ea51f3aa HTTP/1.1" 200 None 2026-03-31 23:33:34,917 - lava_callback - ERROR - Error processing callback for node 69c9b6bcf2866272ea51f3aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48666 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41558 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55716 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:34:54,889 - lava_callback - INFO - Received callback for node 69c9b70bf2866272ea51f3bd INFO: 172.19.0.1:45434 - "POST /node/69c9b70bf2866272ea51f3bd HTTP/1.0" 202 Accepted 2026-03-31 23:34:54,894 - lava_callback - INFO - Starting async job submit for node 69c9b70bf2866272ea51f3bd 2026-03-31 23:34:54,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:34:54,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b70bf2866272ea51f3bd HTTP/1.1" 200 None 2026-03-31 23:34:54,910 - lava_callback - DEBUG - Processing node 69c9b70bf2866272ea51f3bd: baseline-arm64-baylibre 2026-03-31 23:34:54,911 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b70bf2866272ea51f3bd 2026-03-31 23:34:54,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:34:54,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b70bf2866272ea51f3bd HTTP/1.1" 200 None 2026-03-31 23:34:54,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:34:54,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:34:54,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:34:54,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b70bf2866272ea51f3bd HTTP/1.1" 200 None 2026-03-31 23:34:54,990 - lava_callback - ERROR - Error processing callback for node 69c9b70bf2866272ea51f3bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:45444 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40562 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40564 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36550 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60548 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60764 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:35:54,870 - lava_callback - INFO - Received callback for node 69c9b759f2866272ea51f3c4 INFO: 172.19.0.1:56412 - "POST /node/69c9b759f2866272ea51f3c4 HTTP/1.0" 202 Accepted 2026-03-31 23:35:54,875 - lava_callback - INFO - Starting async job submit for node 69c9b759f2866272ea51f3c4 2026-03-31 23:35:54,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:35:54,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b759f2866272ea51f3c4 HTTP/1.1" 200 None 2026-03-31 23:35:54,891 - lava_callback - DEBUG - Processing node 69c9b759f2866272ea51f3c4: baseline-arm64-baylibre 2026-03-31 23:35:54,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b759f2866272ea51f3c4 2026-03-31 23:35:54,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:35:54,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b759f2866272ea51f3c4 HTTP/1.1" 200 None 2026-03-31 23:35:54,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:35:54,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:35:54,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:35:54,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b759f2866272ea51f3c4 HTTP/1.1" 200 None 2026-03-31 23:35:54,952 - lava_callback - ERROR - Error processing callback for node 69c9b759f2866272ea51f3c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59614 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35850 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43188 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36448 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34946 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 23:37:15,246 - lava_callback - INFO - Received callback for node 69c9b7a7f2866272ea51f3d6 INFO: 172.19.0.1:34954 - "POST /node/69c9b7a7f2866272ea51f3d6 HTTP/1.0" 202 Accepted 2026-03-31 23:37:15,252 - lava_callback - INFO - Starting async job submit for node 69c9b7a7f2866272ea51f3d6 2026-03-31 23:37:15,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:37:15,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b7a7f2866272ea51f3d6 HTTP/1.1" 200 None 2026-03-31 23:37:15,268 - lava_callback - DEBUG - Processing node 69c9b7a7f2866272ea51f3d6: baseline-arm64-baylibre 2026-03-31 23:37:15,269 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b7a7f2866272ea51f3d6 2026-03-31 23:37:15,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:37:15,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b7a7f2866272ea51f3d6 HTTP/1.1" 200 None 2026-03-31 23:37:15,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:37:15,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:37:15,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:37:15,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b7a7f2866272ea51f3d6 HTTP/1.1" 200 None 2026-03-31 23:37:15,324 - lava_callback - ERROR - Error processing callback for node 69c9b7a7f2866272ea51f3d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34960 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54050 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55122 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46166 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:38:44,374 - lava_callback - INFO - Received callback for node 69c9b7eef2866272ea51f3dc INFO: 172.19.0.1:51498 - "POST /node/69c9b7eef2866272ea51f3dc HTTP/1.0" 202 Accepted 2026-03-31 23:38:44,380 - lava_callback - INFO - Starting async job submit for node 69c9b7eef2866272ea51f3dc 2026-03-31 23:38:44,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:38:44,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b7eef2866272ea51f3dc HTTP/1.1" 200 None 2026-03-31 23:38:44,397 - lava_callback - DEBUG - Processing node 69c9b7eef2866272ea51f3dc: baseline-arm-clabbe 2026-03-31 23:38:44,397 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b7eef2866272ea51f3dc 2026-03-31 23:38:44,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:38:44,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b7eef2866272ea51f3dc HTTP/1.1" 200 None 2026-03-31 23:38:44,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:38:44,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:38:44,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:38:44,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b7eef2866272ea51f3dc HTTP/1.1" 200 None 2026-03-31 23:38:44,469 - lava_callback - ERROR - Error processing callback for node 69c9b7eef2866272ea51f3dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:51510 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49578 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46442 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40378 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33080 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33082 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:52894 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:40:24,748 - lava_callback - INFO - Received callback for node 69c9b860f2866272ea51f3f5 INFO: 172.19.0.1:50504 - "POST /node/69c9b860f2866272ea51f3f5 HTTP/1.0" 202 Accepted 2026-03-31 23:40:24,754 - lava_callback - INFO - Starting async job submit for node 69c9b860f2866272ea51f3f5 2026-03-31 23:40:24,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:40:24,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b860f2866272ea51f3f5 HTTP/1.1" 200 None 2026-03-31 23:40:24,771 - lava_callback - DEBUG - Processing node 69c9b860f2866272ea51f3f5: baseline-arm-clabbe 2026-03-31 23:40:24,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b860f2866272ea51f3f5 2026-03-31 23:40:24,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:40:24,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b860f2866272ea51f3f5 HTTP/1.1" 200 None 2026-03-31 23:40:24,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:40:24,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-31 23:40:24,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:40:24,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b860f2866272ea51f3f5 HTTP/1.1" 200 None 2026-03-31 23:40:24,830 - lava_callback - ERROR - Error processing callback for node 69c9b860f2866272ea51f3f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34914 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34920 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:41:15,154 - lava_callback - INFO - Received callback for node 69c9b88cf2866272ea51f404 INFO: 172.19.0.1:38964 - "POST /node/69c9b88cf2866272ea51f404 HTTP/1.0" 202 Accepted 2026-03-31 23:41:15,160 - lava_callback - INFO - Starting async job submit for node 69c9b88cf2866272ea51f404 2026-03-31 23:41:15,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.19.0.1:38970 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 23:41:15,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b88cf2866272ea51f404 HTTP/1.1" 200 None 2026-03-31 23:41:15,177 - lava_callback - DEBUG - Processing node 69c9b88cf2866272ea51f404: baseline-arm64-baylibre 2026-03-31 23:41:15,177 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b88cf2866272ea51f404 2026-03-31 23:41:15,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:41:15,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b88cf2866272ea51f404 HTTP/1.1" 200 None 2026-03-31 23:41:15,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:41:15,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:41:15,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:41:15,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b88cf2866272ea51f404 HTTP/1.1" 200 None 2026-03-31 23:41:15,232 - lava_callback - ERROR - Error processing callback for node 69c9b88cf2866272ea51f404: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:38986 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43758 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36452 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:42:44,596 - lava_callback - INFO - Received callback for node 69c9b8e2f2866272ea51f414 INFO: 172.19.0.1:59414 - "POST /node/69c9b8e2f2866272ea51f414 HTTP/1.0" 202 Accepted 2026-03-31 23:42:44,602 - lava_callback - INFO - Starting async job submit for node 69c9b8e2f2866272ea51f414 2026-03-31 23:42:44,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:42:44,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b8e2f2866272ea51f414 HTTP/1.1" 200 None 2026-03-31 23:42:44,619 - lava_callback - DEBUG - Processing node 69c9b8e2f2866272ea51f414: baseline-arm64-clabbe 2026-03-31 23:42:44,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b8e2f2866272ea51f414 2026-03-31 23:42:44,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:42:44,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b8e2f2866272ea51f414 HTTP/1.1" 200 None 2026-03-31 23:42:44,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:42:44,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:42:44,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:42:44,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b8e2f2866272ea51f414 HTTP/1.1" 200 None 2026-03-31 23:42:44,674 - lava_callback - ERROR - Error processing callback for node 69c9b8e2f2866272ea51f414: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59416 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:58912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46092 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:43:55,301 - lava_callback - INFO - Received callback for node 69c9b92cf2866272ea51f429 INFO: 172.19.0.1:48112 - "POST /node/69c9b92cf2866272ea51f429 HTTP/1.0" 202 Accepted 2026-03-31 23:43:55,306 - lava_callback - INFO - Starting async job submit for node 69c9b92cf2866272ea51f429 2026-03-31 23:43:55,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:43:55,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b92cf2866272ea51f429 HTTP/1.1" 200 None 2026-03-31 23:43:55,322 - lava_callback - DEBUG - Processing node 69c9b92cf2866272ea51f429: baseline-arm64-baylibre 2026-03-31 23:43:55,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b92cf2866272ea51f429 2026-03-31 23:43:55,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:43:55,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b92cf2866272ea51f429 HTTP/1.1" 200 None 2026-03-31 23:43:55,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:43:55,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:43:55,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:43:55,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b92cf2866272ea51f429 HTTP/1.1" 200 None 2026-03-31 23:43:55,412 - lava_callback - ERROR - Error processing callback for node 69c9b92cf2866272ea51f429: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51584 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59032 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35264 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38054 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:45:24,682 - lava_callback - INFO - Received callback for node 69c9b982f2866272ea51f442 INFO: 172.19.0.1:55224 - "POST /node/69c9b982f2866272ea51f442 HTTP/1.0" 202 Accepted 2026-03-31 23:45:24,688 - lava_callback - INFO - Starting async job submit for node 69c9b982f2866272ea51f442 2026-03-31 23:45:24,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:45:24,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b982f2866272ea51f442 HTTP/1.1" 200 None 2026-03-31 23:45:24,704 - lava_callback - DEBUG - Processing node 69c9b982f2866272ea51f442: baseline-arm64-clabbe 2026-03-31 23:45:24,704 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b982f2866272ea51f442 2026-03-31 23:45:24,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:45:24,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b982f2866272ea51f442 HTTP/1.1" 200 None 2026-03-31 23:45:24,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:45:24,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:45:24,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:45:24,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b982f2866272ea51f442 HTTP/1.1" 200 None 2026-03-31 23:45:24,767 - lava_callback - ERROR - Error processing callback for node 69c9b982f2866272ea51f442: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55238 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53670 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:32956 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36662 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59330 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:46:44,694 - lava_callback - INFO - Received callback for node 69c9b9d3f2866272ea51f452 INFO: 172.19.0.1:57080 - "POST /node/69c9b9d3f2866272ea51f452 HTTP/1.0" 202 Accepted 2026-03-31 23:46:44,699 - lava_callback - INFO - Starting async job submit for node 69c9b9d3f2866272ea51f452 2026-03-31 23:46:44,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:46:44,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b9d3f2866272ea51f452 HTTP/1.1" 200 None 2026-03-31 23:46:44,886 - lava_callback - DEBUG - Processing node 69c9b9d3f2866272ea51f452: baseline-arm64-clabbe 2026-03-31 23:46:44,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9b9d3f2866272ea51f452 2026-03-31 23:46:44,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:46:44,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9b9d3f2866272ea51f452 HTTP/1.1" 200 None 2026-03-31 23:46:44,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:46:44,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:46:44,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:46:44,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9b9d3f2866272ea51f452 HTTP/1.1" 200 None 2026-03-31 23:46:44,977 - lava_callback - ERROR - Error processing callback for node 69c9b9d3f2866272ea51f452: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:57096 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41182 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50020 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46654 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37952 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:48:04,785 - lava_callback - INFO - Received callback for node 69c9ba22f2866272ea51f468 INFO: 172.19.0.1:37960 - "POST /node/69c9ba22f2866272ea51f468 HTTP/1.0" 202 Accepted 2026-03-31 23:48:04,790 - lava_callback - INFO - Starting async job submit for node 69c9ba22f2866272ea51f468 2026-03-31 23:48:04,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:48:04,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ba22f2866272ea51f468 HTTP/1.1" 200 None 2026-03-31 23:48:04,807 - lava_callback - DEBUG - Processing node 69c9ba22f2866272ea51f468: baseline-arm64-clabbe 2026-03-31 23:48:04,807 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ba22f2866272ea51f468 2026-03-31 23:48:04,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:48:04,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ba22f2866272ea51f468 HTTP/1.1" 200 None 2026-03-31 23:48:04,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:48:04,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:48:04,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:48:04,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ba22f2866272ea51f468 HTTP/1.1" 200 None 2026-03-31 23:48:04,863 - lava_callback - ERROR - Error processing callback for node 69c9ba22f2866272ea51f468: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53300 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47544 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51724 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35872 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57748 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:49:24,919 - lava_callback - INFO - Received callback for node 69c9ba71f2866272ea51f473 INFO: 172.19.0.1:44308 - "POST /node/69c9ba71f2866272ea51f473 HTTP/1.0" 202 Accepted 2026-03-31 23:49:24,925 - lava_callback - INFO - Starting async job submit for node 69c9ba71f2866272ea51f473 2026-03-31 23:49:24,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:49:24,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ba71f2866272ea51f473 HTTP/1.1" 200 None 2026-03-31 23:49:24,941 - lava_callback - DEBUG - Processing node 69c9ba71f2866272ea51f473: baseline-arm64-clabbe 2026-03-31 23:49:24,941 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9ba71f2866272ea51f473 2026-03-31 23:49:24,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:49:24,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9ba71f2866272ea51f473 HTTP/1.1" 200 None 2026-03-31 23:49:24,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:49:24,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:49:24,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:49:24,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9ba71f2866272ea51f473 HTTP/1.1" 200 None 2026-03-31 23:49:24,992 - lava_callback - ERROR - Error processing callback for node 69c9ba71f2866272ea51f473: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52276 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53776 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59186 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56130 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56136 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:56138 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33646 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50396 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:50:44,912 - lava_callback - INFO - Received callback for node 69c9bac1f2866272ea51f483 INFO: 172.19.0.1:37300 - "POST /node/69c9bac1f2866272ea51f483 HTTP/1.0" 202 Accepted 2026-03-31 23:50:44,917 - lava_callback - INFO - Starting async job submit for node 69c9bac1f2866272ea51f483 2026-03-31 23:50:44,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:50:44,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bac1f2866272ea51f483 HTTP/1.1" 200 None 2026-03-31 23:50:44,933 - lava_callback - DEBUG - Processing node 69c9bac1f2866272ea51f483: baseline-arm64-clabbe 2026-03-31 23:50:44,933 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bac1f2866272ea51f483 2026-03-31 23:50:44,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:50:44,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bac1f2866272ea51f483 HTTP/1.1" 200 None 2026-03-31 23:50:44,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:50:44,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:50:44,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:50:44,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bac1f2866272ea51f483 HTTP/1.1" 200 None 2026-03-31 23:50:44,986 - lava_callback - ERROR - Error processing callback for node 69c9bac1f2866272ea51f483: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37308 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42572 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33142 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33156 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:49488 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51104 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50046 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:52:04,993 - lava_callback - INFO - Received callback for node 69c9bb11f2866272ea51f492 INFO: 172.19.0.1:50056 - "POST /node/69c9bb11f2866272ea51f492 HTTP/1.0" 202 Accepted 2026-03-31 23:52:05,002 - lava_callback - INFO - Starting async job submit for node 69c9bb11f2866272ea51f492 2026-03-31 23:52:05,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:52:05,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bb11f2866272ea51f492 HTTP/1.1" 200 None 2026-03-31 23:52:05,019 - lava_callback - DEBUG - Processing node 69c9bb11f2866272ea51f492: baseline-arm64-clabbe 2026-03-31 23:52:05,019 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bb11f2866272ea51f492 2026-03-31 23:52:05,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:52:05,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bb11f2866272ea51f492 HTTP/1.1" 200 None 2026-03-31 23:52:05,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:52:05,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:52:05,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:52:05,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bb11f2866272ea51f492 HTTP/1.1" 200 None 2026-03-31 23:52:05,142 - lava_callback - ERROR - Error processing callback for node 69c9bb11f2866272ea51f492: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51934 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58738 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38804 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33996 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34000 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:36854 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:53:25,068 - lava_callback - INFO - Received callback for node 69c9bb5ff2866272ea51f49c INFO: 172.19.0.1:55232 - "POST /node/69c9bb5ff2866272ea51f49c HTTP/1.0" 202 Accepted 2026-03-31 23:53:25,073 - lava_callback - INFO - Starting async job submit for node 69c9bb5ff2866272ea51f49c 2026-03-31 23:53:25,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:53:25,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bb5ff2866272ea51f49c HTTP/1.1" 200 None 2026-03-31 23:53:25,092 - lava_callback - DEBUG - Processing node 69c9bb5ff2866272ea51f49c: baseline-arm64-clabbe 2026-03-31 23:53:25,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bb5ff2866272ea51f49c 2026-03-31 23:53:25,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:53:25,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bb5ff2866272ea51f49c HTTP/1.1" 200 None 2026-03-31 23:53:25,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:53:25,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:53:25,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:53:25,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bb5ff2866272ea51f49c HTTP/1.1" 200 None 2026-03-31 23:53:25,154 - lava_callback - ERROR - Error processing callback for node 69c9bb5ff2866272ea51f49c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52908 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38254 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:42760 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:54:25,084 - lava_callback - INFO - Received callback for node 69c9bbaef2866272ea51f4ae INFO: 172.19.0.1:52282 - "POST /node/69c9bbaef2866272ea51f4ae HTTP/1.0" 202 Accepted 2026-03-31 23:54:25,089 - lava_callback - INFO - Starting async job submit for node 69c9bbaef2866272ea51f4ae 2026-03-31 23:54:25,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:54:25,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bbaef2866272ea51f4ae HTTP/1.1" 200 None 2026-03-31 23:54:25,106 - lava_callback - DEBUG - Processing node 69c9bbaef2866272ea51f4ae: baseline-arm64-clabbe 2026-03-31 23:54:25,106 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bbaef2866272ea51f4ae 2026-03-31 23:54:25,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:54:25,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bbaef2866272ea51f4ae HTTP/1.1" 200 None 2026-03-31 23:54:25,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:54:25,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:54:25,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:54:25,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bbaef2866272ea51f4ae HTTP/1.1" 200 None 2026-03-31 23:54:25,166 - lava_callback - ERROR - Error processing callback for node 69c9bbaef2866272ea51f4ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52286 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33076 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59962 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:55:45,163 - lava_callback - INFO - Received callback for node 69c9bbfdf2866272ea51f4c4 INFO: 172.19.0.1:34256 - "POST /node/69c9bbfdf2866272ea51f4c4 HTTP/1.0" 202 Accepted 2026-03-31 23:55:45,169 - lava_callback - INFO - Starting async job submit for node 69c9bbfdf2866272ea51f4c4 2026-03-31 23:55:45,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:55:45,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bbfdf2866272ea51f4c4 HTTP/1.1" 200 None 2026-03-31 23:55:45,187 - lava_callback - DEBUG - Processing node 69c9bbfdf2866272ea51f4c4: baseline-arm64-clabbe 2026-03-31 23:55:45,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bbfdf2866272ea51f4c4 2026-03-31 23:55:45,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:55:45,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bbfdf2866272ea51f4c4 HTTP/1.1" 200 None 2026-03-31 23:55:45,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:55:45,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:55:45,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:55:45,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bbfdf2866272ea51f4c4 HTTP/1.1" 200 None 2026-03-31 23:55:45,239 - lava_callback - ERROR - Error processing callback for node 69c9bbfdf2866272ea51f4c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57270 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:44360 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38950 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48328 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38982 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:57:05,184 - lava_callback - INFO - Received callback for node 69c9bc4cf2866272ea51f4cc INFO: 172.19.0.1:38996 - "POST /node/69c9bc4cf2866272ea51f4cc HTTP/1.0" 202 Accepted 2026-03-31 23:57:05,190 - lava_callback - INFO - Starting async job submit for node 69c9bc4cf2866272ea51f4cc 2026-03-31 23:57:05,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:57:05,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bc4cf2866272ea51f4cc HTTP/1.1" 200 None 2026-03-31 23:57:05,206 - lava_callback - DEBUG - Processing node 69c9bc4cf2866272ea51f4cc: baseline-arm64-clabbe 2026-03-31 23:57:05,207 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bc4cf2866272ea51f4cc 2026-03-31 23:57:05,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:57:05,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bc4cf2866272ea51f4cc HTTP/1.1" 200 None 2026-03-31 23:57:05,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:57:05,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:57:05,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:57:05,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bc4cf2866272ea51f4cc HTTP/1.1" 200 None 2026-03-31 23:57:05,257 - lava_callback - ERROR - Error processing callback for node 69c9bc4cf2866272ea51f4cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:39012 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45398 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49992 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51768 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46014 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56676 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-31 23:58:16,134 - lava_callback - INFO - Received callback for node 69c9bc94f2866272ea51f4d9 INFO: 172.19.0.1:56688 - "POST /node/69c9bc94f2866272ea51f4d9 HTTP/1.0" 202 Accepted 2026-03-31 23:58:16,140 - lava_callback - INFO - Starting async job submit for node 69c9bc94f2866272ea51f4d9 2026-03-31 23:58:16,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:58:16,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bc94f2866272ea51f4d9 HTTP/1.1" 200 None 2026-03-31 23:58:16,156 - lava_callback - DEBUG - Processing node 69c9bc94f2866272ea51f4d9: baseline-arm64-baylibre 2026-03-31 23:58:16,157 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bc94f2866272ea51f4d9 2026-03-31 23:58:16,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:58:16,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bc94f2866272ea51f4d9 HTTP/1.1" 200 None 2026-03-31 23:58:16,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:58:16,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-31 23:58:16,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:58:16,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bc94f2866272ea51f4d9 HTTP/1.1" 200 None 2026-03-31 23:58:16,242 - lava_callback - ERROR - Error processing callback for node 69c9bc94f2866272ea51f4d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:56690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53840 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46454 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46460 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35828 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58476 - "GET / HTTP/1.0" 200 OK 2026-03-31 23:59:45,231 - lava_callback - INFO - Received callback for node 69c9bceaf2866272ea51f4e6 INFO: 172.19.0.1:53736 - "POST /node/69c9bceaf2866272ea51f4e6 HTTP/1.0" 202 Accepted 2026-03-31 23:59:45,237 - lava_callback - INFO - Starting async job submit for node 69c9bceaf2866272ea51f4e6 2026-03-31 23:59:45,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:59:45,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bceaf2866272ea51f4e6 HTTP/1.1" 200 None 2026-03-31 23:59:45,257 - lava_callback - DEBUG - Processing node 69c9bceaf2866272ea51f4e6: baseline-arm64-clabbe 2026-03-31 23:59:45,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bceaf2866272ea51f4e6 2026-03-31 23:59:45,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:59:45,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bceaf2866272ea51f4e6 HTTP/1.1" 200 None 2026-03-31 23:59:45,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:59:45,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae1602375b4506df899e HTTP/1.1" 200 None 2026-03-31 23:59:45,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-31 23:59:45,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bceaf2866272ea51f4e6 HTTP/1.1" 200 None 2026-03-31 23:59:45,328 - lava_callback - ERROR - Error processing callback for node 69c9bceaf2866272ea51f4e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:53750 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35780 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46686 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:33708 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53258 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41298 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:00:56,234 - lava_callback - INFO - Received callback for node 69c9bd36f2866272ea51f4ed INFO: 172.19.0.1:50098 - "POST /node/69c9bd36f2866272ea51f4ed HTTP/1.0" 202 Accepted 2026-04-01 00:00:56,241 - lava_callback - INFO - Starting async job submit for node 69c9bd36f2866272ea51f4ed 2026-04-01 00:00:56,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:00:56,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bd36f2866272ea51f4ed HTTP/1.1" 200 None 2026-04-01 00:00:56,265 - lava_callback - DEBUG - Processing node 69c9bd36f2866272ea51f4ed: baseline-arm64-baylibre 2026-04-01 00:00:56,265 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bd36f2866272ea51f4ed 2026-04-01 00:00:56,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:00:56,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bd36f2866272ea51f4ed HTTP/1.1" 200 None 2026-04-01 00:00:56,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:00:56,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-04-01 00:00:56,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:00:56,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bd36f2866272ea51f4ed HTTP/1.1" 200 None 2026-04-01 00:00:56,356 - lava_callback - ERROR - Error processing callback for node 69c9bd36f2866272ea51f4ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50110 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55596 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49580 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57718 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60916 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37958 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33400 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49608 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:43218 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35196 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36634 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:03:56,565 - lava_callback - INFO - Received callback for node 69c9bdd6f2866272ea51f500 INFO: 172.19.0.1:36644 - "POST /node/69c9bdd6f2866272ea51f500 HTTP/1.0" 202 Accepted 2026-04-01 00:03:56,571 - lava_callback - INFO - Starting async job submit for node 69c9bdd6f2866272ea51f500 2026-04-01 00:03:56,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:03:56,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bdd6f2866272ea51f500 HTTP/1.1" 200 None 2026-04-01 00:03:56,587 - lava_callback - DEBUG - Processing node 69c9bdd6f2866272ea51f500: baseline-arm64-baylibre 2026-04-01 00:03:56,587 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bdd6f2866272ea51f500 2026-04-01 00:03:56,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:03:56,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bdd6f2866272ea51f500 HTTP/1.1" 200 None 2026-04-01 00:03:56,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:03:56,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-04-01 00:03:56,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:03:56,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bdd6f2866272ea51f500 HTTP/1.1" 200 None 2026-04-01 00:03:56,639 - lava_callback - ERROR - Error processing callback for node 69c9bdd6f2866272ea51f500: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:50876 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50982 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38800 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:04:56,400 - lava_callback - INFO - Received callback for node 69c9be23f2866272ea51f50d INFO: 172.19.0.1:47422 - "POST /node/69c9be23f2866272ea51f50d HTTP/1.0" 202 Accepted 2026-04-01 00:04:56,409 - lava_callback - INFO - Starting async job submit for node 69c9be23f2866272ea51f50d 2026-04-01 00:04:56,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:04:56,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9be23f2866272ea51f50d HTTP/1.1" 200 None 2026-04-01 00:04:56,425 - lava_callback - DEBUG - Processing node 69c9be23f2866272ea51f50d: baseline-arm64-baylibre 2026-04-01 00:04:56,425 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9be23f2866272ea51f50d 2026-04-01 00:04:56,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:04:56,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9be23f2866272ea51f50d HTTP/1.1" 200 None 2026-04-01 00:04:56,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:04:56,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-04-01 00:04:56,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:04:56,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9be23f2866272ea51f50d HTTP/1.1" 200 None 2026-04-01 00:04:56,477 - lava_callback - ERROR - Error processing callback for node 69c9be23f2866272ea51f50d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47438 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50762 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:50638 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34898 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54600 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57198 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:53128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:53130 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:06:16,636 - lava_callback - INFO - Received callback for node 69c9be75f2866272ea51f51b INFO: 172.19.0.1:53146 - "POST /node/69c9be75f2866272ea51f51b HTTP/1.0" 202 Accepted 2026-04-01 00:06:16,641 - lava_callback - INFO - Starting async job submit for node 69c9be75f2866272ea51f51b 2026-04-01 00:06:16,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:06:16,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9be75f2866272ea51f51b HTTP/1.1" 200 None 2026-04-01 00:06:16,658 - lava_callback - DEBUG - Processing node 69c9be75f2866272ea51f51b: baseline-arm64-baylibre 2026-04-01 00:06:16,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9be75f2866272ea51f51b 2026-04-01 00:06:16,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:06:16,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9be75f2866272ea51f51b HTTP/1.1" 200 None 2026-04-01 00:06:16,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:06:16,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-04-01 00:06:16,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:06:16,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9be75f2866272ea51f51b HTTP/1.1" 200 None 2026-04-01 00:06:16,731 - lava_callback - ERROR - Error processing callback for node 69c9be75f2866272ea51f51b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:52534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49598 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38774 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60920 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:37228 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:40790 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:07:36,561 - lava_callback - INFO - Received callback for node 69c9bec2f2866272ea51f524 INFO: 172.19.0.1:40796 - "POST /node/69c9bec2f2866272ea51f524 HTTP/1.0" 202 Accepted 2026-04-01 00:07:36,567 - lava_callback - INFO - Starting async job submit for node 69c9bec2f2866272ea51f524 2026-04-01 00:07:36,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:07:36,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bec2f2866272ea51f524 HTTP/1.1" 200 None 2026-04-01 00:07:36,585 - lava_callback - DEBUG - Processing node 69c9bec2f2866272ea51f524: baseline-arm64-baylibre 2026-04-01 00:07:36,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bec2f2866272ea51f524 2026-04-01 00:07:36,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:07:36,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bec2f2866272ea51f524 HTTP/1.1" 200 None 2026-04-01 00:07:36,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:07:36,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-04-01 00:07:36,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:07:36,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bec2f2866272ea51f524 HTTP/1.1" 200 None 2026-04-01 00:07:36,637 - lava_callback - ERROR - Error processing callback for node 69c9bec2f2866272ea51f524: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:36882 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46098 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55274 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35820 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:09:05,710 - lava_callback - INFO - Received callback for node 69c9bf1df2866272ea51f532 INFO: 172.19.0.1:35824 - "POST /node/69c9bf1df2866272ea51f532 HTTP/1.0" 202 Accepted 2026-04-01 00:09:05,716 - lava_callback - INFO - Starting async job submit for node 69c9bf1df2866272ea51f532 2026-04-01 00:09:05,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:09:05,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bf1df2866272ea51f532 HTTP/1.1" 200 None 2026-04-01 00:09:05,740 - lava_callback - DEBUG - Processing node 69c9bf1df2866272ea51f532: baseline-arm64-clabbe 2026-04-01 00:09:05,741 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bf1df2866272ea51f532 2026-04-01 00:09:05,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:09:05,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bf1df2866272ea51f532 HTTP/1.1" 200 None 2026-04-01 00:09:05,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:09:05,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-04-01 00:09:05,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:09:05,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bf1df2866272ea51f532 HTTP/1.1" 200 None 2026-04-01 00:09:05,804 - lava_callback - ERROR - Error processing callback for node 69c9bf1df2866272ea51f532: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:46304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:46314 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49966 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35432 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35500 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:57424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47326 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51538 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:41926 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:54726 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49424 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49430 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:39128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:39140 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35158 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:35296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:42734 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:38478 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59466 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:12:05,830 - lava_callback - INFO - Received callback for node 69c9bfc1f2866272ea51f54b INFO: 172.19.0.1:59468 - "POST /node/69c9bfc1f2866272ea51f54b HTTP/1.0" 202 Accepted 2026-04-01 00:12:05,836 - lava_callback - INFO - Starting async job submit for node 69c9bfc1f2866272ea51f54b 2026-04-01 00:12:05,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:12:05,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bfc1f2866272ea51f54b HTTP/1.1" 200 None 2026-04-01 00:12:05,852 - lava_callback - DEBUG - Processing node 69c9bfc1f2866272ea51f54b: baseline-arm64-clabbe 2026-04-01 00:12:05,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9bfc1f2866272ea51f54b 2026-04-01 00:12:05,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:12:05,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9bfc1f2866272ea51f54b HTTP/1.1" 200 None 2026-04-01 00:12:05,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:12:05,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-04-01 00:12:05,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:12:05,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9bfc1f2866272ea51f54b HTTP/1.1" 200 None 2026-04-01 00:12:05,906 - lava_callback - ERROR - Error processing callback for node 69c9bfc1f2866272ea51f54b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:59470 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:51930 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:37600 - "GET / HTTP/1.0" 200 OK 2026-04-01 00:13:05,864 - lava_callback - INFO - Received callback for node 69c9c00af2866272ea51f553 INFO: 172.19.0.1:37612 - "POST /node/69c9c00af2866272ea51f553 HTTP/1.0" 202 Accepted 2026-04-01 00:13:05,870 - lava_callback - INFO - Starting async job submit for node 69c9c00af2866272ea51f553 2026-04-01 00:13:05,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:13:05,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c00af2866272ea51f553 HTTP/1.1" 200 None 2026-04-01 00:13:05,886 - lava_callback - DEBUG - Processing node 69c9c00af2866272ea51f553: baseline-arm64-clabbe 2026-04-01 00:13:05,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c9c00af2866272ea51f553 2026-04-01 00:13:05,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:13:05,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c00af2866272ea51f553 HTTP/1.1" 200 None 2026-04-01 00:13:05,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:13:05,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-04-01 00:13:05,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-01 00:13:05,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c9c00af2866272ea51f553 HTTP/1.1" 200 None 2026-04-01 00:13:05,957 - lava_callback - ERROR - Error processing callback for node 69c9c00af2866272ea51f553: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.19.0.1:37622 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55504 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:51978 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:52792 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60518 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:60522 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:55780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.19.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.19.0.1:45764 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]