INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:41712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41716 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:13:18,141 - lava_callback - INFO - Received callback for node 69c81b85123b47fd23770861 2026-03-29 00:13:18,149 - lava_callback - INFO - Starting async job submit for node 69c81b85123b47fd23770861 INFO: 172.20.0.1:41726 - "POST /node/69c81b85123b47fd23770861 HTTP/1.0" 202 Accepted 2026-03-29 00:13:18,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:13:18,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c81b85123b47fd23770861 HTTP/1.1" 200 None 2026-03-29 00:13:18,210 - lava_callback - DEBUG - Processing node 69c81b85123b47fd23770861: baseline-nfs-arm64 2026-03-29 00:13:18,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:13:18,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:13:18,971 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c81b85123b47fd23770861/log.txt.gz 2026-03-29 00:13:19,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:13:19,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:13:19,465 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c81b85123b47fd23770861/lava-logs.yaml 2026-03-29 00:13:19,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:13:19,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:13:19,653 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c81b85123b47fd23770861/lava_callback.json.gz 2026-03-29 00:13:19,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:13:19,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c81b85123b47fd23770861 HTTP/1.1" 200 None 2026-03-29 00:13:19,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:13:19,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-03-29 00:13:19,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:13:19,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c81b85123b47fd23770861 HTTP/1.1" 200 None 2026-03-29 00:13:19,840 - lava_callback - INFO - Completed processing callback for node 69c81b85123b47fd23770861 INFO: 172.20.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51876 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:13:49,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:13:49,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44416 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:14:02,531 - lava_callback - INFO - Received callback for node 69c5cbc42cb62322ad965d84 INFO: 172.20.0.1:43470 - "POST /node/69c5cbc42cb62322ad965d84 HTTP/1.0" 202 Accepted 2026-03-29 00:14:02,537 - lava_callback - INFO - Starting async job submit for node 69c5cbc42cb62322ad965d84 2026-03-29 00:14:02,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:14:02,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cbc42cb62322ad965d84 HTTP/1.1" 200 None 2026-03-29 00:14:02,563 - lava_callback - DEBUG - Processing node 69c5cbc42cb62322ad965d84: baseline-arm64-clabbe 2026-03-29 00:14:02,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5cbc42cb62322ad965d84 2026-03-29 00:14:02,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:14:02,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cbc42cb62322ad965d84 HTTP/1.1" 200 None 2026-03-29 00:14:02,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:14:02,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-29 00:14:02,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:14:02,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5cbc42cb62322ad965d84 HTTP/1.1" 200 None 2026-03-29 00:14:02,643 - lava_callback - ERROR - Error processing callback for node 69c5cbc42cb62322ad965d84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32922 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:15:42,635 - lava_callback - INFO - Received callback for node 69c5cc1b2cb62322ad965d93 INFO: 172.20.0.1:55520 - "POST /node/69c5cc1b2cb62322ad965d93 HTTP/1.0" 202 Accepted 2026-03-29 00:15:42,644 - lava_callback - INFO - Starting async job submit for node 69c5cc1b2cb62322ad965d93 2026-03-29 00:15:42,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:15:42,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc1b2cb62322ad965d93 HTTP/1.1" 200 None 2026-03-29 00:15:42,665 - lava_callback - DEBUG - Processing node 69c5cc1b2cb62322ad965d93: baseline-arm64-clabbe 2026-03-29 00:15:42,665 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5cc1b2cb62322ad965d93 2026-03-29 00:15:42,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:15:42,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc1b2cb62322ad965d93 HTTP/1.1" 200 None 2026-03-29 00:15:42,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:15:42,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-29 00:15:42,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:15:42,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5cc1b2cb62322ad965d93 HTTP/1.1" 200 None 2026-03-29 00:15:42,726 - lava_callback - ERROR - Error processing callback for node 69c5cc1b2cb62322ad965d93: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57468 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:32:54,292 - lava_callback - INFO - Received callback for node 69c87319f41e4fb6a378552e INFO: 172.20.0.1:47572 - "POST /node/69c87319f41e4fb6a378552e HTTP/1.0" 202 Accepted 2026-03-29 00:32:54,298 - lava_callback - INFO - Starting async job submit for node 69c87319f41e4fb6a378552e 2026-03-29 00:32:54,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:32:54,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87319f41e4fb6a378552e HTTP/1.1" 200 None 2026-03-29 00:32:54,331 - lava_callback - DEBUG - Processing node 69c87319f41e4fb6a378552e: baseline-x86 2026-03-29 00:32:54,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:32:54,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:32:54,630 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c87319f41e4fb6a378552e/log.txt.gz 2026-03-29 00:32:54,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:32:54,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:32:54,952 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c87319f41e4fb6a378552e/lava-logs.yaml 2026-03-29 00:32:54,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:32:55,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:32:55,117 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c87319f41e4fb6a378552e/lava_callback.json.gz 2026-03-29 00:32:55,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:32:55,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87319f41e4fb6a378552e HTTP/1.1" 200 None 2026-03-29 00:32:55,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:32:55,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f91f41e4fb6a3782f6b HTTP/1.1" 200 None 2026-03-29 00:32:55,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:32:55,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87319f41e4fb6a378552e HTTP/1.1" 200 None 2026-03-29 00:32:55,215 - lava_callback - INFO - Completed processing callback for node 69c87319f41e4fb6a378552e INFO: 172.20.0.1:47574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38898 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:33:19,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:19,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51596 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:33:40,015 - lava_callback - INFO - Received callback for node 69c8731af41e4fb6a3785531 INFO: 172.20.0.1:46562 - "POST /node/69c8731af41e4fb6a3785531 HTTP/1.0" 202 Accepted 2026-03-29 00:33:40,021 - lava_callback - INFO - Starting async job submit for node 69c8731af41e4fb6a3785531 2026-03-29 00:33:40,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:40,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8731af41e4fb6a3785531 HTTP/1.1" 200 None 2026-03-29 00:33:40,054 - lava_callback - DEBUG - Processing node 69c8731af41e4fb6a3785531: baseline-x86 2026-03-29 00:33:40,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:33:40,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:33:40,486 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c8731af41e4fb6a3785531/log.txt.gz 2026-03-29 00:33:40,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:33:40,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:33:40,903 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c8731af41e4fb6a3785531/lava-logs.yaml 2026-03-29 00:33:40,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:33:41,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:33:41,056 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c8731af41e4fb6a3785531/lava_callback.json.gz 2026-03-29 00:33:41,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:41,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8731af41e4fb6a3785531 HTTP/1.1" 200 None 2026-03-29 00:33:41,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:41,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f91f41e4fb6a3782f6b HTTP/1.1" 200 None 2026-03-29 00:33:41,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:41,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8731af41e4fb6a3785531 HTTP/1.1" 200 None 2026-03-29 00:33:41,149 - lava_callback - INFO - Completed processing callback for node 69c8731af41e4fb6a3785531 2026-03-29 00:33:43,367 - lava_callback - INFO - Received callback for node 69c87317f41e4fb6a3785528 INFO: 172.20.0.1:46572 - "POST /node/69c87317f41e4fb6a3785528 HTTP/1.0" 202 Accepted 2026-03-29 00:33:43,373 - lava_callback - INFO - Starting async job submit for node 69c87317f41e4fb6a3785528 2026-03-29 00:33:43,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:43,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87317f41e4fb6a3785528 HTTP/1.1" 200 None 2026-03-29 00:33:43,417 - lava_callback - DEBUG - Processing node 69c87317f41e4fb6a3785528: baseline-x86 2026-03-29 00:33:43,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:33:43,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:33:43,839 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c87317f41e4fb6a3785528/log.txt.gz 2026-03-29 00:33:43,921 - lava_callback - INFO - Received callback for node 69c87318f41e4fb6a378552b 2026-03-29 00:33:43,945 - lava_callback - INFO - Starting async job submit for node 69c87318f41e4fb6a378552b INFO: 172.20.0.1:46574 - "POST /node/69c87318f41e4fb6a378552b HTTP/1.0" 202 Accepted 2026-03-29 00:33:43,961 - lava_callback - INFO - Received callback for node 69c5d05d2cb62322ad965e6e 2026-03-29 00:33:44,006 - lava_callback - INFO - Starting async job submit for node 69c5d05d2cb62322ad965e6e INFO: 172.20.0.1:46586 - "POST /node/69c5d05d2cb62322ad965e6e HTTP/1.0" 202 Accepted 2026-03-29 00:33:44,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:33:44,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d05d2cb62322ad965e6e HTTP/1.1" 200 None 2026-03-29 00:33:44,067 - lava_callback - DEBUG - Processing node 69c5d05d2cb62322ad965e6e: baseline-arm-baylibre 2026-03-29 00:33:44,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d05d2cb62322ad965e6e 2026-03-29 00:33:44,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87318f41e4fb6a378552b HTTP/1.1" 200 None 2026-03-29 00:33:44,074 - lava_callback - DEBUG - Processing node 69c87318f41e4fb6a378552b: baseline-x86 2026-03-29 00:33:44,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d05d2cb62322ad965e6e HTTP/1.1" 200 None 2026-03-29 00:33:44,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:33:44,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:33:44,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d05d2cb62322ad965e6e HTTP/1.1" 200 None 2026-03-29 00:33:44,397 - lava_callback - ERROR - Error processing callback for node 69c5d05d2cb62322ad965e6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:33:44,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:33:44,454 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c87317f41e4fb6a3785528/lava-logs.yaml 2026-03-29 00:33:44,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:33:44,622 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:33:44,624 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c87318f41e4fb6a378552b/log.txt.gz 2026-03-29 00:33:44,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:33:44,709 - lava_callback - INFO - Received callback for node 69c5d05e2cb62322ad965e71 INFO: 172.20.0.1:46596 - "POST /node/69c5d05e2cb62322ad965e71 HTTP/1.0" 202 Accepted 2026-03-29 00:33:44,744 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c87317f41e4fb6a3785528/lava_callback.json.gz 2026-03-29 00:33:44,778 - lava_callback - INFO - Starting async job submit for node 69c5d05e2cb62322ad965e71 2026-03-29 00:33:44,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:33:44,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87317f41e4fb6a3785528 HTTP/1.1" 200 None 2026-03-29 00:33:44,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d05e2cb62322ad965e71 HTTP/1.1" 200 None 2026-03-29 00:33:44,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,836 - lava_callback - DEBUG - Processing node 69c5d05e2cb62322ad965e71: baseline-arm-baylibre 2026-03-29 00:33:44,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d05e2cb62322ad965e71 2026-03-29 00:33:44,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f91f41e4fb6a3782f6b HTTP/1.1" 200 None 2026-03-29 00:33:44,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d05e2cb62322ad965e71 HTTP/1.1" 200 None 2026-03-29 00:33:44,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:33:44,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:44,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87317f41e4fb6a3785528 HTTP/1.1" 200 None 2026-03-29 00:33:44,890 - lava_callback - INFO - Completed processing callback for node 69c87317f41e4fb6a3785528 2026-03-29 00:33:44,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d05e2cb62322ad965e71 HTTP/1.1" 200 None 2026-03-29 00:33:44,956 - lava_callback - ERROR - Error processing callback for node 69c5d05e2cb62322ad965e71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:33:45,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:33:45,156 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c87318f41e4fb6a378552b/lava-logs.yaml 2026-03-29 00:33:45,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:33:45,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:33:45,342 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c87318f41e4fb6a378552b/lava_callback.json.gz 2026-03-29 00:33:45,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:45,385 - lava_callback - INFO - Received callback for node 69c5d0602cb62322ad965e78 INFO: 172.20.0.1:46606 - "POST /node/69c5d0602cb62322ad965e78 HTTP/1.0" 202 Accepted 2026-03-29 00:33:45,391 - lava_callback - INFO - Starting async job submit for node 69c5d0602cb62322ad965e78 2026-03-29 00:33:45,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:45,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87318f41e4fb6a378552b HTTP/1.1" 200 None 2026-03-29 00:33:45,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:45,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0602cb62322ad965e78 HTTP/1.1" 200 None 2026-03-29 00:33:45,408 - lava_callback - DEBUG - Processing node 69c5d0602cb62322ad965e78: baseline-arm-baylibre 2026-03-29 00:33:45,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d0602cb62322ad965e78 2026-03-29 00:33:45,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:45,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f91f41e4fb6a3782f6b HTTP/1.1" 200 None 2026-03-29 00:33:45,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:45,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0602cb62322ad965e78 HTTP/1.1" 200 None 2026-03-29 00:33:45,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:45,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87318f41e4fb6a378552b HTTP/1.1" 200 None 2026-03-29 00:33:45,456 - lava_callback - INFO - Completed processing callback for node 69c87318f41e4fb6a378552b 2026-03-29 00:33:45,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:33:45,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:45,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0602cb62322ad965e78 HTTP/1.1" 200 None 2026-03-29 00:33:45,523 - lava_callback - ERROR - Error processing callback for node 69c5d0602cb62322ad965e78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:33:46,041 - lava_callback - INFO - Received callback for node 69c5d0622cb62322ad965e7b INFO: 172.20.0.1:46610 - "POST /node/69c5d0622cb62322ad965e7b HTTP/1.0" 202 Accepted 2026-03-29 00:33:46,048 - lava_callback - INFO - Starting async job submit for node 69c5d0622cb62322ad965e7b 2026-03-29 00:33:46,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:46,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0622cb62322ad965e7b HTTP/1.1" 200 None 2026-03-29 00:33:46,064 - lava_callback - DEBUG - Processing node 69c5d0622cb62322ad965e7b: baseline-arm-baylibre 2026-03-29 00:33:46,064 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d0622cb62322ad965e7b 2026-03-29 00:33:46,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:46,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0622cb62322ad965e7b HTTP/1.1" 200 None 2026-03-29 00:33:46,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:46,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:33:46,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:46,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0622cb62322ad965e7b HTTP/1.1" 200 None 2026-03-29 00:33:46,124 - lava_callback - ERROR - Error processing callback for node 69c5d0622cb62322ad965e7b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46612 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:33:49,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:33:50,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58918 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:34:03,397 - lava_callback - INFO - Received callback for node 69c5d0722cb62322ad965eaa INFO: 172.20.0.1:35300 - "POST /node/69c5d0722cb62322ad965eaa HTTP/1.0" 202 Accepted 2026-03-29 00:34:03,402 - lava_callback - INFO - Starting async job submit for node 69c5d0722cb62322ad965eaa 2026-03-29 00:34:03,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:03,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0722cb62322ad965eaa HTTP/1.1" 200 None 2026-03-29 00:34:03,418 - lava_callback - DEBUG - Processing node 69c5d0722cb62322ad965eaa: baseline-arm-clabbe 2026-03-29 00:34:03,418 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d0722cb62322ad965eaa 2026-03-29 00:34:03,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:03,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0722cb62322ad965eaa HTTP/1.1" 200 None 2026-03-29 00:34:03,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:03,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:34:03,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:03,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0722cb62322ad965eaa HTTP/1.1" 200 None 2026-03-29 00:34:03,478 - lava_callback - ERROR - Error processing callback for node 69c5d0722cb62322ad965eaa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:34:03,787 - lava_callback - INFO - Received callback for node 69c5d0662cb62322ad965e8c INFO: 172.20.0.1:35314 - "POST /node/69c5d0662cb62322ad965e8c HTTP/1.0" 202 Accepted 2026-03-29 00:34:03,793 - lava_callback - INFO - Starting async job submit for node 69c5d0662cb62322ad965e8c 2026-03-29 00:34:03,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:03,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0662cb62322ad965e8c HTTP/1.1" 200 None 2026-03-29 00:34:03,812 - lava_callback - DEBUG - Processing node 69c5d0662cb62322ad965e8c: baseline-arm-baylibre 2026-03-29 00:34:03,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d0662cb62322ad965e8c 2026-03-29 00:34:03,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:03,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0662cb62322ad965e8c HTTP/1.1" 200 None 2026-03-29 00:34:03,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:03,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:34:03,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:03,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0662cb62322ad965e8c HTTP/1.1" 200 None 2026-03-29 00:34:03,864 - lava_callback - ERROR - Error processing callback for node 69c5d0662cb62322ad965e8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:34:04,076 - lava_callback - INFO - Received callback for node 69c5d0772cb62322ad965ecf INFO: 172.20.0.1:35326 - "POST /node/69c5d0772cb62322ad965ecf HTTP/1.0" 202 Accepted 2026-03-29 00:34:04,082 - lava_callback - INFO - Starting async job submit for node 69c5d0772cb62322ad965ecf 2026-03-29 00:34:04,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0772cb62322ad965ecf HTTP/1.1" 200 None 2026-03-29 00:34:04,097 - lava_callback - DEBUG - Processing node 69c5d0772cb62322ad965ecf: baseline-arm-clabbe 2026-03-29 00:34:04,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d0772cb62322ad965ecf 2026-03-29 00:34:04,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0772cb62322ad965ecf HTTP/1.1" 200 None 2026-03-29 00:34:04,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:34:04,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0772cb62322ad965ecf HTTP/1.1" 200 None 2026-03-29 00:34:04,162 - lava_callback - ERROR - Error processing callback for node 69c5d0772cb62322ad965ecf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:34:04,485 - lava_callback - INFO - Received callback for node 69c5d0682cb62322ad965e91 INFO: 172.20.0.1:35332 - "POST /node/69c5d0682cb62322ad965e91 HTTP/1.0" 202 Accepted 2026-03-29 00:34:04,490 - lava_callback - INFO - Starting async job submit for node 69c5d0682cb62322ad965e91 2026-03-29 00:34:04,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0682cb62322ad965e91 HTTP/1.1" 200 None 2026-03-29 00:34:04,507 - lava_callback - DEBUG - Processing node 69c5d0682cb62322ad965e91: baseline-arm-baylibre 2026-03-29 00:34:04,507 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d0682cb62322ad965e91 2026-03-29 00:34:04,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0682cb62322ad965e91 HTTP/1.1" 200 None 2026-03-29 00:34:04,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:34:04,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0682cb62322ad965e91 HTTP/1.1" 200 None 2026-03-29 00:34:04,570 - lava_callback - ERROR - Error processing callback for node 69c5d0682cb62322ad965e91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:34:04,771 - lava_callback - INFO - Received callback for node 69c5d0782cb62322ad965ed7 INFO: 172.20.0.1:35344 - "POST /node/69c5d0782cb62322ad965ed7 HTTP/1.0" 202 Accepted 2026-03-29 00:34:04,777 - lava_callback - INFO - Starting async job submit for node 69c5d0782cb62322ad965ed7 2026-03-29 00:34:04,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0782cb62322ad965ed7 HTTP/1.1" 200 None 2026-03-29 00:34:04,793 - lava_callback - DEBUG - Processing node 69c5d0782cb62322ad965ed7: baseline-arm-clabbe 2026-03-29 00:34:04,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d0782cb62322ad965ed7 2026-03-29 00:34:04,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0782cb62322ad965ed7 HTTP/1.1" 200 None 2026-03-29 00:34:04,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:34:04,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:04,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0782cb62322ad965ed7 HTTP/1.1" 200 None 2026-03-29 00:34:04,856 - lava_callback - ERROR - Error processing callback for node 69c5d0782cb62322ad965ed7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:34:05,470 - lava_callback - INFO - Received callback for node 69c5d0792cb62322ad965edc INFO: 172.20.0.1:35352 - "POST /node/69c5d0792cb62322ad965edc HTTP/1.0" 202 Accepted 2026-03-29 00:34:05,476 - lava_callback - INFO - Starting async job submit for node 69c5d0792cb62322ad965edc 2026-03-29 00:34:05,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:05,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0792cb62322ad965edc HTTP/1.1" 200 None 2026-03-29 00:34:05,493 - lava_callback - DEBUG - Processing node 69c5d0792cb62322ad965edc: baseline-arm-clabbe 2026-03-29 00:34:05,493 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d0792cb62322ad965edc 2026-03-29 00:34:05,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:05,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d0792cb62322ad965edc HTTP/1.1" 200 None 2026-03-29 00:34:05,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:05,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:34:05,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:05,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d0792cb62322ad965edc HTTP/1.1" 200 None 2026-03-29 00:34:05,555 - lava_callback - ERROR - Error processing callback for node 69c5d0792cb62322ad965edc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37438 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:34:23,365 - lava_callback - INFO - Received callback for node 69c5d07d2cb62322ad965ee7 INFO: 172.20.0.1:49050 - "POST /node/69c5d07d2cb62322ad965ee7 HTTP/1.0" 202 Accepted 2026-03-29 00:34:23,370 - lava_callback - INFO - Starting async job submit for node 69c5d07d2cb62322ad965ee7 2026-03-29 00:34:23,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:23,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d07d2cb62322ad965ee7 HTTP/1.1" 200 None 2026-03-29 00:34:23,386 - lava_callback - DEBUG - Processing node 69c5d07d2cb62322ad965ee7: baseline-arm-clabbe 2026-03-29 00:34:23,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d07d2cb62322ad965ee7 2026-03-29 00:34:23,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:23,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d07d2cb62322ad965ee7 HTTP/1.1" 200 None 2026-03-29 00:34:23,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:23,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 00:34:23,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:23,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d07d2cb62322ad965ee7 HTTP/1.1" 200 None 2026-03-29 00:34:23,443 - lava_callback - ERROR - Error processing callback for node 69c5d07d2cb62322ad965ee7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35264 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:34:42,741 - lava_callback - INFO - Received callback for node 69c8731ef41e4fb6a3785538 INFO: 172.20.0.1:44460 - "POST /node/69c8731ef41e4fb6a3785538 HTTP/1.0" 202 Accepted 2026-03-29 00:34:42,751 - lava_callback - INFO - Starting async job submit for node 69c8731ef41e4fb6a3785538 2026-03-29 00:34:42,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:42,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8731ef41e4fb6a3785538 HTTP/1.1" 200 None 2026-03-29 00:34:42,806 - lava_callback - DEBUG - Processing node 69c8731ef41e4fb6a3785538: baseline-arm 2026-03-29 00:34:43,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:34:43,170 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:34:43,172 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c8731ef41e4fb6a3785538/log.txt.gz 2026-03-29 00:34:43,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:34:43,486 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:34:43,488 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c8731ef41e4fb6a3785538/lava-logs.yaml 2026-03-29 00:34:43,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:34:43,633 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:34:43,635 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c8731ef41e4fb6a3785538/lava_callback.json.gz 2026-03-29 00:34:43,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:43,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8731ef41e4fb6a3785538 HTTP/1.1" 200 None 2026-03-29 00:34:43,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:43,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f76f41e4fb6a3782f44 HTTP/1.1" 200 None 2026-03-29 00:34:43,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:43,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8731ef41e4fb6a3785538 HTTP/1.1" 200 None 2026-03-29 00:34:43,775 - lava_callback - INFO - Completed processing callback for node 69c8731ef41e4fb6a3785538 INFO: 172.20.0.1:44466 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:34:50,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:34:50,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60950 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:35:46,737 - lava_callback - INFO - Received callback for node 69c87317f41e4fb6a3785525 INFO: 172.20.0.1:60964 - "POST /node/69c87317f41e4fb6a3785525 HTTP/1.0" 202 Accepted 2026-03-29 00:35:46,745 - lava_callback - INFO - Starting async job submit for node 69c87317f41e4fb6a3785525 2026-03-29 00:35:46,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:35:46,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87317f41e4fb6a3785525 HTTP/1.1" 200 None 2026-03-29 00:35:46,785 - lava_callback - DEBUG - Processing node 69c87317f41e4fb6a3785525: baseline-x86 2026-03-29 00:35:46,901 - lava_callback - WARNING - Failed to upload log 2026-03-29 00:35:46,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:35:47,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:35:47,464 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c87317f41e4fb6a3785525/lava-logs.yaml 2026-03-29 00:35:47,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:35:47,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:35:47,957 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c87317f41e4fb6a3785525/lava_callback.json.gz 2026-03-29 00:35:47,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:35:48,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87317f41e4fb6a3785525 HTTP/1.1" 200 None 2026-03-29 00:35:48,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:35:48,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f91f41e4fb6a3782f6b HTTP/1.1" 200 None 2026-03-29 00:35:48,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:35:48,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87317f41e4fb6a3785525 HTTP/1.1" 200 None 2026-03-29 00:35:48,137 - lava_callback - INFO - Completed processing callback for node 69c87317f41e4fb6a3785525 2026-03-29 00:35:50,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:35:50,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44656 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:37:03,970 - lava_callback - INFO - Received callback for node 69c5d11f2cb62322ad965fdf INFO: 172.20.0.1:41412 - "POST /node/69c5d11f2cb62322ad965fdf HTTP/1.0" 202 Accepted 2026-03-29 00:37:03,980 - lava_callback - INFO - Starting async job submit for node 69c5d11f2cb62322ad965fdf 2026-03-29 00:37:03,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:37:03,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d11f2cb62322ad965fdf HTTP/1.1" 200 None 2026-03-29 00:37:03,997 - lava_callback - DEBUG - Processing node 69c5d11f2cb62322ad965fdf: baseline-arm-baylibre 2026-03-29 00:37:03,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d11f2cb62322ad965fdf 2026-03-29 00:37:03,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:37:04,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d11f2cb62322ad965fdf HTTP/1.1" 200 None 2026-03-29 00:37:04,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:37:04,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 00:37:04,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:37:04,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d11f2cb62322ad965fdf HTTP/1.1" 200 None 2026-03-29 00:37:04,064 - lava_callback - ERROR - Error processing callback for node 69c5d11f2cb62322ad965fdf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39022 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:37:43,982 - lava_callback - INFO - Received callback for node 69c5d1462cb62322ad965ff2 INFO: 172.20.0.1:52968 - "POST /node/69c5d1462cb62322ad965ff2 HTTP/1.0" 202 Accepted 2026-03-29 00:37:43,988 - lava_callback - INFO - Starting async job submit for node 69c5d1462cb62322ad965ff2 2026-03-29 00:37:43,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:37:44,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d1462cb62322ad965ff2 HTTP/1.1" 200 None 2026-03-29 00:37:44,003 - lava_callback - DEBUG - Processing node 69c5d1462cb62322ad965ff2: baseline-arm-baylibre 2026-03-29 00:37:44,004 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d1462cb62322ad965ff2 2026-03-29 00:37:44,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:37:44,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d1462cb62322ad965ff2 HTTP/1.1" 200 None 2026-03-29 00:37:44,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:37:44,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 00:37:44,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:37:44,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d1462cb62322ad965ff2 HTTP/1.1" 200 None 2026-03-29 00:37:44,062 - lava_callback - ERROR - Error processing callback for node 69c5d1462cb62322ad965ff2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48620 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:39:03,648 - lava_callback - INFO - Received callback for node 69c5d1932cb62322ad966019 INFO: 172.20.0.1:45374 - "POST /node/69c5d1932cb62322ad966019 HTTP/1.0" 202 Accepted 2026-03-29 00:39:03,654 - lava_callback - INFO - Starting async job submit for node 69c5d1932cb62322ad966019 2026-03-29 00:39:03,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:03,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d1932cb62322ad966019 HTTP/1.1" 200 None 2026-03-29 00:39:03,670 - lava_callback - DEBUG - Processing node 69c5d1932cb62322ad966019: baseline-arm-clabbe 2026-03-29 00:39:03,670 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d1932cb62322ad966019 2026-03-29 00:39:03,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:03,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d1932cb62322ad966019 HTTP/1.1" 200 None 2026-03-29 00:39:03,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:03,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 00:39:03,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:03,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d1932cb62322ad966019 HTTP/1.1" 200 None 2026-03-29 00:39:03,729 - lava_callback - ERROR - Error processing callback for node 69c5d1932cb62322ad966019: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51060 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:39:36,756 - lava_callback - INFO - Received callback for node 69c874bbf41e4fb6a3785611 INFO: 172.20.0.1:51076 - "POST /node/69c874bbf41e4fb6a3785611 HTTP/1.0" 202 Accepted 2026-03-29 00:39:36,762 - lava_callback - INFO - Starting async job submit for node 69c874bbf41e4fb6a3785611 2026-03-29 00:39:36,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:36,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874bbf41e4fb6a3785611 HTTP/1.1" 200 None 2026-03-29 00:39:36,783 - lava_callback - DEBUG - Processing node 69c874bbf41e4fb6a3785611: baseline-arm 2026-03-29 00:39:36,788 - lava_callback - WARNING - Failed to upload log 2026-03-29 00:39:36,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:39:36,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:39:36,977 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bbf41e4fb6a3785611/lava-logs.yaml 2026-03-29 00:39:36,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:39:37,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:39:37,117 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bbf41e4fb6a3785611/lava_callback.json.gz 2026-03-29 00:39:37,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:37,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874bbf41e4fb6a3785611 HTTP/1.1" 200 None 2026-03-29 00:39:37,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:37,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:39:37,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:37,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874bbf41e4fb6a3785611 HTTP/1.1" 200 None 2026-03-29 00:39:37,178 - lava_callback - INFO - Completed processing callback for node 69c874bbf41e4fb6a3785611 2026-03-29 00:39:43,572 - lava_callback - INFO - Received callback for node 69c5d1bb2cb62322ad96603e INFO: 172.20.0.1:38774 - "POST /node/69c5d1bb2cb62322ad96603e HTTP/1.0" 202 Accepted 2026-03-29 00:39:43,588 - lava_callback - INFO - Starting async job submit for node 69c5d1bb2cb62322ad96603e 2026-03-29 00:39:43,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:43,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d1bb2cb62322ad96603e HTTP/1.1" 200 None 2026-03-29 00:39:43,637 - lava_callback - DEBUG - Processing node 69c5d1bb2cb62322ad96603e: baseline-arm-clabbe 2026-03-29 00:39:43,637 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d1bb2cb62322ad96603e 2026-03-29 00:39:43,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:43,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d1bb2cb62322ad96603e HTTP/1.1" 200 None 2026-03-29 00:39:43,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:43,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 00:39:43,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:43,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d1bb2cb62322ad96603e HTTP/1.1" 200 None 2026-03-29 00:39:43,813 - lava_callback - ERROR - Error processing callback for node 69c5d1bb2cb62322ad96603e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38782 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:39:50,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:39:50,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47190 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:40:05,533 - lava_callback - INFO - Received callback for node 69c874bef41e4fb6a378561d INFO: 172.20.0.1:52434 - "POST /node/69c874bef41e4fb6a378561d HTTP/1.0" 202 Accepted 2026-03-29 00:40:05,585 - lava_callback - INFO - Starting async job submit for node 69c874bef41e4fb6a378561d 2026-03-29 00:40:05,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:05,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874bef41e4fb6a378561d HTTP/1.1" 200 None 2026-03-29 00:40:05,634 - lava_callback - DEBUG - Processing node 69c874bef41e4fb6a378561d: baseline-arm 2026-03-29 00:40:05,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:05,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:05,891 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bef41e4fb6a378561d/log.txt.gz 2026-03-29 00:40:05,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:06,134 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:06,136 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bef41e4fb6a378561d/lava-logs.yaml 2026-03-29 00:40:06,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:06,256 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:06,258 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bef41e4fb6a378561d/lava_callback.json.gz 2026-03-29 00:40:06,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:06,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874bef41e4fb6a378561d HTTP/1.1" 200 None 2026-03-29 00:40:06,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:06,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:40:06,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:06,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874bef41e4fb6a378561d HTTP/1.1" 200 None 2026-03-29 00:40:06,341 - lava_callback - INFO - Completed processing callback for node 69c874bef41e4fb6a378561d INFO: 172.20.0.1:52444 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:40:09,947 - lava_callback - INFO - Received callback for node 69c874bbf41e4fb6a3785614 INFO: 172.20.0.1:34734 - "POST /node/69c874bbf41e4fb6a3785614 HTTP/1.0" 202 Accepted 2026-03-29 00:40:09,993 - lava_callback - INFO - Starting async job submit for node 69c874bbf41e4fb6a3785614 2026-03-29 00:40:10,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:10,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874bbf41e4fb6a3785614 HTTP/1.1" 200 None 2026-03-29 00:40:10,029 - lava_callback - DEBUG - Processing node 69c874bbf41e4fb6a3785614: baseline-arm 2026-03-29 00:40:10,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:10,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:10,281 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bbf41e4fb6a3785614/log.txt.gz 2026-03-29 00:40:10,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:10,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:10,524 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bbf41e4fb6a3785614/lava-logs.yaml 2026-03-29 00:40:10,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:10,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:10,645 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bbf41e4fb6a3785614/lava_callback.json.gz 2026-03-29 00:40:10,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:10,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874bbf41e4fb6a3785614 HTTP/1.1" 200 None 2026-03-29 00:40:10,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:10,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:40:10,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:10,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874bbf41e4fb6a3785614 HTTP/1.1" 200 None 2026-03-29 00:40:10,728 - lava_callback - INFO - Completed processing callback for node 69c874bbf41e4fb6a3785614 INFO: 172.20.0.1:34748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34750 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:40:20,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:20,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44190 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:40:34,426 - lava_callback - INFO - Received callback for node 69c874bdf41e4fb6a378561a INFO: 172.20.0.1:38850 - "POST /node/69c874bdf41e4fb6a378561a HTTP/1.0" 202 Accepted 2026-03-29 00:40:34,432 - lava_callback - INFO - Starting async job submit for node 69c874bdf41e4fb6a378561a 2026-03-29 00:40:34,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:34,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874bdf41e4fb6a378561a HTTP/1.1" 200 None 2026-03-29 00:40:34,471 - lava_callback - DEBUG - Processing node 69c874bdf41e4fb6a378561a: baseline-arm 2026-03-29 00:40:34,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:34,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:34,809 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bdf41e4fb6a378561a/log.txt.gz 2026-03-29 00:40:34,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:35,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:35,158 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bdf41e4fb6a378561a/lava-logs.yaml 2026-03-29 00:40:35,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:35,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:35,329 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bdf41e4fb6a378561a/lava_callback.json.gz 2026-03-29 00:40:35,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:35,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874bdf41e4fb6a378561a HTTP/1.1" 200 None 2026-03-29 00:40:35,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:35,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:40:35,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:35,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874bdf41e4fb6a378561a HTTP/1.1" 200 None 2026-03-29 00:40:35,427 - lava_callback - INFO - Completed processing callback for node 69c874bdf41e4fb6a378561a INFO: 172.20.0.1:38866 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:40:45,262 - lava_callback - INFO - Received callback for node 69c874baf41e4fb6a378560e INFO: 172.20.0.1:53490 - "POST /node/69c874baf41e4fb6a378560e HTTP/1.0" 202 Accepted 2026-03-29 00:40:45,273 - lava_callback - INFO - Starting async job submit for node 69c874baf41e4fb6a378560e 2026-03-29 00:40:45,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:45,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874baf41e4fb6a378560e HTTP/1.1" 200 None 2026-03-29 00:40:45,350 - lava_callback - DEBUG - Processing node 69c874baf41e4fb6a378560e: baseline-arm 2026-03-29 00:40:45,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:45,663 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:45,664 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c874baf41e4fb6a378560e/log.txt.gz 2026-03-29 00:40:45,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:45,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:45,956 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c874baf41e4fb6a378560e/lava-logs.yaml 2026-03-29 00:40:45,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:46,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:46,113 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c874baf41e4fb6a378560e/lava_callback.json.gz 2026-03-29 00:40:46,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:46,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874baf41e4fb6a378560e HTTP/1.1" 200 None 2026-03-29 00:40:46,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:46,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:40:46,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:46,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874baf41e4fb6a378560e HTTP/1.1" 200 None 2026-03-29 00:40:46,199 - lava_callback - INFO - Completed processing callback for node 69c874baf41e4fb6a378560e INFO: 172.20.0.1:53496 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:40:48,009 - lava_callback - INFO - Received callback for node 69c874ddf41e4fb6a37856a2 INFO: 172.20.0.1:53510 - "POST /node/69c874ddf41e4fb6a37856a2 HTTP/1.0" 202 Accepted 2026-03-29 00:40:48,057 - lava_callback - INFO - Starting async job submit for node 69c874ddf41e4fb6a37856a2 2026-03-29 00:40:48,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:48,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874ddf41e4fb6a37856a2 HTTP/1.1" 200 None 2026-03-29 00:40:48,091 - lava_callback - DEBUG - Processing node 69c874ddf41e4fb6a37856a2: baseline-arm-pengutronix 2026-03-29 00:40:48,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:48,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:48,310 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874ddf41e4fb6a37856a2/log.txt.gz 2026-03-29 00:40:48,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:48,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:48,560 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874ddf41e4fb6a37856a2/lava-logs.yaml 2026-03-29 00:40:48,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:48,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:48,678 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874ddf41e4fb6a37856a2/lava_callback.json.gz 2026-03-29 00:40:48,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:48,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874ddf41e4fb6a37856a2 HTTP/1.1" 200 None 2026-03-29 00:40:48,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:48,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:40:48,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:48,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874ddf41e4fb6a37856a2 HTTP/1.1" 200 None 2026-03-29 00:40:48,782 - lava_callback - INFO - Completed processing callback for node 69c874ddf41e4fb6a37856a2 2026-03-29 00:40:50,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:50,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57416 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:40:56,898 - lava_callback - INFO - Received callback for node 69c874d9f41e4fb6a3785699 INFO: 172.20.0.1:57430 - "POST /node/69c874d9f41e4fb6a3785699 HTTP/1.0" 202 Accepted 2026-03-29 00:40:56,904 - lava_callback - INFO - Starting async job submit for node 69c874d9f41e4fb6a3785699 2026-03-29 00:40:56,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:56,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d9f41e4fb6a3785699 HTTP/1.1" 200 None 2026-03-29 00:40:56,938 - lava_callback - DEBUG - Processing node 69c874d9f41e4fb6a3785699: baseline-arm-pengutronix 2026-03-29 00:40:57,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:57,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:57,227 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874d9f41e4fb6a3785699/log.txt.gz 2026-03-29 00:40:57,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:57,499 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:57,501 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874d9f41e4fb6a3785699/lava-logs.yaml 2026-03-29 00:40:57,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:40:57,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:40:57,629 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874d9f41e4fb6a3785699/lava_callback.json.gz 2026-03-29 00:40:57,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:57,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d9f41e4fb6a3785699 HTTP/1.1" 200 None 2026-03-29 00:40:57,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:57,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:40:57,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:40:57,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874d9f41e4fb6a3785699 HTTP/1.1" 200 None 2026-03-29 00:40:57,715 - lava_callback - INFO - Completed processing callback for node 69c874d9f41e4fb6a3785699 2026-03-29 00:41:03,063 - lava_callback - INFO - Received callback for node 69c874caf41e4fb6a3785636 INFO: 172.20.0.1:50420 - "POST /node/69c874caf41e4fb6a3785636 HTTP/1.0" 202 Accepted 2026-03-29 00:41:03,069 - lava_callback - INFO - Starting async job submit for node 69c874caf41e4fb6a3785636 2026-03-29 00:41:03,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:03,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874caf41e4fb6a3785636 HTTP/1.1" 200 None 2026-03-29 00:41:03,107 - lava_callback - DEBUG - Processing node 69c874caf41e4fb6a3785636: baseline-arm-broonie 2026-03-29 00:41:03,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:03,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:03,421 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874caf41e4fb6a3785636/log.txt.gz 2026-03-29 00:41:03,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:03,648 - lava_callback - INFO - Received callback for node 69c5d2092cb62322ad966075 INFO: 172.20.0.1:50434 - "POST /node/69c5d2092cb62322ad966075 HTTP/1.0" 202 Accepted 2026-03-29 00:41:03,654 - lava_callback - INFO - Starting async job submit for node 69c5d2092cb62322ad966075 2026-03-29 00:41:03,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:03,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d2092cb62322ad966075 HTTP/1.1" 200 None 2026-03-29 00:41:03,673 - lava_callback - DEBUG - Processing node 69c5d2092cb62322ad966075: baseline-arm-clabbe 2026-03-29 00:41:03,673 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d2092cb62322ad966075 2026-03-29 00:41:03,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:03,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d2092cb62322ad966075 HTTP/1.1" 200 None 2026-03-29 00:41:03,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:03,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 00:41:03,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:03,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d2092cb62322ad966075 HTTP/1.1" 200 None 2026-03-29 00:41:03,737 - lava_callback - ERROR - Error processing callback for node 69c5d2092cb62322ad966075: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:41:03,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:03,795 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874caf41e4fb6a3785636/lava-logs.yaml 2026-03-29 00:41:03,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:03,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:03,984 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874caf41e4fb6a3785636/lava_callback.json.gz 2026-03-29 00:41:04,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:04,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874caf41e4fb6a3785636 HTTP/1.1" 200 None 2026-03-29 00:41:04,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:04,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:41:04,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:04,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874caf41e4fb6a3785636 HTTP/1.1" 200 None 2026-03-29 00:41:04,068 - lava_callback - INFO - Completed processing callback for node 69c874caf41e4fb6a3785636 2026-03-29 00:41:04,587 - lava_callback - INFO - Received callback for node 69c874cbf41e4fb6a378563b INFO: 172.20.0.1:50436 - "POST /node/69c874cbf41e4fb6a378563b HTTP/1.0" 202 Accepted 2026-03-29 00:41:04,593 - lava_callback - INFO - Starting async job submit for node 69c874cbf41e4fb6a378563b 2026-03-29 00:41:04,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:04,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874cbf41e4fb6a378563b HTTP/1.1" 200 None 2026-03-29 00:41:04,631 - lava_callback - DEBUG - Processing node 69c874cbf41e4fb6a378563b: baseline-arm-broonie 2026-03-29 00:41:04,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:04,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:04,959 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874cbf41e4fb6a378563b/log.txt.gz 2026-03-29 00:41:05,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:05,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:05,262 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874cbf41e4fb6a378563b/lava-logs.yaml 2026-03-29 00:41:05,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:05,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:05,397 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874cbf41e4fb6a378563b/lava_callback.json.gz 2026-03-29 00:41:05,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:05,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874cbf41e4fb6a378563b HTTP/1.1" 200 None 2026-03-29 00:41:05,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:05,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:41:05,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:05,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874cbf41e4fb6a378563b HTTP/1.1" 200 None 2026-03-29 00:41:05,476 - lava_callback - INFO - Completed processing callback for node 69c874cbf41e4fb6a378563b INFO: 172.20.0.1:50448 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:41:11,128 - lava_callback - INFO - Received callback for node 69c874ccf41e4fb6a378563e INFO: 172.20.0.1:37436 - "POST /node/69c874ccf41e4fb6a378563e HTTP/1.0" 202 Accepted 2026-03-29 00:41:11,134 - lava_callback - INFO - Starting async job submit for node 69c874ccf41e4fb6a378563e 2026-03-29 00:41:11,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:11,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874ccf41e4fb6a378563e HTTP/1.1" 200 None 2026-03-29 00:41:11,171 - lava_callback - DEBUG - Processing node 69c874ccf41e4fb6a378563e: baseline-arm-broonie 2026-03-29 00:41:11,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:11,510 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:11,512 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874ccf41e4fb6a378563e/log.txt.gz 2026-03-29 00:41:11,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:11,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:11,850 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874ccf41e4fb6a378563e/lava-logs.yaml 2026-03-29 00:41:11,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:12,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:12,006 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874ccf41e4fb6a378563e/lava_callback.json.gz 2026-03-29 00:41:12,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:12,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874ccf41e4fb6a378563e HTTP/1.1" 200 None 2026-03-29 00:41:12,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:12,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:41:12,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:12,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874ccf41e4fb6a378563e HTTP/1.1" 200 None 2026-03-29 00:41:12,100 - lava_callback - INFO - Completed processing callback for node 69c874ccf41e4fb6a378563e INFO: 172.20.0.1:37440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37450 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:41:20,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:20,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-29 00:41:23,586 - lava_callback - INFO - Received callback for node 69c5d22f2cb62322ad966089 INFO: 172.20.0.1:38792 - "POST /node/69c5d22f2cb62322ad966089 HTTP/1.0" 202 Accepted 2026-03-29 00:41:23,591 - lava_callback - INFO - Starting async job submit for node 69c5d22f2cb62322ad966089 2026-03-29 00:41:23,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:23,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d22f2cb62322ad966089 HTTP/1.1" 200 None 2026-03-29 00:41:23,608 - lava_callback - DEBUG - Processing node 69c5d22f2cb62322ad966089: baseline-arm-clabbe 2026-03-29 00:41:23,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d22f2cb62322ad966089 2026-03-29 00:41:23,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:23,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d22f2cb62322ad966089 HTTP/1.1" 200 None 2026-03-29 00:41:23,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:23,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 00:41:23,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:23,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d22f2cb62322ad966089 HTTP/1.1" 200 None 2026-03-29 00:41:23,660 - lava_callback - ERROR - Error processing callback for node 69c5d22f2cb62322ad966089: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38806 - "GET /node/69876c8ee31b58f834f19992%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-29 00:41:36,520 - lava_callback - INFO - Received callback for node 69c874cff41e4fb6a378564e INFO: 172.20.0.1:49466 - "POST /node/69c874cff41e4fb6a378564e HTTP/1.0" 202 Accepted 2026-03-29 00:41:36,526 - lava_callback - INFO - Starting async job submit for node 69c874cff41e4fb6a378564e INFO: 172.20.0.1:49472 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:41:36,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:36,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874cff41e4fb6a378564e HTTP/1.1" 200 None 2026-03-29 00:41:36,570 - lava_callback - DEBUG - Processing node 69c874cff41e4fb6a378564e: baseline-arm-clabbe 2026-03-29 00:41:36,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:36,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:36,944 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874cff41e4fb6a378564e/log.txt.gz 2026-03-29 00:41:37,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:37,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:37,317 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874cff41e4fb6a378564e/lava-logs.yaml 2026-03-29 00:41:37,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:37,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:37,475 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874cff41e4fb6a378564e/lava_callback.json.gz 2026-03-29 00:41:37,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:37,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874cff41e4fb6a378564e HTTP/1.1" 200 None 2026-03-29 00:41:37,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:37,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:41:37,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:37,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874cff41e4fb6a378564e HTTP/1.1" 200 None 2026-03-29 00:41:37,557 - lava_callback - INFO - Completed processing callback for node 69c874cff41e4fb6a378564e 2026-03-29 00:41:43,097 - lava_callback - INFO - Received callback for node 69c874c8f41e4fb6a3785631 INFO: 172.20.0.1:38730 - "POST /node/69c874c8f41e4fb6a3785631 HTTP/1.0" 202 Accepted 2026-03-29 00:41:43,103 - lava_callback - INFO - Starting async job submit for node 69c874c8f41e4fb6a3785631 2026-03-29 00:41:43,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:43,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874c8f41e4fb6a3785631 HTTP/1.1" 200 None 2026-03-29 00:41:43,161 - lava_callback - DEBUG - Processing node 69c874c8f41e4fb6a3785631: baseline-arm-broonie 2026-03-29 00:41:43,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:43,491 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:43,492 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874c8f41e4fb6a3785631/log.txt.gz 2026-03-29 00:41:43,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:43,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:43,819 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874c8f41e4fb6a3785631/lava-logs.yaml 2026-03-29 00:41:43,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:43,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:43,965 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c874c8f41e4fb6a3785631/lava_callback.json.gz 2026-03-29 00:41:43,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:44,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874c8f41e4fb6a3785631 HTTP/1.1" 200 None 2026-03-29 00:41:44,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:44,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:41:44,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:44,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874c8f41e4fb6a3785631 HTTP/1.1" 200 None 2026-03-29 00:41:44,057 - lava_callback - INFO - Completed processing callback for node 69c874c8f41e4fb6a3785631 INFO: 172.20.0.1:38738 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:41:50,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:50,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 00:41:56,103 - lava_callback - INFO - Received callback for node 69c874daf41e4fb6a378569c INFO: 172.20.0.1:51106 - "POST /node/69c874daf41e4fb6a378569c HTTP/1.0" 202 Accepted 2026-03-29 00:41:56,109 - lava_callback - INFO - Starting async job submit for node 69c874daf41e4fb6a378569c 2026-03-29 00:41:56,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:56,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874daf41e4fb6a378569c HTTP/1.1" 200 None 2026-03-29 00:41:56,147 - lava_callback - DEBUG - Processing node 69c874daf41e4fb6a378569c: baseline-arm-pengutronix 2026-03-29 00:41:56,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:56,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:56,450 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874daf41e4fb6a378569c/log.txt.gz INFO: 172.20.0.1:51118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51120 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:41:56,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:56,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:56,717 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874daf41e4fb6a378569c/lava-logs.yaml 2026-03-29 00:41:56,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:41:56,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:41:56,858 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874daf41e4fb6a378569c/lava_callback.json.gz 2026-03-29 00:41:56,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:56,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874daf41e4fb6a378569c HTTP/1.1" 200 None 2026-03-29 00:41:56,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:56,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:41:56,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:41:56,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874daf41e4fb6a378569c HTTP/1.1" 200 None 2026-03-29 00:41:57,001 - lava_callback - INFO - Completed processing callback for node 69c874daf41e4fb6a378569c INFO: 172.20.0.1:44160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:42:19,524 - lava_callback - INFO - Received callback for node 69c874f4f41e4fb6a37856dd INFO: 172.20.0.1:47042 - "POST /node/69c874f4f41e4fb6a37856dd HTTP/1.0" 202 Accepted 2026-03-29 00:42:19,530 - lava_callback - INFO - Starting async job submit for node 69c874f4f41e4fb6a37856dd 2026-03-29 00:42:19,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:42:19,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f4f41e4fb6a37856dd HTTP/1.1" 200 None 2026-03-29 00:42:19,569 - lava_callback - DEBUG - Processing node 69c874f4f41e4fb6a37856dd: ltp-dio 2026-03-29 00:42:19,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:42:20,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:42:20,134 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c874f4f41e4fb6a37856dd/log.txt.gz 2026-03-29 00:42:20,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:42:20,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 00:42:20,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:42:20,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:42:20,582 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c874f4f41e4fb6a37856dd/lava-logs.yaml 2026-03-29 00:42:20,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:42:20,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:42:20,757 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c874f4f41e4fb6a37856dd/lava_callback.json.gz 2026-03-29 00:42:20,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:42:20,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f4f41e4fb6a37856dd HTTP/1.1" 200 None 2026-03-29 00:42:20,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:42:20,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:42:20,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:42:20,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874f4f41e4fb6a37856dd HTTP/1.1" 200 None 2026-03-29 00:42:20,849 - lava_callback - INFO - Completed processing callback for node 69c874f4f41e4fb6a37856dd INFO: 172.20.0.1:54126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37846 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:42:50,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:42:50,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33480 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:43:23,633 - lava_callback - INFO - Received callback for node 69c874d0f41e4fb6a3785654 INFO: 172.20.0.1:33640 - "POST /node/69c874d0f41e4fb6a3785654 HTTP/1.0" 202 Accepted 2026-03-29 00:43:23,643 - lava_callback - INFO - Starting async job submit for node 69c874d0f41e4fb6a3785654 2026-03-29 00:43:23,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:43:23,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d0f41e4fb6a3785654 HTTP/1.1" 200 None 2026-03-29 00:43:23,679 - lava_callback - DEBUG - Processing node 69c874d0f41e4fb6a3785654: baseline-arm-clabbe 2026-03-29 00:43:23,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:43:24,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:43:24,178 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874d0f41e4fb6a3785654/log.txt.gz 2026-03-29 00:43:24,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:43:24,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:43:24,505 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874d0f41e4fb6a3785654/lava-logs.yaml 2026-03-29 00:43:24,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:43:24,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:43:24,847 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874d0f41e4fb6a3785654/lava_callback.json.gz 2026-03-29 00:43:24,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:43:24,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d0f41e4fb6a3785654 HTTP/1.1" 200 None 2026-03-29 00:43:24,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:43:24,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:43:24,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:43:24,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874d0f41e4fb6a3785654 HTTP/1.1" 200 None 2026-03-29 00:43:24,939 - lava_callback - INFO - Completed processing callback for node 69c874d0f41e4fb6a3785654 INFO: 172.20.0.1:33656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54472 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:43:50,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:43:50,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:44:03,680 - lava_callback - INFO - Received callback for node 69c5d2c62cb62322ad9660ed INFO: 172.20.0.1:42640 - "POST /node/69c5d2c62cb62322ad9660ed HTTP/1.0" 202 Accepted 2026-03-29 00:44:03,686 - lava_callback - INFO - Starting async job submit for node 69c5d2c62cb62322ad9660ed 2026-03-29 00:44:03,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:03,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d2c62cb62322ad9660ed HTTP/1.1" 200 None 2026-03-29 00:44:03,703 - lava_callback - DEBUG - Processing node 69c5d2c62cb62322ad9660ed: baseline-arm-clabbe 2026-03-29 00:44:03,703 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d2c62cb62322ad9660ed 2026-03-29 00:44:03,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:03,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d2c62cb62322ad9660ed HTTP/1.1" 200 None 2026-03-29 00:44:03,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:03,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 00:44:03,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:03,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d2c62cb62322ad9660ed HTTP/1.1" 200 None 2026-03-29 00:44:03,764 - lava_callback - ERROR - Error processing callback for node 69c5d2c62cb62322ad9660ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42650 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:44:09,547 - lava_callback - INFO - Received callback for node 69c875b7f41e4fb6a3785780 INFO: 172.20.0.1:42666 - "POST /node/69c875b7f41e4fb6a3785780 HTTP/1.0" 202 Accepted 2026-03-29 00:44:09,553 - lava_callback - INFO - Starting async job submit for node 69c875b7f41e4fb6a3785780 2026-03-29 00:44:09,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:09,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c875b7f41e4fb6a3785780 HTTP/1.1" 200 None 2026-03-29 00:44:09,586 - lava_callback - DEBUG - Processing node 69c875b7f41e4fb6a3785780: baseline-x86 2026-03-29 00:44:09,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:09,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:09,902 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c875b7f41e4fb6a3785780/log.txt.gz 2026-03-29 00:44:10,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:10,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:10,192 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c875b7f41e4fb6a3785780/lava-logs.yaml 2026-03-29 00:44:10,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:10,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:10,330 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c875b7f41e4fb6a3785780/lava_callback.json.gz 2026-03-29 00:44:10,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:10,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c875b7f41e4fb6a3785780 HTTP/1.1" 200 None 2026-03-29 00:44:10,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:10,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f91f41e4fb6a3782f6b HTTP/1.1" 200 None 2026-03-29 00:44:10,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:10,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c875b7f41e4fb6a3785780 HTTP/1.1" 200 None 2026-03-29 00:44:10,406 - lava_callback - INFO - Completed processing callback for node 69c875b7f41e4fb6a3785780 INFO: 172.20.0.1:34452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34460 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:44:20,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:20,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 00:44:22,780 - lava_callback - INFO - Received callback for node 69c875b6f41e4fb6a378577d INFO: 172.20.0.1:56944 - "POST /node/69c875b6f41e4fb6a378577d HTTP/1.0" 202 Accepted 2026-03-29 00:44:22,786 - lava_callback - INFO - Starting async job submit for node 69c875b6f41e4fb6a378577d 2026-03-29 00:44:22,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:22,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c875b6f41e4fb6a378577d HTTP/1.1" 200 None 2026-03-29 00:44:22,822 - lava_callback - DEBUG - Processing node 69c875b6f41e4fb6a378577d: baseline-x86 2026-03-29 00:44:22,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:23,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:23,115 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c875b6f41e4fb6a378577d/log.txt.gz 2026-03-29 00:44:23,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:23,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:23,443 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c875b6f41e4fb6a378577d/lava-logs.yaml 2026-03-29 00:44:23,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:23,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:23,583 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c875b6f41e4fb6a378577d/lava_callback.json.gz 2026-03-29 00:44:23,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:23,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c875b6f41e4fb6a378577d HTTP/1.1" 200 None 2026-03-29 00:44:23,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:23,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f91f41e4fb6a3782f6b HTTP/1.1" 200 None 2026-03-29 00:44:23,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:23,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c875b6f41e4fb6a378577d HTTP/1.1" 200 None 2026-03-29 00:44:23,667 - lava_callback - INFO - Completed processing callback for node 69c875b6f41e4fb6a378577d INFO: 172.20.0.1:56958 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:44:31,406 - lava_callback - INFO - Received callback for node 69c874edf41e4fb6a37856b1 INFO: 172.20.0.1:38790 - "POST /node/69c874edf41e4fb6a37856b1 HTTP/1.0" 202 Accepted 2026-03-29 00:44:31,412 - lava_callback - INFO - Starting async job submit for node 69c874edf41e4fb6a37856b1 2026-03-29 00:44:31,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:31,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874edf41e4fb6a37856b1 HTTP/1.1" 200 None 2026-03-29 00:44:31,452 - lava_callback - DEBUG - Processing node 69c874edf41e4fb6a37856b1: ltp-cap_bounds 2026-03-29 00:44:31,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:31,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:31,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c874edf41e4fb6a37856b1/log.txt.gz 2026-03-29 00:44:32,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:32,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:32,432 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c874edf41e4fb6a37856b1/lava-logs.yaml 2026-03-29 00:44:32,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:32,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:32,632 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c874edf41e4fb6a37856b1/lava_callback.json.gz 2026-03-29 00:44:32,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:32,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874edf41e4fb6a37856b1 HTTP/1.1" 200 None 2026-03-29 00:44:32,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:32,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:44:32,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:32,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874edf41e4fb6a37856b1 HTTP/1.1" 200 None 2026-03-29 00:44:32,723 - lava_callback - INFO - Completed processing callback for node 69c874edf41e4fb6a37856b1 2026-03-29 00:44:32,936 - lava_callback - INFO - Received callback for node 69c874f2f41e4fb6a37856d4 INFO: 172.20.0.1:38794 - "POST /node/69c874f2f41e4fb6a37856d4 HTTP/1.0" 202 Accepted 2026-03-29 00:44:32,944 - lava_callback - INFO - Starting async job submit for node 69c874f2f41e4fb6a37856d4 2026-03-29 00:44:32,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:32,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f2f41e4fb6a37856d4 HTTP/1.1" 200 None 2026-03-29 00:44:32,999 - lava_callback - DEBUG - Processing node 69c874f2f41e4fb6a37856d4: ltp-cve 2026-03-29 00:44:33,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:33,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:33,542 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c874f2f41e4fb6a37856d4/log.txt.gz 2026-03-29 00:44:33,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:33,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:33,957 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c874f2f41e4fb6a37856d4/lava-logs.yaml 2026-03-29 00:44:33,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:34,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:34,125 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c874f2f41e4fb6a37856d4/lava_callback.json.gz 2026-03-29 00:44:34,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:34,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f2f41e4fb6a37856d4 HTTP/1.1" 200 None 2026-03-29 00:44:34,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:34,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:44:34,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:34,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874f2f41e4fb6a37856d4 HTTP/1.1" 200 None 2026-03-29 00:44:34,213 - lava_callback - INFO - Completed processing callback for node 69c874f2f41e4fb6a37856d4 2026-03-29 00:44:34,713 - lava_callback - INFO - Received callback for node 69c87500f41e4fb6a37856f5 INFO: 172.20.0.1:38798 - "POST /node/69c87500f41e4fb6a37856f5 HTTP/1.0" 202 Accepted 2026-03-29 00:44:34,721 - lava_callback - INFO - Starting async job submit for node 69c87500f41e4fb6a37856f5 2026-03-29 00:44:34,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:34,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87500f41e4fb6a37856f5 HTTP/1.1" 200 None 2026-03-29 00:44:34,763 - lava_callback - DEBUG - Processing node 69c87500f41e4fb6a37856f5: ltp-syscalls 2026-03-29 00:44:35,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:35,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:35,633 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c87500f41e4fb6a37856f5/log.txt.gz 2026-03-29 00:44:36,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:36,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:36,291 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c87500f41e4fb6a37856f5/lava-logs.yaml 2026-03-29 00:44:36,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:36,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:36,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c87500f41e4fb6a37856f5/lava_callback.json.gz 2026-03-29 00:44:36,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:38802 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:44:36,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87500f41e4fb6a37856f5 HTTP/1.1" 200 None 2026-03-29 00:44:36,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:36,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:44:36,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:36,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87500f41e4fb6a37856f5 HTTP/1.1" 200 None 2026-03-29 00:44:36,576 - lava_callback - INFO - Completed processing callback for node 69c87500f41e4fb6a37856f5 INFO: 172.20.0.1:58988 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:44:50,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:50,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-29 00:44:51,797 - lava_callback - INFO - Received callback for node 69c874f0f41e4fb6a37856d0 INFO: 172.20.0.1:51556 - "POST /node/69c874f0f41e4fb6a37856d0 HTTP/1.0" 202 Accepted 2026-03-29 00:44:51,803 - lava_callback - INFO - Starting async job submit for node 69c874f0f41e4fb6a37856d0 2026-03-29 00:44:51,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:51,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f0f41e4fb6a37856d0 HTTP/1.1" 200 None 2026-03-29 00:44:51,844 - lava_callback - DEBUG - Processing node 69c874f0f41e4fb6a37856d0: ltp-controllers 2026-03-29 00:44:52,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:52,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:52,495 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c874f0f41e4fb6a37856d0/log.txt.gz 2026-03-29 00:44:52,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:52,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:52,978 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c874f0f41e4fb6a37856d0/lava-logs.yaml 2026-03-29 00:44:52,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:44:53,133 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:44:53,134 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c874f0f41e4fb6a37856d0/lava_callback.json.gz 2026-03-29 00:44:53,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:53,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f0f41e4fb6a37856d0 HTTP/1.1" 200 None 2026-03-29 00:44:53,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:53,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:44:53,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:44:53,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874f0f41e4fb6a37856d0 HTTP/1.1" 200 None 2026-03-29 00:44:53,232 - lava_callback - INFO - Completed processing callback for node 69c874f0f41e4fb6a37856d0 INFO: 172.20.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45818 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:45:20,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:45:20,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 00:45:22,746 - lava_callback - INFO - Received callback for node 69c874d4f41e4fb6a3785687 INFO: 172.20.0.1:53308 - "POST /node/69c874d4f41e4fb6a3785687 HTTP/1.0" 202 Accepted 2026-03-29 00:45:22,752 - lava_callback - INFO - Starting async job submit for node 69c874d4f41e4fb6a3785687 2026-03-29 00:45:22,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:45:22,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d4f41e4fb6a3785687 HTTP/1.1" 200 None 2026-03-29 00:45:22,816 - lava_callback - DEBUG - Processing node 69c874d4f41e4fb6a3785687: baseline-arm-clabbe 2026-03-29 00:45:23,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:45:23,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:45:23,262 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874d4f41e4fb6a3785687/log.txt.gz 2026-03-29 00:45:23,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:45:23,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:45:23,607 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874d4f41e4fb6a3785687/lava-logs.yaml 2026-03-29 00:45:23,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:45:23,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:45:23,751 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874d4f41e4fb6a3785687/lava_callback.json.gz 2026-03-29 00:45:23,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:45:23,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d4f41e4fb6a3785687 HTTP/1.1" 200 None 2026-03-29 00:45:23,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:45:23,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:45:23,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:45:23,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874d4f41e4fb6a3785687 HTTP/1.1" 200 None 2026-03-29 00:45:23,837 - lava_callback - INFO - Completed processing callback for node 69c874d4f41e4fb6a3785687 INFO: 172.20.0.1:53322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34748 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:45:43,398 - lava_callback - INFO - Received callback for node 69c874f5f41e4fb6a37856e0 INFO: 172.20.0.1:46174 - "POST /node/69c874f5f41e4fb6a37856e0 HTTP/1.0" 202 Accepted 2026-03-29 00:45:43,404 - lava_callback - INFO - Starting async job submit for node 69c874f5f41e4fb6a37856e0 2026-03-29 00:45:43,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:45:43,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f5f41e4fb6a37856e0 HTTP/1.1" 200 None 2026-03-29 00:45:43,449 - lava_callback - DEBUG - Processing node 69c874f5f41e4fb6a37856e0: ltp-fs_bind 2026-03-29 00:45:43,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:45:44,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:45:44,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c874f5f41e4fb6a37856e0/log.txt.gz 2026-03-29 00:45:44,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:45:44,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:45:44,500 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c874f5f41e4fb6a37856e0/lava-logs.yaml 2026-03-29 00:45:44,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:45:44,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:45:44,705 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c874f5f41e4fb6a37856e0/lava_callback.json.gz 2026-03-29 00:45:44,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:45:44,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f5f41e4fb6a37856e0 HTTP/1.1" 200 None 2026-03-29 00:45:44,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:45:44,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:45:44,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:45:44,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874f5f41e4fb6a37856e0 HTTP/1.1" 200 None 2026-03-29 00:45:44,788 - lava_callback - INFO - Completed processing callback for node 69c874f5f41e4fb6a37856e0 INFO: 172.20.0.1:46182 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:45:50,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:45:50,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49212 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:46:56,409 - lava_callback - INFO - Received callback for node 69c87674f41e4fb6a3785803 INFO: 172.20.0.1:57616 - "POST /node/69c87674f41e4fb6a3785803 HTTP/1.0" 202 Accepted 2026-03-29 00:46:56,415 - lava_callback - INFO - Starting async job submit for node 69c87674f41e4fb6a3785803 2026-03-29 00:46:56,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:46:56,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87674f41e4fb6a3785803 HTTP/1.1" 200 None 2026-03-29 00:46:56,434 - lava_callback - DEBUG - Processing node 69c87674f41e4fb6a3785803: baseline-arm 2026-03-29 00:46:56,439 - lava_callback - WARNING - Failed to upload log 2026-03-29 00:46:56,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:57624 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:46:56,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:46:56,592 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c87674f41e4fb6a3785803/lava-logs.yaml 2026-03-29 00:46:56,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:57628 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:46:56,737 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:46:56,739 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c87674f41e4fb6a3785803/lava_callback.json.gz 2026-03-29 00:46:56,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:46:56,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87674f41e4fb6a3785803 HTTP/1.1" 200 None 2026-03-29 00:46:56,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:46:56,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:46:56,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:46:56,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87674f41e4fb6a3785803 HTTP/1.1" 200 None 2026-03-29 00:46:56,807 - lava_callback - INFO - Completed processing callback for node 69c87674f41e4fb6a3785803 2026-03-29 00:47:01,809 - lava_callback - INFO - Received callback for node 69c874d5f41e4fb6a378568a INFO: 172.20.0.1:38404 - "POST /node/69c874d5f41e4fb6a378568a HTTP/1.0" 202 Accepted 2026-03-29 00:47:01,815 - lava_callback - INFO - Starting async job submit for node 69c874d5f41e4fb6a378568a 2026-03-29 00:47:01,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:47:01,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d5f41e4fb6a378568a HTTP/1.1" 200 None 2026-03-29 00:47:01,854 - lava_callback - DEBUG - Processing node 69c874d5f41e4fb6a378568a: baseline-arm-clabbe 2026-03-29 00:47:02,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:47:02,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:47:02,226 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874d5f41e4fb6a378568a/log.txt.gz 2026-03-29 00:47:02,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:47:02,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:47:02,733 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874d5f41e4fb6a378568a/lava-logs.yaml 2026-03-29 00:47:02,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:47:02,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:47:02,894 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c874d5f41e4fb6a378568a/lava_callback.json.gz 2026-03-29 00:47:02,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:47:02,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874d5f41e4fb6a378568a HTTP/1.1" 200 None 2026-03-29 00:47:02,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:47:02,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:47:02,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:47:02,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874d5f41e4fb6a378568a HTTP/1.1" 200 None 2026-03-29 00:47:02,980 - lava_callback - INFO - Completed processing callback for node 69c874d5f41e4fb6a378568a INFO: 172.20.0.1:38416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44064 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:47:20,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:47:20,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60868 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:47:31,543 - lava_callback - INFO - Received callback for node 69c874eff41e4fb6a37856b4 INFO: 172.20.0.1:55024 - "POST /node/69c874eff41e4fb6a37856b4 HTTP/1.0" 202 Accepted 2026-03-29 00:47:31,549 - lava_callback - INFO - Starting async job submit for node 69c874eff41e4fb6a37856b4 2026-03-29 00:47:31,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:47:31,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874eff41e4fb6a37856b4 HTTP/1.1" 200 None 2026-03-29 00:47:31,588 - lava_callback - DEBUG - Processing node 69c874eff41e4fb6a37856b4: ltp-containers 2026-03-29 00:47:31,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:47:32,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:47:32,091 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c874eff41e4fb6a37856b4/log.txt.gz 2026-03-29 00:47:32,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:47:32,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:47:32,527 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c874eff41e4fb6a37856b4/lava-logs.yaml 2026-03-29 00:47:32,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:47:32,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:47:32,697 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c874eff41e4fb6a37856b4/lava_callback.json.gz 2026-03-29 00:47:32,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:47:32,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874eff41e4fb6a37856b4 HTTP/1.1" 200 None 2026-03-29 00:47:32,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:47:32,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:47:32,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:47:32,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874eff41e4fb6a37856b4 HTTP/1.1" 200 None 2026-03-29 00:47:32,781 - lava_callback - INFO - Completed processing callback for node 69c874eff41e4fb6a37856b4 INFO: 172.20.0.1:55026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44454 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:47:50,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:47:50,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35458 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:48:31,617 - lava_callback - INFO - Received callback for node 69c874f7f41e4fb6a37856e3 INFO: 172.20.0.1:41786 - "POST /node/69c874f7f41e4fb6a37856e3 HTTP/1.0" 202 Accepted 2026-03-29 00:48:31,627 - lava_callback - INFO - Starting async job submit for node 69c874f7f41e4fb6a37856e3 2026-03-29 00:48:31,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:48:31,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f7f41e4fb6a37856e3 HTTP/1.1" 200 None 2026-03-29 00:48:31,691 - lava_callback - DEBUG - Processing node 69c874f7f41e4fb6a37856e3: ltp-fsx 2026-03-29 00:48:32,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:48:32,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:48:32,409 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c874f7f41e4fb6a37856e3/log.txt.gz 2026-03-29 00:48:32,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:48:33,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:48:33,006 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c874f7f41e4fb6a37856e3/lava-logs.yaml 2026-03-29 00:48:33,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:48:33,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:48:33,234 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c874f7f41e4fb6a37856e3/lava_callback.json.gz 2026-03-29 00:48:33,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:48:33,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f7f41e4fb6a37856e3 HTTP/1.1" 200 None 2026-03-29 00:48:33,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:48:33,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:48:33,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:48:33,363 - lava_callback - INFO - Received callback for node 69c874f9f41e4fb6a37856e6 INFO: 172.20.0.1:41788 - "POST /node/69c874f9f41e4fb6a37856e6 HTTP/1.0" 202 Accepted 2026-03-29 00:48:33,375 - lava_callback - INFO - Starting async job submit for node 69c874f9f41e4fb6a37856e6 2026-03-29 00:48:33,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874f7f41e4fb6a37856e3 HTTP/1.1" 200 None 2026-03-29 00:48:33,407 - lava_callback - INFO - Completed processing callback for node 69c874f7f41e4fb6a37856e3 2026-03-29 00:48:33,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:48:33,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f9f41e4fb6a37856e6 HTTP/1.1" 200 None 2026-03-29 00:48:33,532 - lava_callback - DEBUG - Processing node 69c874f9f41e4fb6a37856e6: ltp-input 2026-03-29 00:48:34,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:48:34,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:48:34,259 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c874f9f41e4fb6a37856e6/log.txt.gz 2026-03-29 00:48:34,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:48:34,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:48:34,790 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c874f9f41e4fb6a37856e6/lava-logs.yaml 2026-03-29 00:48:34,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:48:34,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:48:34,982 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c874f9f41e4fb6a37856e6/lava_callback.json.gz 2026-03-29 00:48:35,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:48:35,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874f9f41e4fb6a37856e6 HTTP/1.1" 200 None 2026-03-29 00:48:35,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:48:35,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:48:35,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:48:35,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874f9f41e4fb6a37856e6 HTTP/1.1" 200 None 2026-03-29 00:48:35,091 - lava_callback - INFO - Completed processing callback for node 69c874f9f41e4fb6a37856e6 INFO: 172.20.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52820 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:48:50,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:48:50,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42068 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:49:11,633 - lava_callback - INFO - Received callback for node 69c874faf41e4fb6a37856e9 INFO: 172.20.0.1:38628 - "POST /node/69c874faf41e4fb6a37856e9 HTTP/1.0" 202 Accepted 2026-03-29 00:49:11,640 - lava_callback - INFO - Starting async job submit for node 69c874faf41e4fb6a37856e9 2026-03-29 00:49:11,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:49:11,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874faf41e4fb6a37856e9 HTTP/1.1" 200 None 2026-03-29 00:49:11,701 - lava_callback - DEBUG - Processing node 69c874faf41e4fb6a37856e9: ltp-ipc 2026-03-29 00:49:12,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:49:12,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:49:12,302 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c874faf41e4fb6a37856e9/log.txt.gz 2026-03-29 00:49:12,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:49:12,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:49:12,831 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c874faf41e4fb6a37856e9/lava-logs.yaml 2026-03-29 00:49:12,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:49:13,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:49:13,028 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c874faf41e4fb6a37856e9/lava_callback.json.gz 2026-03-29 00:49:13,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:49:13,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874faf41e4fb6a37856e9 HTTP/1.1" 200 None 2026-03-29 00:49:13,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:49:13,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:49:13,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:49:13,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874faf41e4fb6a37856e9 HTTP/1.1" 200 None 2026-03-29 00:49:13,136 - lava_callback - INFO - Completed processing callback for node 69c874faf41e4fb6a37856e9 INFO: 172.20.0.1:38642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38652 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:49:20,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:49:20,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36168 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:49:43,585 - lava_callback - INFO - Received callback for node 69c874fcf41e4fb6a37856ec INFO: 172.20.0.1:55622 - "POST /node/69c874fcf41e4fb6a37856ec HTTP/1.0" 202 Accepted 2026-03-29 00:49:43,594 - lava_callback - INFO - Starting async job submit for node 69c874fcf41e4fb6a37856ec 2026-03-29 00:49:43,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:49:43,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874fcf41e4fb6a37856ec HTTP/1.1" 200 None 2026-03-29 00:49:43,668 - lava_callback - DEBUG - Processing node 69c874fcf41e4fb6a37856ec: ltp-pty 2026-03-29 00:49:44,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:49:44,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:49:44,294 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c874fcf41e4fb6a37856ec/log.txt.gz 2026-03-29 00:49:44,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:49:44,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:49:44,734 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c874fcf41e4fb6a37856ec/lava-logs.yaml 2026-03-29 00:49:44,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:49:44,936 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:49:44,938 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c874fcf41e4fb6a37856ec/lava_callback.json.gz 2026-03-29 00:49:44,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:49:44,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874fcf41e4fb6a37856ec HTTP/1.1" 200 None 2026-03-29 00:49:44,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:49:44,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:49:45,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:49:45,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874fcf41e4fb6a37856ec HTTP/1.1" 200 None 2026-03-29 00:49:45,029 - lava_callback - INFO - Completed processing callback for node 69c874fcf41e4fb6a37856ec INFO: 172.20.0.1:55634 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:49:50,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:49:50,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51614 - "GET /node/6988693ce408e9ea60faf7ef%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42228 - "GET /node/6993be7c3e9ba4930770c237%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34826 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:51:51,952 - lava_callback - INFO - Received callback for node 69c874fdf41e4fb6a37856ef INFO: 172.20.0.1:33710 - "POST /node/69c874fdf41e4fb6a37856ef HTTP/1.0" 202 Accepted 2026-03-29 00:51:51,963 - lava_callback - INFO - Starting async job submit for node 69c874fdf41e4fb6a37856ef 2026-03-29 00:51:51,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:51:52,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874fdf41e4fb6a37856ef HTTP/1.1" 200 None 2026-03-29 00:51:52,015 - lava_callback - DEBUG - Processing node 69c874fdf41e4fb6a37856ef: ltp-sched 2026-03-29 00:51:52,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:51:53,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:51:53,411 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c874fdf41e4fb6a37856ef/log.txt.gz 2026-03-29 00:51:53,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:51:54,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:51:54,582 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c874fdf41e4fb6a37856ef/lava-logs.yaml 2026-03-29 00:51:54,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:51:55,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:51:55,405 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c874fdf41e4fb6a37856ef/lava_callback.json.gz 2026-03-29 00:51:55,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:51:55,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874fdf41e4fb6a37856ef HTTP/1.1" 200 None 2026-03-29 00:51:55,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:51:55,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:51:55,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:51:55,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874fdf41e4fb6a37856ef HTTP/1.1" 200 None 2026-03-29 00:51:55,505 - lava_callback - INFO - Completed processing callback for node 69c874fdf41e4fb6a37856ef INFO: 172.20.0.1:33726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33744 - "GET /node/69afe0c0ae557f2600f0fb0b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38770 - "GET /node/6940265e575bc060ce154358%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47306 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:52:20,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:20,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58360 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:52:29,313 - lava_callback - INFO - Received callback for node 69c87320f41e4fb6a378553b INFO: 172.20.0.1:58364 - "POST /node/69c87320f41e4fb6a378553b HTTP/1.0" 202 Accepted 2026-03-29 00:52:29,324 - lava_callback - INFO - Starting async job submit for node 69c87320f41e4fb6a378553b 2026-03-29 00:52:29,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:29,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87320f41e4fb6a378553b HTTP/1.1" 200 None 2026-03-29 00:52:29,479 - lava_callback - DEBUG - Processing node 69c87320f41e4fb6a378553b: ltp-smoke 2026-03-29 00:52:30,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:30,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:30,340 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c87320f41e4fb6a378553b/log.txt.gz 2026-03-29 00:52:30,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:31,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:31,116 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c87320f41e4fb6a378553b/lava-logs.yaml 2026-03-29 00:52:31,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:31,562 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:31,565 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c87320f41e4fb6a378553b/lava_callback.json.gz 2026-03-29 00:52:31,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:31,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87320f41e4fb6a378553b HTTP/1.1" 200 None 2026-03-29 00:52:31,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:31,754 - lava_callback - INFO - Received callback for node 69c874fff41e4fb6a37856f2 INFO: 172.20.0.1:46504 - "POST /node/69c874fff41e4fb6a37856f2 HTTP/1.0" 202 Accepted 2026-03-29 00:52:31,760 - lava_callback - INFO - Starting async job submit for node 69c874fff41e4fb6a37856f2 2026-03-29 00:52:31,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:31,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f76f41e4fb6a3782f44 HTTP/1.1" 200 None 2026-03-29 00:52:31,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:31,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874fff41e4fb6a37856f2 HTTP/1.1" 200 None 2026-03-29 00:52:31,804 - lava_callback - DEBUG - Processing node 69c874fff41e4fb6a37856f2: ltp-smoke 2026-03-29 00:52:31,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87320f41e4fb6a378553b HTTP/1.1" 200 None 2026-03-29 00:52:32,193 - lava_callback - INFO - Completed processing callback for node 69c87320f41e4fb6a378553b 2026-03-29 00:52:32,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:32,688 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:32,690 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c874fff41e4fb6a37856f2/log.txt.gz 2026-03-29 00:52:32,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:33,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:33,332 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c874fff41e4fb6a37856f2/lava-logs.yaml 2026-03-29 00:52:33,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:33,345 - lava_callback - INFO - Received callback for node 69c87502f41e4fb6a37856f8 INFO: 172.20.0.1:46514 - "POST /node/69c87502f41e4fb6a37856f8 HTTP/1.0" 202 Accepted 2026-03-29 00:52:33,354 - lava_callback - INFO - Starting async job submit for node 69c87502f41e4fb6a37856f8 2026-03-29 00:52:33,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:33,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87502f41e4fb6a37856f8 HTTP/1.1" 200 None 2026-03-29 00:52:33,396 - lava_callback - DEBUG - Processing node 69c87502f41e4fb6a37856f8: ltp-syscalls-ipc 2026-03-29 00:52:33,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:33,583 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c874fff41e4fb6a37856f2/lava_callback.json.gz 2026-03-29 00:52:33,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:33,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874fff41e4fb6a37856f2 HTTP/1.1" 200 None 2026-03-29 00:52:33,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:33,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:52:33,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:33,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874fff41e4fb6a37856f2 HTTP/1.1" 200 None 2026-03-29 00:52:33,889 - lava_callback - INFO - Completed processing callback for node 69c874fff41e4fb6a37856f2 2026-03-29 00:52:34,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:34,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:34,315 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c87502f41e4fb6a37856f8/log.txt.gz 2026-03-29 00:52:34,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:35,030 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:35,032 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c87502f41e4fb6a37856f8/lava-logs.yaml 2026-03-29 00:52:35,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:35,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:35,176 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c87502f41e4fb6a37856f8/lava_callback.json.gz 2026-03-29 00:52:35,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:35,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87502f41e4fb6a37856f8 HTTP/1.1" 200 None 2026-03-29 00:52:35,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:35,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:52:35,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:35,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87502f41e4fb6a37856f8 HTTP/1.1" 200 None 2026-03-29 00:52:35,293 - lava_callback - INFO - Completed processing callback for node 69c87502f41e4fb6a37856f8 INFO: 172.20.0.1:46530 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:52:44,277 - lava_callback - INFO - Received callback for node 69c5d4c52cb62322ad9661e2 INFO: 172.20.0.1:51186 - "POST /node/69c5d4c52cb62322ad9661e2 HTTP/1.0" 202 Accepted 2026-03-29 00:52:44,287 - lava_callback - INFO - Starting async job submit for node 69c5d4c52cb62322ad9661e2 2026-03-29 00:52:44,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:44,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d4c52cb62322ad9661e2 HTTP/1.1" 200 None 2026-03-29 00:52:44,317 - lava_callback - DEBUG - Processing node 69c5d4c52cb62322ad9661e2: baseline-arm-clabbe 2026-03-29 00:52:44,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d4c52cb62322ad9661e2 2026-03-29 00:52:44,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:44,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d4c52cb62322ad9661e2 HTTP/1.1" 200 None 2026-03-29 00:52:44,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:44,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-29 00:52:44,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:44,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d4c52cb62322ad9661e2 HTTP/1.1" 200 None 2026-03-29 00:52:44,414 - lava_callback - ERROR - Error processing callback for node 69c5d4c52cb62322ad9661e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:52:50,041 - lava_callback - INFO - Received callback for node 69c877a4f41e4fb6a37858a9 INFO: 172.20.0.1:60498 - "POST /node/69c877a4f41e4fb6a37858a9 HTTP/1.0" 202 Accepted 2026-03-29 00:52:50,050 - lava_callback - INFO - Starting async job submit for node 69c877a4f41e4fb6a37858a9 2026-03-29 00:52:50,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:50,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c877a4f41e4fb6a37858a9 HTTP/1.1" 200 None 2026-03-29 00:52:50,120 - lava_callback - DEBUG - Processing node 69c877a4f41e4fb6a37858a9: baseline-arm-pengutronix 2026-03-29 00:52:50,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:50,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:50,491 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:50,493 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c877a4f41e4fb6a37858a9/log.txt.gz 2026-03-29 00:52:50,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 00:52:50,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:51,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:51,080 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c877a4f41e4fb6a37858a9/lava-logs.yaml 2026-03-29 00:52:51,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:52:51,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:52:51,244 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c877a4f41e4fb6a37858a9/lava_callback.json.gz 2026-03-29 00:52:51,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:51,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c877a4f41e4fb6a37858a9 HTTP/1.1" 200 None 2026-03-29 00:52:51,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:51,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:52:51,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:52:51,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c877a4f41e4fb6a37858a9 HTTP/1.1" 200 None 2026-03-29 00:52:51,333 - lava_callback - INFO - Completed processing callback for node 69c877a4f41e4fb6a37858a9 INFO: 172.20.0.1:60512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34832 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:53:20,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:53:20,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46740 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:53:44,026 - lava_callback - INFO - Received callback for node 69c87505f41e4fb6a3785702 INFO: 172.20.0.1:44818 - "POST /node/69c87505f41e4fb6a3785702 HTTP/1.0" 202 Accepted 2026-03-29 00:53:44,040 - lava_callback - INFO - Starting async job submit for node 69c87505f41e4fb6a3785702 2026-03-29 00:53:44,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:53:44,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87505f41e4fb6a3785702 HTTP/1.1" 200 None 2026-03-29 00:53:44,116 - lava_callback - DEBUG - Processing node 69c87505f41e4fb6a3785702: ltp-watchqueue 2026-03-29 00:53:44,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:53:44,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:53:44,964 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c87505f41e4fb6a3785702/log.txt.gz 2026-03-29 00:53:45,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:53:45,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:53:45,429 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c87505f41e4fb6a3785702/lava-logs.yaml 2026-03-29 00:53:45,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:53:45,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:53:45,592 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c87505f41e4fb6a3785702/lava_callback.json.gz 2026-03-29 00:53:45,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:53:45,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87505f41e4fb6a3785702 HTTP/1.1" 200 None 2026-03-29 00:53:45,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:53:45,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:53:45,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:53:45,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87505f41e4fb6a3785702 HTTP/1.1" 200 None 2026-03-29 00:53:45,678 - lava_callback - INFO - Completed processing callback for node 69c87505f41e4fb6a3785702 INFO: 172.20.0.1:44830 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:53:50,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:53:50,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 00:53:52,064 - lava_callback - INFO - Received callback for node 69c87503f41e4fb6a37856ff INFO: 172.20.0.1:53112 - "POST /node/69c87503f41e4fb6a37856ff HTTP/1.0" 202 Accepted 2026-03-29 00:53:52,070 - lava_callback - INFO - Starting async job submit for node 69c87503f41e4fb6a37856ff 2026-03-29 00:53:52,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:53:52,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87503f41e4fb6a37856ff HTTP/1.1" 200 None 2026-03-29 00:53:52,117 - lava_callback - DEBUG - Processing node 69c87503f41e4fb6a37856ff: ltp-timers 2026-03-29 00:53:52,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:53:52,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:53:52,643 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c87503f41e4fb6a37856ff/log.txt.gz 2026-03-29 00:53:52,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:53:53,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:53:53,051 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c87503f41e4fb6a37856ff/lava-logs.yaml 2026-03-29 00:53:53,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:53:53,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:53:53,225 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c87503f41e4fb6a37856ff/lava_callback.json.gz 2026-03-29 00:53:53,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:53:53,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87503f41e4fb6a37856ff HTTP/1.1" 200 None 2026-03-29 00:53:53,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:53:53,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:53:53,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:53:53,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87503f41e4fb6a37856ff HTTP/1.1" 200 None 2026-03-29 00:53:53,313 - lava_callback - INFO - Completed processing callback for node 69c87503f41e4fb6a37856ff INFO: 172.20.0.1:53128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53142 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:54:04,289 - lava_callback - INFO - Received callback for node 69c5d51d2cb62322ad9661fd INFO: 172.20.0.1:36142 - "POST /node/69c5d51d2cb62322ad9661fd HTTP/1.0" 202 Accepted 2026-03-29 00:54:04,295 - lava_callback - INFO - Starting async job submit for node 69c5d51d2cb62322ad9661fd 2026-03-29 00:54:04,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:54:04,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d51d2cb62322ad9661fd HTTP/1.1" 200 None 2026-03-29 00:54:04,311 - lava_callback - DEBUG - Processing node 69c5d51d2cb62322ad9661fd: baseline-arm-clabbe 2026-03-29 00:54:04,311 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d51d2cb62322ad9661fd 2026-03-29 00:54:04,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:54:04,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d51d2cb62322ad9661fd HTTP/1.1" 200 None 2026-03-29 00:54:04,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:54:04,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-29 00:54:04,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:54:04,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d51d2cb62322ad9661fd HTTP/1.1" 200 None 2026-03-29 00:54:04,378 - lava_callback - ERROR - Error processing callback for node 69c5d51d2cb62322ad9661fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42492 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:54:20,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:54:20,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57794 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:54:28,400 - lava_callback - INFO - Received callback for node 69c8782cf41e4fb6a378590b INFO: 172.20.0.1:57798 - "POST /node/69c8782cf41e4fb6a378590b HTTP/1.0" 202 Accepted 2026-03-29 00:54:28,406 - lava_callback - INFO - Starting async job submit for node 69c8782cf41e4fb6a378590b 2026-03-29 00:54:28,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:54:28,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8782cf41e4fb6a378590b HTTP/1.1" 200 None 2026-03-29 00:54:28,426 - lava_callback - DEBUG - Processing node 69c8782cf41e4fb6a378590b: baseline-arm 2026-03-29 00:54:28,431 - lava_callback - WARNING - Failed to upload log 2026-03-29 00:54:28,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:54:28,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:54:28,558 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c8782cf41e4fb6a378590b/lava-logs.yaml 2026-03-29 00:54:28,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:54:28,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:54:28,670 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c8782cf41e4fb6a378590b/lava_callback.json.gz 2026-03-29 00:54:28,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:54:28,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8782cf41e4fb6a378590b HTTP/1.1" 200 None 2026-03-29 00:54:28,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:54:28,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:54:28,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:54:28,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8782cf41e4fb6a378590b HTTP/1.1" 200 None 2026-03-29 00:54:28,779 - lava_callback - INFO - Completed processing callback for node 69c8782cf41e4fb6a378590b INFO: 172.20.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34360 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:54:50,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:54:50,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41544 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:55:24,954 - lava_callback - INFO - Received callback for node 69c5d56e2cb62322ad96628c INFO: 172.20.0.1:49398 - "POST /node/69c5d56e2cb62322ad96628c HTTP/1.0" 202 Accepted 2026-03-29 00:55:24,962 - lava_callback - INFO - Starting async job submit for node 69c5d56e2cb62322ad96628c 2026-03-29 00:55:24,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:24,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d56e2cb62322ad96628c HTTP/1.1" 200 None 2026-03-29 00:55:24,990 - lava_callback - DEBUG - Processing node 69c5d56e2cb62322ad96628c: baseline-arm-baylibre 2026-03-29 00:55:24,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d56e2cb62322ad96628c 2026-03-29 00:55:24,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:25,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d56e2cb62322ad96628c HTTP/1.1" 200 None 2026-03-29 00:55:25,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:25,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 00:55:25,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:25,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d56e2cb62322ad96628c HTTP/1.1" 200 None 2026-03-29 00:55:25,098 - lava_callback - ERROR - Error processing callback for node 69c5d56e2cb62322ad96628c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60826 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:55:44,882 - lava_callback - INFO - Received callback for node 69c5d58a2cb62322ad9662cb INFO: 172.20.0.1:49696 - "POST /node/69c5d58a2cb62322ad9662cb HTTP/1.0" 202 Accepted 2026-03-29 00:55:44,889 - lava_callback - INFO - Starting async job submit for node 69c5d58a2cb62322ad9662cb 2026-03-29 00:55:44,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:44,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d58a2cb62322ad9662cb HTTP/1.1" 200 None 2026-03-29 00:55:44,918 - lava_callback - DEBUG - Processing node 69c5d58a2cb62322ad9662cb: baseline-arm64-baylibre 2026-03-29 00:55:44,919 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d58a2cb62322ad9662cb 2026-03-29 00:55:44,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:44,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d58a2cb62322ad9662cb HTTP/1.1" 200 None 2026-03-29 00:55:44,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:44,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 00:55:44,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:44,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d58a2cb62322ad9662cb HTTP/1.1" 200 None 2026-03-29 00:55:45,000 - lava_callback - ERROR - Error processing callback for node 69c5d58a2cb62322ad9662cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:55:45,570 - lava_callback - INFO - Received callback for node 69c5d5862cb62322ad9662c4 INFO: 172.20.0.1:49702 - "POST /node/69c5d5862cb62322ad9662c4 HTTP/1.0" 202 Accepted 2026-03-29 00:55:45,576 - lava_callback - INFO - Starting async job submit for node 69c5d5862cb62322ad9662c4 2026-03-29 00:55:45,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:45,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5862cb62322ad9662c4 HTTP/1.1" 200 None 2026-03-29 00:55:45,601 - lava_callback - DEBUG - Processing node 69c5d5862cb62322ad9662c4: baseline-arm64-baylibre 2026-03-29 00:55:45,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d5862cb62322ad9662c4 2026-03-29 00:55:45,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:45,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5862cb62322ad9662c4 HTTP/1.1" 200 None 2026-03-29 00:55:45,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:45,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 00:55:45,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:55:45,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5862cb62322ad9662c4 HTTP/1.1" 200 None 2026-03-29 00:55:45,656 - lava_callback - ERROR - Error processing callback for node 69c5d5862cb62322ad9662c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41360 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:56:04,922 - lava_callback - INFO - Received callback for node 69c5d58e2cb62322ad9662d8 INFO: 172.20.0.1:39304 - "POST /node/69c5d58e2cb62322ad9662d8 HTTP/1.0" 202 Accepted 2026-03-29 00:56:04,931 - lava_callback - INFO - Starting async job submit for node 69c5d58e2cb62322ad9662d8 2026-03-29 00:56:04,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:04,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d58e2cb62322ad9662d8 HTTP/1.1" 200 None 2026-03-29 00:56:04,959 - lava_callback - DEBUG - Processing node 69c5d58e2cb62322ad9662d8: baseline-arm64-baylibre 2026-03-29 00:56:04,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d58e2cb62322ad9662d8 2026-03-29 00:56:04,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:04,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d58e2cb62322ad9662d8 HTTP/1.1" 200 None 2026-03-29 00:56:04,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:05,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 00:56:05,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:05,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d58e2cb62322ad9662d8 HTTP/1.1" 200 None 2026-03-29 00:56:05,071 - lava_callback - ERROR - Error processing callback for node 69c5d58e2cb62322ad9662d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:56:05,593 - lava_callback - INFO - Received callback for node 69c5d5952cb62322ad9662e1 INFO: 172.20.0.1:39320 - "POST /node/69c5d5952cb62322ad9662e1 HTTP/1.0" 202 Accepted 2026-03-29 00:56:05,608 - lava_callback - INFO - Starting async job submit for node 69c5d5952cb62322ad9662e1 2026-03-29 00:56:05,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:05,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5952cb62322ad9662e1 HTTP/1.1" 200 None 2026-03-29 00:56:05,652 - lava_callback - DEBUG - Processing node 69c5d5952cb62322ad9662e1: baseline-arm64-baylibre 2026-03-29 00:56:05,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d5952cb62322ad9662e1 2026-03-29 00:56:05,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:05,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5952cb62322ad9662e1 HTTP/1.1" 200 None 2026-03-29 00:56:05,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:05,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 00:56:05,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:05,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5952cb62322ad9662e1 HTTP/1.1" 200 None 2026-03-29 00:56:05,752 - lava_callback - ERROR - Error processing callback for node 69c5d5952cb62322ad9662e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:56:06,263 - lava_callback - INFO - Received callback for node 69c5d5982cb62322ad9662e3 INFO: 172.20.0.1:39332 - "POST /node/69c5d5982cb62322ad9662e3 HTTP/1.0" 202 Accepted 2026-03-29 00:56:06,272 - lava_callback - INFO - Starting async job submit for node 69c5d5982cb62322ad9662e3 2026-03-29 00:56:06,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:06,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5982cb62322ad9662e3 HTTP/1.1" 200 None 2026-03-29 00:56:06,305 - lava_callback - DEBUG - Processing node 69c5d5982cb62322ad9662e3: baseline-arm-baylibre 2026-03-29 00:56:06,305 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d5982cb62322ad9662e3 2026-03-29 00:56:06,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:06,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5982cb62322ad9662e3 HTTP/1.1" 200 None 2026-03-29 00:56:06,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:06,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 00:56:06,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:06,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5982cb62322ad9662e3 HTTP/1.1" 200 None 2026-03-29 00:56:06,397 - lava_callback - ERROR - Error processing callback for node 69c5d5982cb62322ad9662e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55240 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:56:44,382 - lava_callback - INFO - Received callback for node 69c5d5b92cb62322ad966352 INFO: 172.20.0.1:39910 - "POST /node/69c5d5b92cb62322ad966352 HTTP/1.0" 202 Accepted 2026-03-29 00:56:44,388 - lava_callback - INFO - Starting async job submit for node 69c5d5b92cb62322ad966352 2026-03-29 00:56:44,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:44,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5b92cb62322ad966352 HTTP/1.1" 200 None 2026-03-29 00:56:44,404 - lava_callback - DEBUG - Processing node 69c5d5b92cb62322ad966352: baseline-arm64-clabbe 2026-03-29 00:56:44,404 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d5b92cb62322ad966352 2026-03-29 00:56:44,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:44,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5b92cb62322ad966352 HTTP/1.1" 200 None 2026-03-29 00:56:44,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:44,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 00:56:44,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:44,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5b92cb62322ad966352 HTTP/1.1" 200 None 2026-03-29 00:56:44,481 - lava_callback - ERROR - Error processing callback for node 69c5d5b92cb62322ad966352: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:56:45,074 - lava_callback - INFO - Received callback for node 69c5d5bc2cb62322ad966355 INFO: 172.20.0.1:39922 - "POST /node/69c5d5bc2cb62322ad966355 HTTP/1.0" 202 Accepted 2026-03-29 00:56:45,082 - lava_callback - INFO - Starting async job submit for node 69c5d5bc2cb62322ad966355 2026-03-29 00:56:45,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:45,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5bc2cb62322ad966355 HTTP/1.1" 200 None 2026-03-29 00:56:45,109 - lava_callback - DEBUG - Processing node 69c5d5bc2cb62322ad966355: baseline-arm64-clabbe 2026-03-29 00:56:45,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d5bc2cb62322ad966355 2026-03-29 00:56:45,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:45,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5bc2cb62322ad966355 HTTP/1.1" 200 None 2026-03-29 00:56:45,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:45,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 00:56:45,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:45,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5bc2cb62322ad966355 HTTP/1.1" 200 None 2026-03-29 00:56:45,204 - lava_callback - ERROR - Error processing callback for node 69c5d5bc2cb62322ad966355: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:56:45,769 - lava_callback - INFO - Received callback for node 69c5d5be2cb62322ad966370 INFO: 172.20.0.1:39930 - "POST /node/69c5d5be2cb62322ad966370 HTTP/1.0" 202 Accepted 2026-03-29 00:56:45,775 - lava_callback - INFO - Starting async job submit for node 69c5d5be2cb62322ad966370 2026-03-29 00:56:45,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:45,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5be2cb62322ad966370 HTTP/1.1" 200 None 2026-03-29 00:56:45,795 - lava_callback - DEBUG - Processing node 69c5d5be2cb62322ad966370: baseline-arm64-clabbe 2026-03-29 00:56:45,795 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d5be2cb62322ad966370 2026-03-29 00:56:45,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:45,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5be2cb62322ad966370 HTTP/1.1" 200 None 2026-03-29 00:56:45,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:45,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 00:56:45,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:45,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5be2cb62322ad966370 HTTP/1.1" 200 None 2026-03-29 00:56:45,863 - lava_callback - ERROR - Error processing callback for node 69c5d5be2cb62322ad966370: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:56:46,462 - lava_callback - INFO - Received callback for node 69c5d5c02cb62322ad966373 INFO: 172.20.0.1:39946 - "POST /node/69c5d5c02cb62322ad966373 HTTP/1.0" 202 Accepted 2026-03-29 00:56:46,470 - lava_callback - INFO - Starting async job submit for node 69c5d5c02cb62322ad966373 2026-03-29 00:56:46,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:46,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c02cb62322ad966373 HTTP/1.1" 200 None 2026-03-29 00:56:46,498 - lava_callback - DEBUG - Processing node 69c5d5c02cb62322ad966373: baseline-arm64-clabbe 2026-03-29 00:56:46,499 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d5c02cb62322ad966373 2026-03-29 00:56:46,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:39956 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:56:46,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c02cb62322ad966373 HTTP/1.1" 200 None 2026-03-29 00:56:46,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:46,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 00:56:46,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:46,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5c02cb62322ad966373 HTTP/1.1" 200 None 2026-03-29 00:56:46,590 - lava_callback - ERROR - Error processing callback for node 69c5d5c02cb62322ad966373: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:56:47,161 - lava_callback - INFO - Received callback for node 69c5d5c42cb62322ad96637c INFO: 172.20.0.1:39962 - "POST /node/69c5d5c42cb62322ad96637c HTTP/1.0" 202 Accepted 2026-03-29 00:56:47,169 - lava_callback - INFO - Starting async job submit for node 69c5d5c42cb62322ad96637c 2026-03-29 00:56:47,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:47,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c42cb62322ad96637c HTTP/1.1" 200 None 2026-03-29 00:56:47,201 - lava_callback - DEBUG - Processing node 69c5d5c42cb62322ad96637c: baseline-arm64-clabbe 2026-03-29 00:56:47,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d5c42cb62322ad96637c 2026-03-29 00:56:47,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:47,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c42cb62322ad96637c HTTP/1.1" 200 None 2026-03-29 00:56:47,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:47,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 00:56:47,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:47,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5c42cb62322ad96637c HTTP/1.1" 200 None 2026-03-29 00:56:47,301 - lava_callback - ERROR - Error processing callback for node 69c5d5c42cb62322ad96637c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 00:56:47,848 - lava_callback - INFO - Received callback for node 69c5d5c62cb62322ad96637f INFO: 172.20.0.1:39976 - "POST /node/69c5d5c62cb62322ad96637f HTTP/1.0" 202 Accepted 2026-03-29 00:56:47,858 - lava_callback - INFO - Starting async job submit for node 69c5d5c62cb62322ad96637f 2026-03-29 00:56:47,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:47,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c62cb62322ad96637f HTTP/1.1" 200 None 2026-03-29 00:56:47,889 - lava_callback - DEBUG - Processing node 69c5d5c62cb62322ad96637f: baseline-arm64-clabbe 2026-03-29 00:56:47,889 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d5c62cb62322ad96637f 2026-03-29 00:56:47,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:47,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5c62cb62322ad96637f HTTP/1.1" 200 None 2026-03-29 00:56:47,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:47,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 00:56:47,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:56:47,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5c62cb62322ad96637f HTTP/1.1" 200 None 2026-03-29 00:56:47,989 - lava_callback - ERROR - Error processing callback for node 69c5d5c62cb62322ad96637f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42816 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:57:25,062 - lava_callback - INFO - Received callback for node 69c5d5eb2cb62322ad9663cb INFO: 172.20.0.1:57222 - "POST /node/69c5d5eb2cb62322ad9663cb HTTP/1.0" 202 Accepted 2026-03-29 00:57:25,071 - lava_callback - INFO - Starting async job submit for node 69c5d5eb2cb62322ad9663cb 2026-03-29 00:57:25,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:57:25,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5eb2cb62322ad9663cb HTTP/1.1" 200 None 2026-03-29 00:57:25,097 - lava_callback - DEBUG - Processing node 69c5d5eb2cb62322ad9663cb: baseline-arm-baylibre 2026-03-29 00:57:25,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d5eb2cb62322ad9663cb 2026-03-29 00:57:25,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:57:25,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d5eb2cb62322ad9663cb HTTP/1.1" 200 None 2026-03-29 00:57:25,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:57:25,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 00:57:25,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:57:25,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d5eb2cb62322ad9663cb HTTP/1.1" 200 None 2026-03-29 00:57:25,199 - lava_callback - ERROR - Error processing callback for node 69c5d5eb2cb62322ad9663cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48756 - "GET /node/69942d0b5ae459b9df3d1493%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48758 - "GET /node/693a12fb78d7681e6b490241%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36478 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:58:36,957 - lava_callback - INFO - Received callback for node 69c8792bf41e4fb6a3785985 INFO: 172.20.0.1:36494 - "POST /node/69c8792bf41e4fb6a3785985 HTTP/1.0" 202 Accepted 2026-03-29 00:58:36,962 - lava_callback - INFO - Starting async job submit for node 69c8792bf41e4fb6a3785985 2026-03-29 00:58:36,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:58:36,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8792bf41e4fb6a3785985 HTTP/1.1" 200 None 2026-03-29 00:58:36,980 - lava_callback - DEBUG - Processing node 69c8792bf41e4fb6a3785985: baseline-arm 2026-03-29 00:58:36,985 - lava_callback - WARNING - Failed to upload log 2026-03-29 00:58:36,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:58:37,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:58:37,127 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c8792bf41e4fb6a3785985/lava-logs.yaml 2026-03-29 00:58:37,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:58:37,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:58:37,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c8792bf41e4fb6a3785985/lava_callback.json.gz 2026-03-29 00:58:37,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:58:37,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8792bf41e4fb6a3785985 HTTP/1.1" 200 None 2026-03-29 00:58:37,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:58:37,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:58:37,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:58:37,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8792bf41e4fb6a3785985 HTTP/1.1" 200 None 2026-03-29 00:58:37,328 - lava_callback - INFO - Completed processing callback for node 69c8792bf41e4fb6a3785985 2026-03-29 00:58:44,449 - lava_callback - INFO - Received callback for node 69c5d63c2cb62322ad96642e INFO: 172.20.0.1:43966 - "POST /node/69c5d63c2cb62322ad96642e HTTP/1.0" 202 Accepted 2026-03-29 00:58:44,454 - lava_callback - INFO - Starting async job submit for node 69c5d63c2cb62322ad96642e 2026-03-29 00:58:44,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:58:44,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d63c2cb62322ad96642e HTTP/1.1" 200 None 2026-03-29 00:58:44,470 - lava_callback - DEBUG - Processing node 69c5d63c2cb62322ad96642e: baseline-arm-clabbe 2026-03-29 00:58:44,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d63c2cb62322ad96642e 2026-03-29 00:58:44,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:58:44,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d63c2cb62322ad96642e HTTP/1.1" 200 None 2026-03-29 00:58:44,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:58:44,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-29 00:58:44,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:58:44,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d63c2cb62322ad96642e HTTP/1.1" 200 None 2026-03-29 00:58:44,567 - lava_callback - ERROR - Error processing callback for node 69c5d63c2cb62322ad96642e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43972 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:58:50,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:58:50,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35098 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:59:17,087 - lava_callback - INFO - Received callback for node 69c874dcf41e4fb6a378569f INFO: 172.20.0.1:35106 - "POST /node/69c874dcf41e4fb6a378569f HTTP/1.0" 202 Accepted 2026-03-29 00:59:17,133 - lava_callback - INFO - Starting async job submit for node 69c874dcf41e4fb6a378569f 2026-03-29 00:59:17,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:59:17,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874dcf41e4fb6a378569f HTTP/1.1" 200 None 2026-03-29 00:59:17,174 - lava_callback - DEBUG - Processing node 69c874dcf41e4fb6a378569f: baseline-arm-pengutronix 2026-03-29 00:59:17,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:59:17,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:59:17,430 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874dcf41e4fb6a378569f/log.txt.gz 2026-03-29 00:59:17,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:59:17,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:59:17,656 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874dcf41e4fb6a378569f/lava-logs.yaml 2026-03-29 00:59:17,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:59:17,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:59:17,773 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c874dcf41e4fb6a378569f/lava_callback.json.gz DEBUG: Setup failed for 69c874dcf41e4fb6a378569f. Transitting job node result: incomplete -> fail 2026-03-29 00:59:17,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:59:17,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874dcf41e4fb6a378569f HTTP/1.1" 200 None 2026-03-29 00:59:17,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:59:17,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:59:17,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:59:17,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874dcf41e4fb6a378569f HTTP/1.1" 200 None 2026-03-29 00:59:17,950 - lava_callback - INFO - Completed processing callback for node 69c874dcf41e4fb6a378569f 2026-03-29 00:59:20,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:59:20,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32982 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:59:34,191 - lava_callback - INFO - Received callback for node 69c874bcf41e4fb6a3785617 INFO: 172.20.0.1:53998 - "POST /node/69c874bcf41e4fb6a3785617 HTTP/1.0" 202 Accepted 2026-03-29 00:59:34,198 - lava_callback - INFO - Starting async job submit for node 69c874bcf41e4fb6a3785617 2026-03-29 00:59:34,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:59:34,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874bcf41e4fb6a3785617 HTTP/1.1" 200 None 2026-03-29 00:59:34,272 - lava_callback - DEBUG - Processing node 69c874bcf41e4fb6a3785617: baseline-arm 2026-03-29 00:59:34,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:59:35,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:59:35,035 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bcf41e4fb6a3785617/log.txt.gz 2026-03-29 00:59:35,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:59:35,650 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:59:35,652 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bcf41e4fb6a3785617/lava-logs.yaml 2026-03-29 00:59:35,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 00:59:35,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 00:59:35,860 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c874bcf41e4fb6a3785617/lava_callback.json.gz DEBUG: Setup failed for 69c874bcf41e4fb6a3785617. Transitting job node result: incomplete -> fail 2026-03-29 00:59:35,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:59:35,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c874bcf41e4fb6a3785617 HTTP/1.1" 200 None 2026-03-29 00:59:35,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:59:35,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 00:59:36,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:59:36,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c874bcf41e4fb6a3785617 HTTP/1.1" 200 None 2026-03-29 00:59:36,138 - lava_callback - INFO - Completed processing callback for node 69c874bcf41e4fb6a3785617 INFO: 172.20.0.1:54006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55886 - "GET / HTTP/1.0" 200 OK 2026-03-29 00:59:50,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 00:59:50,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43348 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:00:05,176 - lava_callback - INFO - Received callback for node 69c5d6842cb62322ad966449 INFO: 172.20.0.1:48700 - "POST /node/69c5d6842cb62322ad966449 HTTP/1.0" 202 Accepted 2026-03-29 01:00:05,182 - lava_callback - INFO - Starting async job submit for node 69c5d6842cb62322ad966449 2026-03-29 01:00:05,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:00:05,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d6842cb62322ad966449 HTTP/1.1" 200 None 2026-03-29 01:00:05,198 - lava_callback - DEBUG - Processing node 69c5d6842cb62322ad966449: baseline-arm-baylibre 2026-03-29 01:00:05,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d6842cb62322ad966449 2026-03-29 01:00:05,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:00:05,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d6842cb62322ad966449 HTTP/1.1" 200 None 2026-03-29 01:00:05,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:00:05,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 01:00:05,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:00:05,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d6842cb62322ad966449 HTTP/1.1" 200 None 2026-03-29 01:00:05,256 - lava_callback - ERROR - Error processing callback for node 69c5d6842cb62322ad966449: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41244 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:00:45,232 - lava_callback - INFO - Received callback for node 69c5d6ac2cb62322ad966460 INFO: 172.20.0.1:57714 - "POST /node/69c5d6ac2cb62322ad966460 HTTP/1.0" 202 Accepted 2026-03-29 01:00:45,237 - lava_callback - INFO - Starting async job submit for node 69c5d6ac2cb62322ad966460 2026-03-29 01:00:45,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:00:45,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d6ac2cb62322ad966460 HTTP/1.1" 200 None 2026-03-29 01:00:45,254 - lava_callback - DEBUG - Processing node 69c5d6ac2cb62322ad966460: baseline-arm-baylibre 2026-03-29 01:00:45,254 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d6ac2cb62322ad966460 2026-03-29 01:00:45,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:00:45,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d6ac2cb62322ad966460 HTTP/1.1" 200 None 2026-03-29 01:00:45,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:00:45,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 01:00:45,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:00:45,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d6ac2cb62322ad966460 HTTP/1.1" 200 None 2026-03-29 01:00:45,309 - lava_callback - ERROR - Error processing callback for node 69c5d6ac2cb62322ad966460: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48270 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:02:25,289 - lava_callback - INFO - Received callback for node 69c5d71f2cb62322ad968a2d INFO: 172.20.0.1:49224 - "POST /node/69c5d71f2cb62322ad968a2d HTTP/1.0" 202 Accepted 2026-03-29 01:02:25,294 - lava_callback - INFO - Starting async job submit for node 69c5d71f2cb62322ad968a2d 2026-03-29 01:02:25,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:02:25,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d71f2cb62322ad968a2d HTTP/1.1" 200 None 2026-03-29 01:02:25,311 - lava_callback - DEBUG - Processing node 69c5d71f2cb62322ad968a2d: baseline-arm-baylibre 2026-03-29 01:02:25,312 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d71f2cb62322ad968a2d 2026-03-29 01:02:25,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:02:25,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d71f2cb62322ad968a2d HTTP/1.1" 200 None 2026-03-29 01:02:25,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:02:25,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 01:02:25,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:02:25,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d71f2cb62322ad968a2d HTTP/1.1" 200 None 2026-03-29 01:02:25,373 - lava_callback - ERROR - Error processing callback for node 69c5d71f2cb62322ad968a2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48632 - "GET /node/69117010f15111fc8ce49eb5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36210 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:03:44,653 - lava_callback - INFO - Received callback for node 69c5d76d2cb62322ad968a4b INFO: 172.20.0.1:36188 - "POST /node/69c5d76d2cb62322ad968a4b HTTP/1.0" 202 Accepted 2026-03-29 01:03:44,658 - lava_callback - INFO - Starting async job submit for node 69c5d76d2cb62322ad968a4b 2026-03-29 01:03:44,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:03:44,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d76d2cb62322ad968a4b HTTP/1.1" 200 None 2026-03-29 01:03:44,674 - lava_callback - DEBUG - Processing node 69c5d76d2cb62322ad968a4b: baseline-arm-clabbe 2026-03-29 01:03:44,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d76d2cb62322ad968a4b 2026-03-29 01:03:44,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:03:44,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d76d2cb62322ad968a4b HTTP/1.1" 200 None 2026-03-29 01:03:44,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:03:44,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 01:03:44,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:03:44,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d76d2cb62322ad968a4b HTTP/1.1" 200 None 2026-03-29 01:03:44,735 - lava_callback - ERROR - Error processing callback for node 69c5d76d2cb62322ad968a4b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36212 - "GET /node/6988e68b9fc35f969e1f0169%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57232 - "GET /node/693a15a278d7681e6b490695%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57242 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:04:44,712 - lava_callback - INFO - Received callback for node 69c5d7972cb62322ad968a56 INFO: 172.20.0.1:35664 - "POST /node/69c5d7972cb62322ad968a56 HTTP/1.0" 202 Accepted 2026-03-29 01:04:44,718 - lava_callback - INFO - Starting async job submit for node 69c5d7972cb62322ad968a56 2026-03-29 01:04:44,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:04:44,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d7972cb62322ad968a56 HTTP/1.1" 200 None 2026-03-29 01:04:44,742 - lava_callback - DEBUG - Processing node 69c5d7972cb62322ad968a56: baseline-arm-clabbe 2026-03-29 01:04:44,742 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d7972cb62322ad968a56 2026-03-29 01:04:44,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:04:44,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d7972cb62322ad968a56 HTTP/1.1" 200 None 2026-03-29 01:04:44,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:04:44,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 01:04:44,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:04:44,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d7972cb62322ad968a56 HTTP/1.1" 200 None 2026-03-29 01:04:44,804 - lava_callback - ERROR - Error processing callback for node 69c5d7972cb62322ad968a56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48594 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:05:24,677 - lava_callback - INFO - Received callback for node 69c5d7be2cb62322ad968a69 INFO: 172.20.0.1:41892 - "POST /node/69c5d7be2cb62322ad968a69 HTTP/1.0" 202 Accepted 2026-03-29 01:05:24,682 - lava_callback - INFO - Starting async job submit for node 69c5d7be2cb62322ad968a69 2026-03-29 01:05:24,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:05:24,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d7be2cb62322ad968a69 HTTP/1.1" 200 None 2026-03-29 01:05:24,708 - lava_callback - DEBUG - Processing node 69c5d7be2cb62322ad968a69: baseline-arm-clabbe 2026-03-29 01:05:24,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d7be2cb62322ad968a69 2026-03-29 01:05:24,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:05:24,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d7be2cb62322ad968a69 HTTP/1.1" 200 None 2026-03-29 01:05:24,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:05:24,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 01:05:24,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:05:24,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d7be2cb62322ad968a69 HTTP/1.1" 200 None 2026-03-29 01:05:24,776 - lava_callback - ERROR - Error processing callback for node 69c5d7be2cb62322ad968a69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48528 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:06:24,760 - lava_callback - INFO - Received callback for node 69c5d80a2cb62322ad968a7d INFO: 172.20.0.1:54166 - "POST /node/69c5d80a2cb62322ad968a7d HTTP/1.0" 202 Accepted 2026-03-29 01:06:24,769 - lava_callback - INFO - Starting async job submit for node 69c5d80a2cb62322ad968a7d 2026-03-29 01:06:24,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:06:24,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d80a2cb62322ad968a7d HTTP/1.1" 200 None 2026-03-29 01:06:24,787 - lava_callback - DEBUG - Processing node 69c5d80a2cb62322ad968a7d: baseline-arm-clabbe 2026-03-29 01:06:24,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d80a2cb62322ad968a7d 2026-03-29 01:06:24,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:06:24,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d80a2cb62322ad968a7d HTTP/1.1" 200 None 2026-03-29 01:06:24,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:06:24,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 01:06:24,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:06:24,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d80a2cb62322ad968a7d HTTP/1.1" 200 None 2026-03-29 01:06:24,850 - lava_callback - ERROR - Error processing callback for node 69c5d80a2cb62322ad968a7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43950 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:07:44,821 - lava_callback - INFO - Received callback for node 69c5d8572cb62322ad968a9b INFO: 172.20.0.1:52282 - "POST /node/69c5d8572cb62322ad968a9b HTTP/1.0" 202 Accepted 2026-03-29 01:07:44,828 - lava_callback - INFO - Starting async job submit for node 69c5d8572cb62322ad968a9b 2026-03-29 01:07:44,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:07:44,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d8572cb62322ad968a9b HTTP/1.1" 200 None 2026-03-29 01:07:44,856 - lava_callback - DEBUG - Processing node 69c5d8572cb62322ad968a9b: baseline-arm-clabbe 2026-03-29 01:07:44,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d8572cb62322ad968a9b 2026-03-29 01:07:44,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:07:44,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d8572cb62322ad968a9b HTTP/1.1" 200 None 2026-03-29 01:07:44,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:07:44,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 01:07:44,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:07:44,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d8572cb62322ad968a9b HTTP/1.1" 200 None 2026-03-29 01:07:44,929 - lava_callback - ERROR - Error processing callback for node 69c5d8572cb62322ad968a9b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35776 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:09:04,898 - lava_callback - INFO - Received callback for node 69c5d8a32cb62322ad968aad INFO: 172.20.0.1:37314 - "POST /node/69c5d8a32cb62322ad968aad HTTP/1.0" 202 Accepted 2026-03-29 01:09:04,904 - lava_callback - INFO - Starting async job submit for node 69c5d8a32cb62322ad968aad 2026-03-29 01:09:04,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:09:04,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d8a32cb62322ad968aad HTTP/1.1" 200 None 2026-03-29 01:09:04,938 - lava_callback - DEBUG - Processing node 69c5d8a32cb62322ad968aad: baseline-arm-clabbe 2026-03-29 01:09:04,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5d8a32cb62322ad968aad 2026-03-29 01:09:04,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:09:04,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d8a32cb62322ad968aad HTTP/1.1" 200 None 2026-03-29 01:09:04,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:09:04,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-29 01:09:04,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:09:04,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d8a32cb62322ad968aad HTTP/1.1" 200 None 2026-03-29 01:09:05,000 - lava_callback - ERROR - Error processing callback for node 69c5d8a32cb62322ad968aad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47086 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:15:04,481 - lava_callback - INFO - Received callback for node 69c878e2f41e4fb6a378596a INFO: 172.20.0.1:37098 - "POST /node/69c878e2f41e4fb6a378596a HTTP/1.0" 202 Accepted 2026-03-29 01:15:04,488 - lava_callback - INFO - Starting async job submit for node 69c878e2f41e4fb6a378596a 2026-03-29 01:15:04,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:15:04,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c878e2f41e4fb6a378596a HTTP/1.1" 200 None 2026-03-29 01:15:04,561 - lava_callback - DEBUG - Processing node 69c878e2f41e4fb6a378596a: ltp-smoke 2026-03-29 01:15:05,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:15:05,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:15:05,182 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c878e2f41e4fb6a378596a/log.txt.gz 2026-03-29 01:15:05,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:15:05,705 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:15:05,708 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c878e2f41e4fb6a378596a/lava-logs.yaml 2026-03-29 01:15:05,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:15:05,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:15:05,902 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c878e2f41e4fb6a378596a/lava_callback.json.gz 2026-03-29 01:15:06,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:15:06,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c878e2f41e4fb6a378596a HTTP/1.1" 200 None 2026-03-29 01:15:06,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:15:06,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f76f41e4fb6a3782f44 HTTP/1.1" 200 None 2026-03-29 01:15:06,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:15:06,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c878e2f41e4fb6a378596a HTTP/1.1" 200 None 2026-03-29 01:15:06,219 - lava_callback - INFO - Completed processing callback for node 69c878e2f41e4fb6a378596a INFO: 172.20.0.1:37100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39538 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:15:20,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:15:20,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 01:15:25,163 - lava_callback - INFO - Received callback for node 69c5da1f2cb62322ad968afe INFO: 172.20.0.1:33166 - "POST /node/69c5da1f2cb62322ad968afe HTTP/1.0" 202 Accepted 2026-03-29 01:15:25,168 - lava_callback - INFO - Starting async job submit for node 69c5da1f2cb62322ad968afe 2026-03-29 01:15:25,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:15:25,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5da1f2cb62322ad968afe HTTP/1.1" 200 None 2026-03-29 01:15:25,185 - lava_callback - DEBUG - Processing node 69c5da1f2cb62322ad968afe: baseline-arm-clabbe 2026-03-29 01:15:25,185 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5da1f2cb62322ad968afe 2026-03-29 01:15:25,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:15:25,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5da1f2cb62322ad968afe HTTP/1.1" 200 None 2026-03-29 01:15:25,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:15:25,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-29 01:15:25,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:15:25,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5da1f2cb62322ad968afe HTTP/1.1" 200 None 2026-03-29 01:15:25,245 - lava_callback - ERROR - Error processing callback for node 69c5da1f2cb62322ad968afe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48028 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:28:06,718 - lava_callback - INFO - Received callback for node 69c5dd172cb62322ad968b94 INFO: 172.20.0.1:48040 - "POST /node/69c5dd172cb62322ad968b94 HTTP/1.0" 202 Accepted 2026-03-29 01:28:06,724 - lava_callback - INFO - Starting async job submit for node 69c5dd172cb62322ad968b94 2026-03-29 01:28:06,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:28:06,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dd172cb62322ad968b94 HTTP/1.1" 200 None 2026-03-29 01:28:06,748 - lava_callback - DEBUG - Processing node 69c5dd172cb62322ad968b94: baseline-arm64-baylibre 2026-03-29 01:28:06,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5dd172cb62322ad968b94 2026-03-29 01:28:06,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:28:06,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dd172cb62322ad968b94 HTTP/1.1" 200 None 2026-03-29 01:28:06,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:28:06,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:28:06,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:28:06,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5dd172cb62322ad968b94 HTTP/1.1" 200 None 2026-03-29 01:28:06,809 - lava_callback - ERROR - Error processing callback for node 69c5dd172cb62322ad968b94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57272 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:29:26,799 - lava_callback - INFO - Received callback for node 69c5dd6c2cb62322ad968bb4 INFO: 172.20.0.1:57282 - "POST /node/69c5dd6c2cb62322ad968bb4 HTTP/1.0" 202 Accepted 2026-03-29 01:29:26,804 - lava_callback - INFO - Starting async job submit for node 69c5dd6c2cb62322ad968bb4 2026-03-29 01:29:26,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:29:26,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dd6c2cb62322ad968bb4 HTTP/1.1" 200 None 2026-03-29 01:29:26,830 - lava_callback - DEBUG - Processing node 69c5dd6c2cb62322ad968bb4: baseline-arm64-baylibre 2026-03-29 01:29:26,830 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5dd6c2cb62322ad968bb4 2026-03-29 01:29:26,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:29:26,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dd6c2cb62322ad968bb4 HTTP/1.1" 200 None 2026-03-29 01:29:26,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:29:26,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:29:26,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:29:26,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5dd6c2cb62322ad968bb4 HTTP/1.1" 200 None 2026-03-29 01:29:26,919 - lava_callback - ERROR - Error processing callback for node 69c5dd6c2cb62322ad968bb4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54468 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:30:46,821 - lava_callback - INFO - Received callback for node 69c5ddc22cb62322ad968bcd INFO: 172.20.0.1:54472 - "POST /node/69c5ddc22cb62322ad968bcd HTTP/1.0" 202 Accepted 2026-03-29 01:30:46,826 - lava_callback - INFO - Starting async job submit for node 69c5ddc22cb62322ad968bcd 2026-03-29 01:30:46,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:30:46,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ddc22cb62322ad968bcd HTTP/1.1" 200 None 2026-03-29 01:30:46,842 - lava_callback - DEBUG - Processing node 69c5ddc22cb62322ad968bcd: baseline-arm64-baylibre 2026-03-29 01:30:46,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5ddc22cb62322ad968bcd 2026-03-29 01:30:46,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:30:46,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ddc22cb62322ad968bcd HTTP/1.1" 200 None 2026-03-29 01:30:46,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:30:46,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:30:46,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:30:46,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ddc22cb62322ad968bcd HTTP/1.1" 200 None 2026-03-29 01:30:46,906 - lava_callback - ERROR - Error processing callback for node 69c5ddc22cb62322ad968bcd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50324 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:32:26,904 - lava_callback - INFO - Received callback for node 69c5de162cb62322ad968bef INFO: 172.20.0.1:50340 - "POST /node/69c5de162cb62322ad968bef HTTP/1.0" 202 Accepted 2026-03-29 01:32:26,910 - lava_callback - INFO - Starting async job submit for node 69c5de162cb62322ad968bef 2026-03-29 01:32:26,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:32:26,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5de162cb62322ad968bef HTTP/1.1" 200 None 2026-03-29 01:32:26,949 - lava_callback - DEBUG - Processing node 69c5de162cb62322ad968bef: baseline-arm64-baylibre 2026-03-29 01:32:26,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5de162cb62322ad968bef 2026-03-29 01:32:26,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:32:26,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5de162cb62322ad968bef HTTP/1.1" 200 None 2026-03-29 01:32:26,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:32:26,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:32:26,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:32:27,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5de162cb62322ad968bef HTTP/1.1" 200 None 2026-03-29 01:32:27,001 - lava_callback - ERROR - Error processing callback for node 69c5de162cb62322ad968bef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60214 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:33:47,023 - lava_callback - INFO - Received callback for node 69c5de6a2cb62322ad968c0f INFO: 172.20.0.1:60228 - "POST /node/69c5de6a2cb62322ad968c0f HTTP/1.0" 202 Accepted 2026-03-29 01:33:47,029 - lava_callback - INFO - Starting async job submit for node 69c5de6a2cb62322ad968c0f 2026-03-29 01:33:47,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:33:47,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5de6a2cb62322ad968c0f HTTP/1.1" 200 None 2026-03-29 01:33:47,045 - lava_callback - DEBUG - Processing node 69c5de6a2cb62322ad968c0f: baseline-arm64-baylibre 2026-03-29 01:33:47,045 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5de6a2cb62322ad968c0f 2026-03-29 01:33:47,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:33:47,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5de6a2cb62322ad968c0f HTTP/1.1" 200 None 2026-03-29 01:33:47,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:33:47,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:33:47,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:33:47,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5de6a2cb62322ad968c0f HTTP/1.1" 200 None 2026-03-29 01:33:47,120 - lava_callback - ERROR - Error processing callback for node 69c5de6a2cb62322ad968c0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40512 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:33:59,832 - lava_callback - INFO - Received callback for node 69c87d69f41e4fb6a3785aa0 INFO: 172.20.0.1:44110 - "POST /node/69c87d69f41e4fb6a3785aa0 HTTP/1.0" 202 Accepted 2026-03-29 01:33:59,839 - lava_callback - INFO - Starting async job submit for node 69c87d69f41e4fb6a3785aa0 2026-03-29 01:33:59,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:33:59,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87d69f41e4fb6a3785aa0 HTTP/1.1" 200 None 2026-03-29 01:33:59,907 - lava_callback - DEBUG - Processing node 69c87d69f41e4fb6a3785aa0: baseline-arm 2026-03-29 01:34:00,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:34:00,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:34:00,642 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c87d69f41e4fb6a3785aa0/log.txt.gz 2026-03-29 01:34:01,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:34:01,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:34:01,253 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c87d69f41e4fb6a3785aa0/lava-logs.yaml 2026-03-29 01:34:01,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:34:01,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:34:01,615 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c87d69f41e4fb6a3785aa0/lava_callback.json.gz DEBUG: Setup failed for 69c87d69f41e4fb6a3785aa0. Transitting job node result: incomplete -> fail 2026-03-29 01:34:01,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:34:01,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87d69f41e4fb6a3785aa0 HTTP/1.1" 200 None 2026-03-29 01:34:01,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:34:01,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 01:34:01,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:34:01,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87d69f41e4fb6a3785aa0 HTTP/1.1" 200 None 2026-03-29 01:34:01,889 - lava_callback - INFO - Completed processing callback for node 69c87d69f41e4fb6a3785aa0 INFO: 172.20.0.1:44122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42406 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:34:20,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:34:20,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40566 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:35:07,041 - lava_callback - INFO - Received callback for node 69c5dec02cb62322ad968c14 INFO: 172.20.0.1:40582 - "POST /node/69c5dec02cb62322ad968c14 HTTP/1.0" 202 Accepted 2026-03-29 01:35:07,050 - lava_callback - INFO - Starting async job submit for node 69c5dec02cb62322ad968c14 2026-03-29 01:35:07,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:35:07,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dec02cb62322ad968c14 HTTP/1.1" 200 None 2026-03-29 01:35:07,075 - lava_callback - DEBUG - Processing node 69c5dec02cb62322ad968c14: baseline-arm64-baylibre 2026-03-29 01:35:07,075 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5dec02cb62322ad968c14 2026-03-29 01:35:07,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:35:07,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dec02cb62322ad968c14 HTTP/1.1" 200 None 2026-03-29 01:35:07,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:35:07,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:35:07,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:35:07,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5dec02cb62322ad968c14 HTTP/1.1" 200 None 2026-03-29 01:35:07,159 - lava_callback - ERROR - Error processing callback for node 69c5dec02cb62322ad968c14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 01:35:12,767 - lava_callback - INFO - Received callback for node 69c87d50f41e4fb6a3785a94 INFO: 172.20.0.1:36240 - "POST /node/69c87d50f41e4fb6a3785a94 HTTP/1.0" 202 Accepted 2026-03-29 01:35:12,820 - lava_callback - INFO - Starting async job submit for node 69c87d50f41e4fb6a3785a94 2026-03-29 01:35:12,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:35:12,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87d50f41e4fb6a3785a94 HTTP/1.1" 200 None 2026-03-29 01:35:12,905 - lava_callback - DEBUG - Processing node 69c87d50f41e4fb6a3785a94: baseline-arm-pengutronix 2026-03-29 01:35:13,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:35:13,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:35:13,552 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c87d50f41e4fb6a3785a94/log.txt.gz 2026-03-29 01:35:13,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:35:13,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:35:13,823 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c87d50f41e4fb6a3785a94/lava-logs.yaml 2026-03-29 01:35:13,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:35:13,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:35:13,963 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c87d50f41e4fb6a3785a94/lava_callback.json.gz DEBUG: Setup failed for 69c87d50f41e4fb6a3785a94. Transitting job node result: incomplete -> fail 2026-03-29 01:35:14,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:35:14,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87d50f41e4fb6a3785a94 HTTP/1.1" 200 None 2026-03-29 01:35:14,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:35:14,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 01:35:14,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:35:14,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87d50f41e4fb6a3785a94 HTTP/1.1" 200 None 2026-03-29 01:35:14,132 - lava_callback - INFO - Completed processing callback for node 69c87d50f41e4fb6a3785a94 INFO: 172.20.0.1:36242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36256 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:35:20,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:35:20,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60666 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:36:46,014 - lava_callback - INFO - Received callback for node 69c5df1b2cb62322ad968c1d INFO: 172.20.0.1:38920 - "POST /node/69c5df1b2cb62322ad968c1d HTTP/1.0" 202 Accepted 2026-03-29 01:36:46,021 - lava_callback - INFO - Starting async job submit for node 69c5df1b2cb62322ad968c1d 2026-03-29 01:36:46,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:36:46,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5df1b2cb62322ad968c1d HTTP/1.1" 200 None 2026-03-29 01:36:46,039 - lava_callback - DEBUG - Processing node 69c5df1b2cb62322ad968c1d: baseline-arm64-clabbe 2026-03-29 01:36:46,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5df1b2cb62322ad968c1d 2026-03-29 01:36:46,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:36:46,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5df1b2cb62322ad968c1d HTTP/1.1" 200 None 2026-03-29 01:36:46,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:36:46,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:36:46,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:36:46,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5df1b2cb62322ad968c1d HTTP/1.1" 200 None 2026-03-29 01:36:46,117 - lava_callback - ERROR - Error processing callback for node 69c5df1b2cb62322ad968c1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56016 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:38:06,017 - lava_callback - INFO - Received callback for node 69c5df722cb62322ad968c38 INFO: 172.20.0.1:52000 - "POST /node/69c5df722cb62322ad968c38 HTTP/1.0" 202 Accepted 2026-03-29 01:38:06,024 - lava_callback - INFO - Starting async job submit for node 69c5df722cb62322ad968c38 2026-03-29 01:38:06,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:38:06,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5df722cb62322ad968c38 HTTP/1.1" 200 None 2026-03-29 01:38:06,045 - lava_callback - DEBUG - Processing node 69c5df722cb62322ad968c38: baseline-arm64-clabbe 2026-03-29 01:38:06,045 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5df722cb62322ad968c38 2026-03-29 01:38:06,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:38:06,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5df722cb62322ad968c38 HTTP/1.1" 200 None 2026-03-29 01:38:06,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:38:06,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:38:06,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:38:06,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5df722cb62322ad968c38 HTTP/1.1" 200 None 2026-03-29 01:38:06,101 - lava_callback - ERROR - Error processing callback for node 69c5df722cb62322ad968c38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36260 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:38:57,711 - lava_callback - INFO - Received callback for node 69c882a4f41e4fb6a3785c0b INFO: 172.20.0.1:36268 - "POST /node/69c882a4f41e4fb6a3785c0b HTTP/1.0" 202 Accepted 2026-03-29 01:38:57,718 - lava_callback - INFO - Starting async job submit for node 69c882a4f41e4fb6a3785c0b 2026-03-29 01:38:57,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:38:57,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a4f41e4fb6a3785c0b HTTP/1.1" 200 None 2026-03-29 01:38:57,743 - lava_callback - DEBUG - Processing node 69c882a4f41e4fb6a3785c0b: baseline-nfs-arm64 2026-03-29 01:38:57,744 - lava_callback - WARNING - Failed to upload log 2026-03-29 01:38:57,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:38:57,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:38:57,919 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a4f41e4fb6a3785c0b/lava-logs.yaml 2026-03-29 01:38:57,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:38:58,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:38:58,099 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a4f41e4fb6a3785c0b/lava_callback.json.gz 2026-03-29 01:38:58,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:38:58,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a4f41e4fb6a3785c0b HTTP/1.1" 200 None 2026-03-29 01:38:58,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:38:58,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:38:58,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:38:58,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882a4f41e4fb6a3785c0b HTTP/1.1" 200 None 2026-03-29 01:38:58,182 - lava_callback - INFO - Completed processing callback for node 69c882a4f41e4fb6a3785c0b 2026-03-29 01:39:05,320 - lava_callback - INFO - Received callback for node 69c88286f41e4fb6a3785bb5 INFO: 172.20.0.1:46496 - "POST /node/69c88286f41e4fb6a3785bb5 HTTP/1.0" 202 Accepted 2026-03-29 01:39:05,327 - lava_callback - INFO - Starting async job submit for node 69c88286f41e4fb6a3785bb5 2026-03-29 01:39:05,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:05,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88286f41e4fb6a3785bb5 HTTP/1.1" 200 None 2026-03-29 01:39:05,362 - lava_callback - DEBUG - Processing node 69c88286f41e4fb6a3785bb5: baseline-arm64 2026-03-29 01:39:05,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:05,688 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:05,690 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88286f41e4fb6a3785bb5/log.txt.gz 2026-03-29 01:39:05,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:06,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:06,027 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88286f41e4fb6a3785bb5/lava-logs.yaml 2026-03-29 01:39:06,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:06,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:06,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88286f41e4fb6a3785bb5/lava_callback.json.gz 2026-03-29 01:39:06,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:06,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88286f41e4fb6a3785bb5 HTTP/1.1" 200 None 2026-03-29 01:39:06,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:06,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:39:06,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:06,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88286f41e4fb6a3785bb5 HTTP/1.1" 200 None 2026-03-29 01:39:06,418 - lava_callback - INFO - Completed processing callback for node 69c88286f41e4fb6a3785bb5 INFO: 172.20.0.1:46508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39862 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:39:20,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:20,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 01:39:23,478 - lava_callback - INFO - Received callback for node 69c88288f41e4fb6a3785bbb INFO: 172.20.0.1:53586 - "POST /node/69c88288f41e4fb6a3785bbb HTTP/1.0" 202 Accepted 2026-03-29 01:39:23,485 - lava_callback - INFO - Starting async job submit for node 69c88288f41e4fb6a3785bbb 2026-03-29 01:39:23,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:23,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88288f41e4fb6a3785bbb HTTP/1.1" 200 None 2026-03-29 01:39:23,524 - lava_callback - DEBUG - Processing node 69c88288f41e4fb6a3785bbb: baseline-arm64 2026-03-29 01:39:23,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:23,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:23,957 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88288f41e4fb6a3785bbb/log.txt.gz 2026-03-29 01:39:24,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:24,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:24,439 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88288f41e4fb6a3785bbb/lava-logs.yaml 2026-03-29 01:39:24,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:24,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:24,608 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88288f41e4fb6a3785bbb/lava_callback.json.gz 2026-03-29 01:39:24,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:24,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88288f41e4fb6a3785bbb HTTP/1.1" 200 None 2026-03-29 01:39:24,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:24,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:39:24,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:24,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88288f41e4fb6a3785bbb HTTP/1.1" 200 None 2026-03-29 01:39:24,700 - lava_callback - INFO - Completed processing callback for node 69c88288f41e4fb6a3785bbb 2026-03-29 01:39:26,176 - lava_callback - INFO - Received callback for node 69c5dfc82cb62322ad968c5f INFO: 172.20.0.1:53592 - "POST /node/69c5dfc82cb62322ad968c5f HTTP/1.0" 202 Accepted 2026-03-29 01:39:26,183 - lava_callback - INFO - Starting async job submit for node 69c5dfc82cb62322ad968c5f 2026-03-29 01:39:26,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:26,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dfc82cb62322ad968c5f HTTP/1.1" 200 None 2026-03-29 01:39:26,206 - lava_callback - DEBUG - Processing node 69c5dfc82cb62322ad968c5f: baseline-arm64-clabbe 2026-03-29 01:39:26,206 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5dfc82cb62322ad968c5f 2026-03-29 01:39:26,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:26,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5dfc82cb62322ad968c5f HTTP/1.1" 200 None 2026-03-29 01:39:26,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:26,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:39:26,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:26,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5dfc82cb62322ad968c5f HTTP/1.1" 200 None 2026-03-29 01:39:26,267 - lava_callback - ERROR - Error processing callback for node 69c5dfc82cb62322ad968c5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53594 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:39:35,125 - lava_callback - INFO - Received callback for node 69c8828cf41e4fb6a3785bd2 INFO: 172.20.0.1:37998 - "POST /node/69c8828cf41e4fb6a3785bd2 HTTP/1.0" 202 Accepted 2026-03-29 01:39:35,132 - lava_callback - INFO - Starting async job submit for node 69c8828cf41e4fb6a3785bd2 2026-03-29 01:39:35,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:35,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8828cf41e4fb6a3785bd2 HTTP/1.1" 200 None 2026-03-29 01:39:35,176 - lava_callback - DEBUG - Processing node 69c8828cf41e4fb6a3785bd2: baseline-arm64 2026-03-29 01:39:35,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:35,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:35,441 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828cf41e4fb6a3785bd2/log.txt.gz 2026-03-29 01:39:35,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:35,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:35,704 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828cf41e4fb6a3785bd2/lava-logs.yaml 2026-03-29 01:39:35,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:35,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:35,841 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828cf41e4fb6a3785bd2/lava_callback.json.gz 2026-03-29 01:39:35,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:35,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8828cf41e4fb6a3785bd2 HTTP/1.1" 200 None 2026-03-29 01:39:35,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:35,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:39:35,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:35,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8828cf41e4fb6a3785bd2 HTTP/1.1" 200 None 2026-03-29 01:39:35,989 - lava_callback - INFO - Completed processing callback for node 69c8828cf41e4fb6a3785bd2 INFO: 172.20.0.1:38006 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:39:37,795 - lava_callback - INFO - Received callback for node 69c8828bf41e4fb6a3785bcf INFO: 172.20.0.1:38008 - "POST /node/69c8828bf41e4fb6a3785bcf HTTP/1.0" 202 Accepted 2026-03-29 01:39:37,802 - lava_callback - INFO - Starting async job submit for node 69c8828bf41e4fb6a3785bcf 2026-03-29 01:39:37,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:37,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8828bf41e4fb6a3785bcf HTTP/1.1" 200 None 2026-03-29 01:39:37,847 - lava_callback - DEBUG - Processing node 69c8828bf41e4fb6a3785bcf: baseline-arm64 2026-03-29 01:39:37,983 - lava_callback - WARNING - Failed to upload log 2026-03-29 01:39:38,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:38,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:38,206 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828bf41e4fb6a3785bcf/lava-logs.yaml 2026-03-29 01:39:38,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:38,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:38,334 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828bf41e4fb6a3785bcf/lava_callback.json.gz 2026-03-29 01:39:38,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:38,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8828bf41e4fb6a3785bcf HTTP/1.1" 200 None 2026-03-29 01:39:38,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:38,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:39:38,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:38,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8828bf41e4fb6a3785bcf HTTP/1.1" 200 None 2026-03-29 01:39:38,502 - lava_callback - INFO - Completed processing callback for node 69c8828bf41e4fb6a3785bcf 2026-03-29 01:39:45,767 - lava_callback - INFO - Received callback for node 69c87eaff41e4fb6a3785b10 INFO: 172.20.0.1:40358 - "POST /node/69c87eaff41e4fb6a3785b10 HTTP/1.0" 202 Accepted 2026-03-29 01:39:45,774 - lava_callback - INFO - Starting async job submit for node 69c87eaff41e4fb6a3785b10 2026-03-29 01:39:45,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:45,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87eaff41e4fb6a3785b10 HTTP/1.1" 200 None 2026-03-29 01:39:45,850 - lava_callback - DEBUG - Processing node 69c87eaff41e4fb6a3785b10: ltp-smoke 2026-03-29 01:39:46,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:46,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:46,522 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c87eaff41e4fb6a3785b10/log.txt.gz INFO: 172.20.0.1:40368 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:39:46,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:47,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:47,033 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c87eaff41e4fb6a3785b10/lava-logs.yaml 2026-03-29 01:39:47,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:47,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:47,210 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c87eaff41e4fb6a3785b10/lava_callback.json.gz 2026-03-29 01:39:47,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:47,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c87eaff41e4fb6a3785b10 HTTP/1.1" 200 None 2026-03-29 01:39:47,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:47,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f76f41e4fb6a3782f44 HTTP/1.1" 200 None 2026-03-29 01:39:47,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:47,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c87eaff41e4fb6a3785b10 HTTP/1.1" 200 None 2026-03-29 01:39:47,529 - lava_callback - INFO - Completed processing callback for node 69c87eaff41e4fb6a3785b10 2026-03-29 01:39:50,137 - lava_callback - INFO - Received callback for node 69c88299f41e4fb6a3785bf0 INFO: 172.20.0.1:52222 - "POST /node/69c88299f41e4fb6a3785bf0 HTTP/1.0" 202 Accepted 2026-03-29 01:39:50,143 - lava_callback - INFO - Starting async job submit for node 69c88299f41e4fb6a3785bf0 2026-03-29 01:39:50,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:50,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88299f41e4fb6a3785bf0 HTTP/1.1" 200 None 2026-03-29 01:39:50,181 - lava_callback - DEBUG - Processing node 69c88299f41e4fb6a3785bf0: baseline-arm64 2026-03-29 01:39:50,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:50,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:50,587 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88299f41e4fb6a3785bf0/log.txt.gz 2026-03-29 01:39:50,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:50,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:50,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 01:39:50,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:50,946 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88299f41e4fb6a3785bf0/lava-logs.yaml 2026-03-29 01:39:50,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:51,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:51,093 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88299f41e4fb6a3785bf0/lava_callback.json.gz 2026-03-29 01:39:51,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:51,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88299f41e4fb6a3785bf0 HTTP/1.1" 200 None 2026-03-29 01:39:51,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:51,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:39:51,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:51,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88299f41e4fb6a3785bf0 HTTP/1.1" 200 None 2026-03-29 01:39:51,170 - lava_callback - INFO - Completed processing callback for node 69c88299f41e4fb6a3785bf0 2026-03-29 01:39:51,960 - lava_callback - INFO - Received callback for node 69c88291f41e4fb6a3785bde INFO: 172.20.0.1:52226 - "POST /node/69c88291f41e4fb6a3785bde HTTP/1.0" 202 Accepted 2026-03-29 01:39:51,966 - lava_callback - INFO - Starting async job submit for node 69c88291f41e4fb6a3785bde 2026-03-29 01:39:51,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:52,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88291f41e4fb6a3785bde HTTP/1.1" 200 None 2026-03-29 01:39:52,014 - lava_callback - DEBUG - Processing node 69c88291f41e4fb6a3785bde: baseline-arm64 2026-03-29 01:39:52,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:52,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:52,472 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88291f41e4fb6a3785bde/log.txt.gz 2026-03-29 01:39:52,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:52,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:52,911 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88291f41e4fb6a3785bde/lava-logs.yaml 2026-03-29 01:39:52,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:53,056 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:53,057 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88291f41e4fb6a3785bde/lava_callback.json.gz 2026-03-29 01:39:53,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:53,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88291f41e4fb6a3785bde HTTP/1.1" 200 None 2026-03-29 01:39:53,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:53,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:39:53,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:53,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88291f41e4fb6a3785bde HTTP/1.1" 200 None 2026-03-29 01:39:53,137 - lava_callback - INFO - Completed processing callback for node 69c88291f41e4fb6a3785bde 2026-03-29 01:39:53,418 - lava_callback - INFO - Received callback for node 69c88298f41e4fb6a3785bed INFO: 172.20.0.1:52242 - "POST /node/69c88298f41e4fb6a3785bed HTTP/1.0" 202 Accepted 2026-03-29 01:39:53,427 - lava_callback - INFO - Starting async job submit for node 69c88298f41e4fb6a3785bed 2026-03-29 01:39:53,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:53,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88298f41e4fb6a3785bed HTTP/1.1" 200 None 2026-03-29 01:39:53,476 - lava_callback - DEBUG - Processing node 69c88298f41e4fb6a3785bed: baseline-arm64 2026-03-29 01:39:53,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:53,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:53,883 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88298f41e4fb6a3785bed/log.txt.gz 2026-03-29 01:39:54,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:54,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:54,250 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88298f41e4fb6a3785bed/lava-logs.yaml 2026-03-29 01:39:54,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:54,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:54,399 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88298f41e4fb6a3785bed/lava_callback.json.gz 2026-03-29 01:39:54,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:54,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88298f41e4fb6a3785bed HTTP/1.1" 200 None 2026-03-29 01:39:54,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:54,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:39:54,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:54,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88298f41e4fb6a3785bed HTTP/1.1" 200 None 2026-03-29 01:39:54,479 - lava_callback - INFO - Completed processing callback for node 69c88298f41e4fb6a3785bed 2026-03-29 01:39:55,555 - lava_callback - INFO - Received callback for node 69c88290f41e4fb6a3785bdb INFO: 172.20.0.1:52246 - "POST /node/69c88290f41e4fb6a3785bdb HTTP/1.0" 202 Accepted 2026-03-29 01:39:55,562 - lava_callback - INFO - Starting async job submit for node 69c88290f41e4fb6a3785bdb 2026-03-29 01:39:55,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:55,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88290f41e4fb6a3785bdb HTTP/1.1" 200 None 2026-03-29 01:39:55,598 - lava_callback - DEBUG - Processing node 69c88290f41e4fb6a3785bdb: baseline-arm64 2026-03-29 01:39:55,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:56,069 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:56,071 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88290f41e4fb6a3785bdb/log.txt.gz 2026-03-29 01:39:56,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:39:56,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:56,495 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88290f41e4fb6a3785bdb/lava-logs.yaml 2026-03-29 01:39:56,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:52256 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:39:56,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:39:56,649 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88290f41e4fb6a3785bdb/lava_callback.json.gz 2026-03-29 01:39:56,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:56,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88290f41e4fb6a3785bdb HTTP/1.1" 200 None 2026-03-29 01:39:56,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:56,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:39:56,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:39:56,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88290f41e4fb6a3785bdb HTTP/1.1" 200 None 2026-03-29 01:39:56,742 - lava_callback - INFO - Completed processing callback for node 69c88290f41e4fb6a3785bdb INFO: 172.20.0.1:52270 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:40:04,494 - lava_callback - INFO - Received callback for node 69c8828ef41e4fb6a3785bd8 INFO: 172.20.0.1:55078 - "POST /node/69c8828ef41e4fb6a3785bd8 HTTP/1.0" 202 Accepted 2026-03-29 01:40:04,500 - lava_callback - INFO - Starting async job submit for node 69c8828ef41e4fb6a3785bd8 2026-03-29 01:40:04,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:04,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8828ef41e4fb6a3785bd8 HTTP/1.1" 200 None 2026-03-29 01:40:04,536 - lava_callback - DEBUG - Processing node 69c8828ef41e4fb6a3785bd8: baseline-arm64 2026-03-29 01:40:04,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:04,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:04,910 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828ef41e4fb6a3785bd8/log.txt.gz 2026-03-29 01:40:05,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:05,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:05,247 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828ef41e4fb6a3785bd8/lava-logs.yaml 2026-03-29 01:40:05,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:05,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:05,386 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828ef41e4fb6a3785bd8/lava_callback.json.gz 2026-03-29 01:40:05,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:05,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8828ef41e4fb6a3785bd8 HTTP/1.1" 200 None 2026-03-29 01:40:05,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:05,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:05,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:05,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8828ef41e4fb6a3785bd8 HTTP/1.1" 200 None 2026-03-29 01:40:05,475 - lava_callback - INFO - Completed processing callback for node 69c8828ef41e4fb6a3785bd8 INFO: 172.20.0.1:55090 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:40:08,786 - lava_callback - INFO - Received callback for node 69c8829af41e4fb6a3785bf3 INFO: 172.20.0.1:55106 - "POST /node/69c8829af41e4fb6a3785bf3 HTTP/1.0" 202 Accepted 2026-03-29 01:40:08,794 - lava_callback - INFO - Starting async job submit for node 69c8829af41e4fb6a3785bf3 2026-03-29 01:40:08,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:08,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8829af41e4fb6a3785bf3 HTTP/1.1" 200 None 2026-03-29 01:40:08,836 - lava_callback - DEBUG - Processing node 69c8829af41e4fb6a3785bf3: baseline-arm64 2026-03-29 01:40:09,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:09,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:09,279 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8829af41e4fb6a3785bf3/log.txt.gz 2026-03-29 01:40:09,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:09,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:09,701 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8829af41e4fb6a3785bf3/lava-logs.yaml 2026-03-29 01:40:09,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:09,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:09,846 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8829af41e4fb6a3785bf3/lava_callback.json.gz 2026-03-29 01:40:09,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:09,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8829af41e4fb6a3785bf3 HTTP/1.1" 200 None 2026-03-29 01:40:09,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:09,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:09,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:09,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8829af41e4fb6a3785bf3 HTTP/1.1" 200 None 2026-03-29 01:40:09,943 - lava_callback - INFO - Completed processing callback for node 69c8829af41e4fb6a3785bf3 2026-03-29 01:40:11,507 - lava_callback - INFO - Received callback for node 69c88287f41e4fb6a3785bb8 INFO: 172.20.0.1:37580 - "POST /node/69c88287f41e4fb6a3785bb8 HTTP/1.0" 202 Accepted 2026-03-29 01:40:11,513 - lava_callback - INFO - Starting async job submit for node 69c88287f41e4fb6a3785bb8 2026-03-29 01:40:11,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:11,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88287f41e4fb6a3785bb8 HTTP/1.1" 200 None 2026-03-29 01:40:11,549 - lava_callback - DEBUG - Processing node 69c88287f41e4fb6a3785bb8: baseline-arm64 2026-03-29 01:40:11,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:11,801 - lava_callback - INFO - Received callback for node 69c882a2f41e4fb6a3785c05 INFO: 172.20.0.1:37584 - "POST /node/69c882a2f41e4fb6a3785c05 HTTP/1.0" 202 Accepted 2026-03-29 01:40:11,808 - lava_callback - INFO - Starting async job submit for node 69c882a2f41e4fb6a3785c05 2026-03-29 01:40:11,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:11,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a2f41e4fb6a3785c05 HTTP/1.1" 200 None 2026-03-29 01:40:11,858 - lava_callback - DEBUG - Processing node 69c882a2f41e4fb6a3785c05: baseline-nfs-arm64 2026-03-29 01:40:11,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:11,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88287f41e4fb6a3785bb8/log.txt.gz 2026-03-29 01:40:12,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:12,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:12,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:12,598 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88287f41e4fb6a3785bb8/lava-logs.yaml 2026-03-29 01:40:12,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:12,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:12,621 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a2f41e4fb6a3785c05/log.txt.gz 2026-03-29 01:40:12,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:12,802 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88287f41e4fb6a3785bb8/lava_callback.json.gz 2026-03-29 01:40:12,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:12,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88287f41e4fb6a3785bb8 HTTP/1.1" 200 None 2026-03-29 01:40:12,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:12,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:12,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:12,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:12,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88287f41e4fb6a3785bb8 HTTP/1.1" 200 None 2026-03-29 01:40:12,995 - lava_callback - INFO - Completed processing callback for node 69c88287f41e4fb6a3785bb8 2026-03-29 01:40:13,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:13,189 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a2f41e4fb6a3785c05/lava-logs.yaml 2026-03-29 01:40:13,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:13,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:13,371 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a2f41e4fb6a3785c05/lava_callback.json.gz 2026-03-29 01:40:13,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:13,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a2f41e4fb6a3785c05 HTTP/1.1" 200 None 2026-03-29 01:40:13,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:13,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:13,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:13,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882a2f41e4fb6a3785c05 HTTP/1.1" 200 None 2026-03-29 01:40:13,459 - lava_callback - INFO - Completed processing callback for node 69c882a2f41e4fb6a3785c05 2026-03-29 01:40:13,766 - lava_callback - INFO - Received callback for node 69c88292f41e4fb6a3785be1 INFO: 172.20.0.1:37586 - "POST /node/69c88292f41e4fb6a3785be1 HTTP/1.0" 202 Accepted 2026-03-29 01:40:13,773 - lava_callback - INFO - Starting async job submit for node 69c88292f41e4fb6a3785be1 2026-03-29 01:40:13,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:13,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88292f41e4fb6a3785be1 HTTP/1.1" 200 None 2026-03-29 01:40:13,811 - lava_callback - DEBUG - Processing node 69c88292f41e4fb6a3785be1: baseline-arm64 2026-03-29 01:40:14,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:14,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:14,426 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88292f41e4fb6a3785be1/log.txt.gz 2026-03-29 01:40:14,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:15,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:15,010 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88292f41e4fb6a3785be1/lava-logs.yaml 2026-03-29 01:40:15,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:15,113 - lava_callback - INFO - Received callback for node 69c882a3f41e4fb6a3785c08 INFO: 172.20.0.1:37600 - "POST /node/69c882a3f41e4fb6a3785c08 HTTP/1.0" 202 Accepted 2026-03-29 01:40:15,121 - lava_callback - INFO - Starting async job submit for node 69c882a3f41e4fb6a3785c08 2026-03-29 01:40:15,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:15,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a3f41e4fb6a3785c08 HTTP/1.1" 200 None 2026-03-29 01:40:15,169 - lava_callback - DEBUG - Processing node 69c882a3f41e4fb6a3785c08: baseline-nfs-arm64 INFO: 172.20.0.1:37614 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 01:40:15,210 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:15,257 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88292f41e4fb6a3785be1/lava_callback.json.gz 2026-03-29 01:40:15,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:15,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88292f41e4fb6a3785be1 HTTP/1.1" 200 None 2026-03-29 01:40:15,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:15,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:15,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:15,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88292f41e4fb6a3785be1 HTTP/1.1" 200 None 2026-03-29 01:40:15,804 - lava_callback - INFO - Completed processing callback for node 69c88292f41e4fb6a3785be1 2026-03-29 01:40:16,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:16,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:16,194 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a3f41e4fb6a3785c08/log.txt.gz INFO: 172.20.0.1:37628 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:40:16,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:16,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:16,793 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a3f41e4fb6a3785c08/lava-logs.yaml 2026-03-29 01:40:16,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:16,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:16,994 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a3f41e4fb6a3785c08/lava_callback.json.gz 2026-03-29 01:40:17,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:17,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a3f41e4fb6a3785c08 HTTP/1.1" 200 None 2026-03-29 01:40:17,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:17,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:17,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:17,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882a3f41e4fb6a3785c08 HTTP/1.1" 200 None 2026-03-29 01:40:17,103 - lava_callback - INFO - Completed processing callback for node 69c882a3f41e4fb6a3785c08 2026-03-29 01:40:17,891 - lava_callback - INFO - Received callback for node 69c882ccf41e4fb6a3785ca6 INFO: 172.20.0.1:37638 - "POST /node/69c882ccf41e4fb6a3785ca6 HTTP/1.0" 202 Accepted 2026-03-29 01:40:17,898 - lava_callback - INFO - Starting async job submit for node 69c882ccf41e4fb6a3785ca6 2026-03-29 01:40:17,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:17,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882ccf41e4fb6a3785ca6 HTTP/1.1" 200 None 2026-03-29 01:40:17,944 - lava_callback - DEBUG - Processing node 69c882ccf41e4fb6a3785ca6: baseline-arm64-foundriesio 2026-03-29 01:40:18,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:18,297 - lava_callback - INFO - Received callback for node 69c882caf41e4fb6a3785ca0 INFO: 172.20.0.1:37654 - "POST /node/69c882caf41e4fb6a3785ca0 HTTP/1.0" 202 Accepted 2026-03-29 01:40:18,306 - lava_callback - INFO - Starting async job submit for node 69c882caf41e4fb6a3785ca0 2026-03-29 01:40:18,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:18,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882caf41e4fb6a3785ca0 HTTP/1.1" 200 None 2026-03-29 01:40:18,363 - lava_callback - DEBUG - Processing node 69c882caf41e4fb6a3785ca0: baseline-arm64-pengutronix 2026-03-29 01:40:18,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:18,482 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c882ccf41e4fb6a3785ca6/log.txt.gz 2026-03-29 01:40:18,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:18,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:18,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:18,985 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c882caf41e4fb6a3785ca0/log.txt.gz 2026-03-29 01:40:18,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:19,013 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c882ccf41e4fb6a3785ca6/lava-logs.yaml 2026-03-29 01:40:19,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:19,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:19,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:19,262 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c882ccf41e4fb6a3785ca6/lava_callback.json.gz 2026-03-29 01:40:19,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:19,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882ccf41e4fb6a3785ca6 HTTP/1.1" 200 None 2026-03-29 01:40:19,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:19,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:19,308 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c882caf41e4fb6a3785ca0/lava-logs.yaml 2026-03-29 01:40:19,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:19,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:19,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:19,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882ccf41e4fb6a3785ca6 HTTP/1.1" 200 None 2026-03-29 01:40:19,375 - lava_callback - INFO - Completed processing callback for node 69c882ccf41e4fb6a3785ca6 2026-03-29 01:40:19,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:19,533 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c882caf41e4fb6a3785ca0/lava_callback.json.gz 2026-03-29 01:40:19,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:19,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882caf41e4fb6a3785ca0 HTTP/1.1" 200 None 2026-03-29 01:40:19,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:19,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:19,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:19,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882caf41e4fb6a3785ca0 HTTP/1.1" 200 None 2026-03-29 01:40:19,647 - lava_callback - INFO - Completed processing callback for node 69c882caf41e4fb6a3785ca0 2026-03-29 01:40:20,823 - lava_callback - INFO - Received callback for node 69c882a8f41e4fb6a3785c14 INFO: 172.20.0.1:37416 - "POST /node/69c882a8f41e4fb6a3785c14 HTTP/1.0" 202 Accepted 2026-03-29 01:40:20,829 - lava_callback - INFO - Starting async job submit for node 69c882a8f41e4fb6a3785c14 2026-03-29 01:40:20,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:20,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:20,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a8f41e4fb6a3785c14 HTTP/1.1" 200 None 2026-03-29 01:40:20,875 - lava_callback - DEBUG - Processing node 69c882a8f41e4fb6a3785c14: baseline-nfs-arm64 2026-03-29 01:40:20,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-29 01:40:21,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:21,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:21,394 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a8f41e4fb6a3785c14/log.txt.gz 2026-03-29 01:40:21,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:21,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:21,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a8f41e4fb6a3785c14/lava-logs.yaml 2026-03-29 01:40:21,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:21,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:21,971 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a8f41e4fb6a3785c14/lava_callback.json.gz 2026-03-29 01:40:21,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:22,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a8f41e4fb6a3785c14 HTTP/1.1" 200 None 2026-03-29 01:40:22,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:22,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:22,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:22,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882a8f41e4fb6a3785c14 HTTP/1.1" 200 None 2026-03-29 01:40:22,057 - lava_callback - INFO - Completed processing callback for node 69c882a8f41e4fb6a3785c14 2026-03-29 01:40:22,574 - lava_callback - INFO - Received callback for node 69c882a9f41e4fb6a3785c32 INFO: 172.20.0.1:37424 - "POST /node/69c882a9f41e4fb6a3785c32 HTTP/1.0" 202 Accepted 2026-03-29 01:40:22,581 - lava_callback - INFO - Starting async job submit for node 69c882a9f41e4fb6a3785c32 2026-03-29 01:40:22,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:22,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a9f41e4fb6a3785c32 HTTP/1.1" 200 None 2026-03-29 01:40:22,633 - lava_callback - DEBUG - Processing node 69c882a9f41e4fb6a3785c32: baseline-nfs-arm64 2026-03-29 01:40:23,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:23,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:23,249 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a9f41e4fb6a3785c32/log.txt.gz 2026-03-29 01:40:23,344 - lava_callback - INFO - Received callback for node 69c8829ff41e4fb6a3785bff INFO: 172.20.0.1:37438 - "POST /node/69c8829ff41e4fb6a3785bff HTTP/1.0" 202 Accepted 2026-03-29 01:40:23,367 - lava_callback - INFO - Starting async job submit for node 69c8829ff41e4fb6a3785bff 2026-03-29 01:40:23,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:23,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8829ff41e4fb6a3785bff HTTP/1.1" 200 None 2026-03-29 01:40:23,481 - lava_callback - DEBUG - Processing node 69c8829ff41e4fb6a3785bff: baseline-nfs-arm64 2026-03-29 01:40:23,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:23,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:23,849 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a9f41e4fb6a3785c32/lava-logs.yaml 2026-03-29 01:40:23,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:23,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:24,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:24,028 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8829ff41e4fb6a3785bff/log.txt.gz 2026-03-29 01:40:24,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:24,088 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a9f41e4fb6a3785c32/lava_callback.json.gz 2026-03-29 01:40:24,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:24,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a9f41e4fb6a3785c32 HTTP/1.1" 200 None 2026-03-29 01:40:24,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:24,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:24,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:24,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:24,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882a9f41e4fb6a3785c32 HTTP/1.1" 200 None 2026-03-29 01:40:24,298 - lava_callback - INFO - Completed processing callback for node 69c882a9f41e4fb6a3785c32 2026-03-29 01:40:24,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:24,528 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8829ff41e4fb6a3785bff/lava-logs.yaml 2026-03-29 01:40:24,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:24,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:24,676 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8829ff41e4fb6a3785bff/lava_callback.json.gz 2026-03-29 01:40:24,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:24,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8829ff41e4fb6a3785bff HTTP/1.1" 200 None 2026-03-29 01:40:24,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:24,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:24,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:24,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8829ff41e4fb6a3785bff HTTP/1.1" 200 None 2026-03-29 01:40:24,764 - lava_callback - INFO - Completed processing callback for node 69c8829ff41e4fb6a3785bff 2026-03-29 01:40:24,802 - lava_callback - INFO - Received callback for node 69c882c8f41e4fb6a3785c99 INFO: 172.20.0.1:37446 - "POST /node/69c882c8f41e4fb6a3785c99 HTTP/1.0" 202 Accepted 2026-03-29 01:40:24,812 - lava_callback - INFO - Starting async job submit for node 69c882c8f41e4fb6a3785c99 2026-03-29 01:40:24,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:24,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c8f41e4fb6a3785c99 HTTP/1.1" 200 None 2026-03-29 01:40:24,863 - lava_callback - DEBUG - Processing node 69c882c8f41e4fb6a3785c99: baseline-arm64-pengutronix 2026-03-29 01:40:25,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:25,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:25,187 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c882c8f41e4fb6a3785c99/log.txt.gz 2026-03-29 01:40:25,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:25,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:25,477 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c882c8f41e4fb6a3785c99/lava-logs.yaml 2026-03-29 01:40:25,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:25,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:25,644 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c882c8f41e4fb6a3785c99/lava_callback.json.gz 2026-03-29 01:40:25,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:25,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c8f41e4fb6a3785c99 HTTP/1.1" 200 None 2026-03-29 01:40:25,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:25,683 - lava_callback - INFO - Received callback for node 69c882b2f41e4fb6a3785c41 INFO: 172.20.0.1:37460 - "POST /node/69c882b2f41e4fb6a3785c41 HTTP/1.0" 202 Accepted 2026-03-29 01:40:25,690 - lava_callback - INFO - Starting async job submit for node 69c882b2f41e4fb6a3785c41 2026-03-29 01:40:25,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:25,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:25,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:25,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b2f41e4fb6a3785c41 HTTP/1.1" 200 None 2026-03-29 01:40:25,741 - lava_callback - DEBUG - Processing node 69c882b2f41e4fb6a3785c41: baseline-arm64-broonie 2026-03-29 01:40:25,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882c8f41e4fb6a3785c99 HTTP/1.1" 200 None 2026-03-29 01:40:25,764 - lava_callback - INFO - Completed processing callback for node 69c882c8f41e4fb6a3785c99 2026-03-29 01:40:26,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:26,189 - lava_callback - INFO - Received callback for node 69c882cbf41e4fb6a3785ca3 INFO: 172.20.0.1:37466 - "POST /node/69c882cbf41e4fb6a3785ca3 HTTP/1.0" 202 Accepted 2026-03-29 01:40:26,197 - lava_callback - INFO - Starting async job submit for node 69c882cbf41e4fb6a3785ca3 2026-03-29 01:40:26,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:26,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:26,223 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b2f41e4fb6a3785c41/log.txt.gz 2026-03-29 01:40:26,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882cbf41e4fb6a3785ca3 HTTP/1.1" 200 None 2026-03-29 01:40:26,309 - lava_callback - DEBUG - Processing node 69c882cbf41e4fb6a3785ca3: baseline-arm64-pengutronix 2026-03-29 01:40:26,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:37478 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:40:26,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:26,695 - lava_callback - INFO - Received callback for node 69c882aaf41e4fb6a3785c35 INFO: 172.20.0.1:37486 - "POST /node/69c882aaf41e4fb6a3785c35 HTTP/1.0" 202 Accepted 2026-03-29 01:40:26,705 - lava_callback - INFO - Starting async job submit for node 69c882aaf41e4fb6a3785c35 2026-03-29 01:40:26,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:26,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:26,736 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c882cbf41e4fb6a3785ca3/log.txt.gz 2026-03-29 01:40:26,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882aaf41e4fb6a3785c35 HTTP/1.1" 200 None 2026-03-29 01:40:26,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:26,798 - lava_callback - DEBUG - Processing node 69c882aaf41e4fb6a3785c35: baseline-nfs-arm64 2026-03-29 01:40:26,992 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b2f41e4fb6a3785c41/lava-logs.yaml 2026-03-29 01:40:27,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:27,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:27,134 - lava_callback - INFO - Received callback for node 69c882baf41e4fb6a3785c50 2026-03-29 01:40:27,197 - lava_callback - INFO - Starting async job submit for node 69c882baf41e4fb6a3785c50 INFO: 172.20.0.1:37502 - "POST /node/69c882baf41e4fb6a3785c50 HTTP/1.0" 202 Accepted 2026-03-29 01:40:27,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:27,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882baf41e4fb6a3785c50 HTTP/1.1" 200 None 2026-03-29 01:40:27,392 - lava_callback - DEBUG - Processing node 69c882baf41e4fb6a3785c50: baseline-arm64-broonie 2026-03-29 01:40:27,426 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:27,443 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c882cbf41e4fb6a3785ca3/lava-logs.yaml 2026-03-29 01:40:27,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:27,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:27,806 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b2f41e4fb6a3785c41/lava_callback.json.gz 2026-03-29 01:40:27,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:27,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:27,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:27,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b2f41e4fb6a3785c41 HTTP/1.1" 200 None 2026-03-29 01:40:27,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:27,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:27,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:28,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:28,006 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c882cbf41e4fb6a3785ca3/lava_callback.json.gz 2026-03-29 01:40:28,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882b2f41e4fb6a3785c41 HTTP/1.1" 200 None 2026-03-29 01:40:28,055 - lava_callback - INFO - Completed processing callback for node 69c882b2f41e4fb6a3785c41 2026-03-29 01:40:28,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:28,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882cbf41e4fb6a3785ca3 HTTP/1.1" 200 None 2026-03-29 01:40:28,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:28,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:28,148 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882aaf41e4fb6a3785c35/log.txt.gz 2026-03-29 01:40:28,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:28,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:28,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:28,223 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882baf41e4fb6a3785c50/log.txt.gz 2026-03-29 01:40:28,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882cbf41e4fb6a3785ca3 HTTP/1.1" 200 None 2026-03-29 01:40:28,270 - lava_callback - INFO - Completed processing callback for node 69c882cbf41e4fb6a3785ca3 2026-03-29 01:40:28,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:28,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:28,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:28,829 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882baf41e4fb6a3785c50/lava-logs.yaml 2026-03-29 01:40:28,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:28,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:28,891 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882aaf41e4fb6a3785c35/lava-logs.yaml 2026-03-29 01:40:28,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:28,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:28,989 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882baf41e4fb6a3785c50/lava_callback.json.gz 2026-03-29 01:40:29,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:29,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882baf41e4fb6a3785c50 HTTP/1.1" 200 None 2026-03-29 01:40:29,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:29,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:29,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:29,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:29,088 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882aaf41e4fb6a3785c35/lava_callback.json.gz 2026-03-29 01:40:29,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882baf41e4fb6a3785c50 HTTP/1.1" 200 None 2026-03-29 01:40:29,137 - lava_callback - INFO - Completed processing callback for node 69c882baf41e4fb6a3785c50 2026-03-29 01:40:29,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:29,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882aaf41e4fb6a3785c35 HTTP/1.1" 200 None 2026-03-29 01:40:29,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:29,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:29,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:29,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882aaf41e4fb6a3785c35 HTTP/1.1" 200 None 2026-03-29 01:40:29,245 - lava_callback - INFO - Completed processing callback for node 69c882aaf41e4fb6a3785c35 INFO: 172.20.0.1:48216 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:40:45,385 - lava_callback - INFO - Received callback for node 69c882cdf41e4fb6a3785cad INFO: 172.20.0.1:48898 - "POST /node/69c882cdf41e4fb6a3785cad HTTP/1.0" 202 Accepted 2026-03-29 01:40:45,392 - lava_callback - INFO - Starting async job submit for node 69c882cdf41e4fb6a3785cad 2026-03-29 01:40:45,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:45,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882cdf41e4fb6a3785cad HTTP/1.1" 200 None 2026-03-29 01:40:45,436 - lava_callback - DEBUG - Processing node 69c882cdf41e4fb6a3785cad: baseline-arm64-foundriesio 2026-03-29 01:40:45,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:45,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:45,993 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c882cdf41e4fb6a3785cad/log.txt.gz 2026-03-29 01:40:46,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:46,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:46,437 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c882cdf41e4fb6a3785cad/lava-logs.yaml 2026-03-29 01:40:46,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:48908 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:40:46,778 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:46,780 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c882cdf41e4fb6a3785cad/lava_callback.json.gz 2026-03-29 01:40:46,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:46,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882cdf41e4fb6a3785cad HTTP/1.1" 200 None 2026-03-29 01:40:46,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:46,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:46,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:46,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882cdf41e4fb6a3785cad HTTP/1.1" 200 None 2026-03-29 01:40:46,876 - lava_callback - INFO - Completed processing callback for node 69c882cdf41e4fb6a3785cad 2026-03-29 01:40:50,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:50,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-29 01:40:53,341 - lava_callback - INFO - Received callback for node 69c882b5f41e4fb6a3785c47 INFO: 172.20.0.1:55068 - "POST /node/69c882b5f41e4fb6a3785c47 HTTP/1.0" 202 Accepted 2026-03-29 01:40:53,348 - lava_callback - INFO - Starting async job submit for node 69c882b5f41e4fb6a3785c47 2026-03-29 01:40:53,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:53,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b5f41e4fb6a3785c47 HTTP/1.1" 200 None 2026-03-29 01:40:53,385 - lava_callback - DEBUG - Processing node 69c882b5f41e4fb6a3785c47: baseline-arm64-broonie 2026-03-29 01:40:53,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:53,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:53,783 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b5f41e4fb6a3785c47/log.txt.gz 2026-03-29 01:40:53,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:54,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:54,168 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b5f41e4fb6a3785c47/lava-logs.yaml 2026-03-29 01:40:54,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:54,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:54,312 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b5f41e4fb6a3785c47/lava_callback.json.gz 2026-03-29 01:40:54,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:54,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b5f41e4fb6a3785c47 HTTP/1.1" 200 None 2026-03-29 01:40:54,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:54,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:54,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:54,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882b5f41e4fb6a3785c47 HTTP/1.1" 200 None 2026-03-29 01:40:54,398 - lava_callback - INFO - Completed processing callback for node 69c882b5f41e4fb6a3785c47 2026-03-29 01:40:54,920 - lava_callback - INFO - Received callback for node 69c882b7f41e4fb6a3785c4a INFO: 172.20.0.1:55082 - "POST /node/69c882b7f41e4fb6a3785c4a HTTP/1.0" 202 Accepted 2026-03-29 01:40:54,930 - lava_callback - INFO - Starting async job submit for node 69c882b7f41e4fb6a3785c4a 2026-03-29 01:40:54,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:54,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b7f41e4fb6a3785c4a HTTP/1.1" 200 None 2026-03-29 01:40:54,977 - lava_callback - DEBUG - Processing node 69c882b7f41e4fb6a3785c4a: baseline-arm64-broonie 2026-03-29 01:40:55,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:55,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:55,395 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b7f41e4fb6a3785c4a/log.txt.gz 2026-03-29 01:40:55,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:55,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:55,762 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b7f41e4fb6a3785c4a/lava-logs.yaml 2026-03-29 01:40:55,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:55,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:55,928 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b7f41e4fb6a3785c4a/lava_callback.json.gz 2026-03-29 01:40:55,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:55,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b7f41e4fb6a3785c4a HTTP/1.1" 200 None 2026-03-29 01:40:55,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:55,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:55,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:56,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882b7f41e4fb6a3785c4a HTTP/1.1" 200 None 2026-03-29 01:40:56,014 - lava_callback - INFO - Completed processing callback for node 69c882b7f41e4fb6a3785c4a 2026-03-29 01:40:56,500 - lava_callback - INFO - Received callback for node 69c882bbf41e4fb6a3785c57 INFO: 172.20.0.1:55094 - "POST /node/69c882bbf41e4fb6a3785c57 HTTP/1.0" 202 Accepted 2026-03-29 01:40:56,508 - lava_callback - INFO - Starting async job submit for node 69c882bbf41e4fb6a3785c57 2026-03-29 01:40:56,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:55108 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:40:56,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882bbf41e4fb6a3785c57 HTTP/1.1" 200 None 2026-03-29 01:40:56,559 - lava_callback - DEBUG - Processing node 69c882bbf41e4fb6a3785c57: baseline-arm64-broonie 2026-03-29 01:40:56,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:56,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:56,962 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882bbf41e4fb6a3785c57/log.txt.gz 2026-03-29 01:40:57,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:57,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:57,319 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882bbf41e4fb6a3785c57/lava-logs.yaml 2026-03-29 01:40:57,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:55120 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:40:57,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:57,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882bbf41e4fb6a3785c57/lava_callback.json.gz 2026-03-29 01:40:57,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:57,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882bbf41e4fb6a3785c57 HTTP/1.1" 200 None 2026-03-29 01:40:57,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:57,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:57,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:57,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882bbf41e4fb6a3785c57 HTTP/1.1" 200 None 2026-03-29 01:40:57,547 - lava_callback - INFO - Completed processing callback for node 69c882bbf41e4fb6a3785c57 2026-03-29 01:40:57,924 - lava_callback - INFO - Received callback for node 69c882bdf41e4fb6a3785c5a INFO: 172.20.0.1:55136 - "POST /node/69c882bdf41e4fb6a3785c5a HTTP/1.0" 202 Accepted 2026-03-29 01:40:57,931 - lava_callback - INFO - Starting async job submit for node 69c882bdf41e4fb6a3785c5a 2026-03-29 01:40:57,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:57,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882bdf41e4fb6a3785c5a HTTP/1.1" 200 None 2026-03-29 01:40:57,970 - lava_callback - DEBUG - Processing node 69c882bdf41e4fb6a3785c5a: baseline-arm64-broonie 2026-03-29 01:40:58,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:58,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:58,342 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882bdf41e4fb6a3785c5a/log.txt.gz 2026-03-29 01:40:58,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:58,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:58,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882bdf41e4fb6a3785c5a/lava-logs.yaml 2026-03-29 01:40:58,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:40:58,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:40:58,827 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882bdf41e4fb6a3785c5a/lava_callback.json.gz 2026-03-29 01:40:58,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:58,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882bdf41e4fb6a3785c5a HTTP/1.1" 200 None 2026-03-29 01:40:58,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:58,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:40:58,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:40:58,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882bdf41e4fb6a3785c5a HTTP/1.1" 200 None 2026-03-29 01:40:58,917 - lava_callback - INFO - Completed processing callback for node 69c882bdf41e4fb6a3785c5a 2026-03-29 01:41:06,179 - lava_callback - INFO - Received callback for node 69c5e0202cb62322ad968c7d INFO: 172.20.0.1:56366 - "POST /node/69c5e0202cb62322ad968c7d HTTP/1.0" 202 Accepted 2026-03-29 01:41:06,185 - lava_callback - INFO - Starting async job submit for node 69c5e0202cb62322ad968c7d 2026-03-29 01:41:06,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:06,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e0202cb62322ad968c7d HTTP/1.1" 200 None 2026-03-29 01:41:06,201 - lava_callback - DEBUG - Processing node 69c5e0202cb62322ad968c7d: baseline-arm64-clabbe 2026-03-29 01:41:06,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e0202cb62322ad968c7d 2026-03-29 01:41:06,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:06,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e0202cb62322ad968c7d HTTP/1.1" 200 None 2026-03-29 01:41:06,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:06,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:41:06,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:06,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e0202cb62322ad968c7d HTTP/1.1" 200 None 2026-03-29 01:41:06,253 - lava_callback - ERROR - Error processing callback for node 69c5e0202cb62322ad968c7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40334 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:41:20,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:20,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:49342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43094 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:41:43,232 - lava_callback - INFO - Received callback for node 69c8828df41e4fb6a3785bd5 INFO: 172.20.0.1:55952 - "POST /node/69c8828df41e4fb6a3785bd5 HTTP/1.0" 202 Accepted 2026-03-29 01:41:43,238 - lava_callback - INFO - Starting async job submit for node 69c8828df41e4fb6a3785bd5 2026-03-29 01:41:43,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:43,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8828df41e4fb6a3785bd5 HTTP/1.1" 200 None 2026-03-29 01:41:43,270 - lava_callback - DEBUG - Processing node 69c8828df41e4fb6a3785bd5: baseline-arm64 2026-03-29 01:41:43,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:41:43,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:41:43,776 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828df41e4fb6a3785bd5/log.txt.gz 2026-03-29 01:41:43,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:41:44,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:41:44,174 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828df41e4fb6a3785bd5/lava-logs.yaml 2026-03-29 01:41:44,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:41:44,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:41:44,340 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8828df41e4fb6a3785bd5/lava_callback.json.gz 2026-03-29 01:41:44,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:44,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8828df41e4fb6a3785bd5 HTTP/1.1" 200 None 2026-03-29 01:41:44,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:44,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:41:44,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:44,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8828df41e4fb6a3785bd5 HTTP/1.1" 200 None 2026-03-29 01:41:44,418 - lava_callback - INFO - Completed processing callback for node 69c8828df41e4fb6a3785bd5 2026-03-29 01:41:45,507 - lava_callback - INFO - Received callback for node 69c882b3f41e4fb6a3785c44 INFO: 172.20.0.1:55964 - "POST /node/69c882b3f41e4fb6a3785c44 HTTP/1.0" 202 Accepted 2026-03-29 01:41:45,513 - lava_callback - INFO - Starting async job submit for node 69c882b3f41e4fb6a3785c44 2026-03-29 01:41:45,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:45,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b3f41e4fb6a3785c44 HTTP/1.1" 200 None 2026-03-29 01:41:45,618 - lava_callback - DEBUG - Processing node 69c882b3f41e4fb6a3785c44: baseline-arm64-broonie 2026-03-29 01:41:45,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:41:45,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:41:45,987 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b3f41e4fb6a3785c44/log.txt.gz 2026-03-29 01:41:46,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:41:46,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:41:46,347 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b3f41e4fb6a3785c44/lava-logs.yaml 2026-03-29 01:41:46,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:41:46,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:41:46,525 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b3f41e4fb6a3785c44/lava_callback.json.gz INFO: 172.20.0.1:55968 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:41:46,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:46,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b3f41e4fb6a3785c44 HTTP/1.1" 200 None 2026-03-29 01:41:46,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:46,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:41:46,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:46,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882b3f41e4fb6a3785c44 HTTP/1.1" 200 None 2026-03-29 01:41:46,620 - lava_callback - INFO - Completed processing callback for node 69c882b3f41e4fb6a3785c44 2026-03-29 01:41:50,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:50,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35598 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:41:58,240 - lava_callback - INFO - Received callback for node 69c8834ff41e4fb6a3785d92 INFO: 172.20.0.1:35610 - "POST /node/69c8834ff41e4fb6a3785d92 HTTP/1.0" 202 Accepted 2026-03-29 01:41:58,246 - lava_callback - INFO - Starting async job submit for node 69c8834ff41e4fb6a3785d92 2026-03-29 01:41:58,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:58,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8834ff41e4fb6a3785d92 HTTP/1.1" 200 None 2026-03-29 01:41:58,262 - lava_callback - DEBUG - Processing node 69c8834ff41e4fb6a3785d92: baseline-nfs-arm64 2026-03-29 01:41:58,264 - lava_callback - WARNING - Failed to upload log 2026-03-29 01:41:58,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:41:58,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:41:58,381 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8834ff41e4fb6a3785d92/lava-logs.yaml 2026-03-29 01:41:58,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:41:58,491 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:41:58,492 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8834ff41e4fb6a3785d92/lava_callback.json.gz 2026-03-29 01:41:58,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:58,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8834ff41e4fb6a3785d92 HTTP/1.1" 200 None 2026-03-29 01:41:58,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:58,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:41:58,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:41:58,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8834ff41e4fb6a3785d92 HTTP/1.1" 200 None 2026-03-29 01:41:58,557 - lava_callback - INFO - Completed processing callback for node 69c8834ff41e4fb6a3785d92 2026-03-29 01:42:05,832 - lava_callback - INFO - Received callback for node 69c882bef41e4fb6a3785c5d INFO: 172.20.0.1:41820 - "POST /node/69c882bef41e4fb6a3785c5d HTTP/1.0" 202 Accepted 2026-03-29 01:42:05,838 - lava_callback - INFO - Starting async job submit for node 69c882bef41e4fb6a3785c5d 2026-03-29 01:42:05,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:05,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882bef41e4fb6a3785c5d HTTP/1.1" 200 None 2026-03-29 01:42:05,905 - lava_callback - INFO - Received callback for node 69c88294f41e4fb6a3785be4 2026-03-29 01:42:05,905 - lava_callback - DEBUG - Processing node 69c882bef41e4fb6a3785c5d: baseline-arm64-broonie INFO: 172.20.0.1:41822 - "POST /node/69c88294f41e4fb6a3785be4 HTTP/1.0" 202 Accepted 2026-03-29 01:42:05,927 - lava_callback - INFO - Starting async job submit for node 69c88294f41e4fb6a3785be4 2026-03-29 01:42:05,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:06,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88294f41e4fb6a3785be4 HTTP/1.1" 200 None 2026-03-29 01:42:06,058 - lava_callback - DEBUG - Processing node 69c88294f41e4fb6a3785be4: baseline-arm64 2026-03-29 01:42:06,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:42:06,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:41832 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:42:06,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:42:06,565 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88294f41e4fb6a3785be4/log.txt.gz 2026-03-29 01:42:06,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:42:06,640 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882bef41e4fb6a3785c5d/log.txt.gz 2026-03-29 01:42:06,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:42:06,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:42:06,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:42:06,914 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88294f41e4fb6a3785be4/lava-logs.yaml 2026-03-29 01:42:06,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:42:07,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:42:07,065 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88294f41e4fb6a3785be4/lava_callback.json.gz 2026-03-29 01:42:07,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:42:07,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:07,086 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882bef41e4fb6a3785c5d/lava-logs.yaml 2026-03-29 01:42:07,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:42:07,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88294f41e4fb6a3785be4 HTTP/1.1" 200 None 2026-03-29 01:42:07,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:07,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:42:07,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:07,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88294f41e4fb6a3785be4 HTTP/1.1" 200 None 2026-03-29 01:42:07,167 - lava_callback - INFO - Completed processing callback for node 69c88294f41e4fb6a3785be4 2026-03-29 01:42:07,305 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:42:07,307 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882bef41e4fb6a3785c5d/lava_callback.json.gz 2026-03-29 01:42:07,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:07,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882bef41e4fb6a3785c5d HTTP/1.1" 200 None 2026-03-29 01:42:07,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:07,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:42:07,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:07,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882bef41e4fb6a3785c5d HTTP/1.1" 200 None 2026-03-29 01:42:07,427 - lava_callback - INFO - Completed processing callback for node 69c882bef41e4fb6a3785c5d INFO: 172.20.0.1:34056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:42:20,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:20,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 01:42:26,086 - lava_callback - INFO - Received callback for node 69c5e0762cb62322ad968c9e INFO: 172.20.0.1:51562 - "POST /node/69c5e0762cb62322ad968c9e HTTP/1.0" 202 Accepted 2026-03-29 01:42:26,091 - lava_callback - INFO - Starting async job submit for node 69c5e0762cb62322ad968c9e 2026-03-29 01:42:26,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:26,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e0762cb62322ad968c9e HTTP/1.1" 200 None 2026-03-29 01:42:26,108 - lava_callback - DEBUG - Processing node 69c5e0762cb62322ad968c9e: baseline-arm64-clabbe 2026-03-29 01:42:26,108 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e0762cb62322ad968c9e 2026-03-29 01:42:26,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:26,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e0762cb62322ad968c9e HTTP/1.1" 200 None 2026-03-29 01:42:26,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:26,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:42:26,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:26,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e0762cb62322ad968c9e HTTP/1.1" 200 None 2026-03-29 01:42:26,212 - lava_callback - ERROR - Error processing callback for node 69c5e0762cb62322ad968c9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60604 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:42:45,442 - lava_callback - INFO - Received callback for node 69c882b8f41e4fb6a3785c4d INFO: 172.20.0.1:48286 - "POST /node/69c882b8f41e4fb6a3785c4d HTTP/1.0" 202 Accepted 2026-03-29 01:42:45,448 - lava_callback - INFO - Starting async job submit for node 69c882b8f41e4fb6a3785c4d 2026-03-29 01:42:45,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:45,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b8f41e4fb6a3785c4d HTTP/1.1" 200 None 2026-03-29 01:42:45,485 - lava_callback - DEBUG - Processing node 69c882b8f41e4fb6a3785c4d: baseline-arm64-broonie 2026-03-29 01:42:45,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:42:45,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:42:45,910 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b8f41e4fb6a3785c4d/log.txt.gz 2026-03-29 01:42:46,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:42:46,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:42:46,286 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b8f41e4fb6a3785c4d/lava-logs.yaml 2026-03-29 01:42:46,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:42:46,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:42:46,472 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c882b8f41e4fb6a3785c4d/lava_callback.json.gz 2026-03-29 01:42:46,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:46,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882b8f41e4fb6a3785c4d HTTP/1.1" 200 None 2026-03-29 01:42:46,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:48290 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:42:46,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:42:46,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:46,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882b8f41e4fb6a3785c4d HTTP/1.1" 200 None 2026-03-29 01:42:46,561 - lava_callback - INFO - Completed processing callback for node 69c882b8f41e4fb6a3785c4d 2026-03-29 01:42:50,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:42:51,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36006 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:43:06,270 - lava_callback - INFO - Received callback for node 69c8829ef41e4fb6a3785bfc INFO: 172.20.0.1:34594 - "POST /node/69c8829ef41e4fb6a3785bfc HTTP/1.0" 202 Accepted 2026-03-29 01:43:06,276 - lava_callback - INFO - Starting async job submit for node 69c8829ef41e4fb6a3785bfc 2026-03-29 01:43:06,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:43:06,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8829ef41e4fb6a3785bfc HTTP/1.1" 200 None 2026-03-29 01:43:06,308 - lava_callback - DEBUG - Processing node 69c8829ef41e4fb6a3785bfc: baseline-nfs-arm64 INFO: 172.20.0.1:34600 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:43:06,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:43:06,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:43:06,869 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8829ef41e4fb6a3785bfc/log.txt.gz 2026-03-29 01:43:07,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:43:07,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:43:07,422 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8829ef41e4fb6a3785bfc/lava-logs.yaml 2026-03-29 01:43:07,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:43:07,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:43:07,608 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8829ef41e4fb6a3785bfc/lava_callback.json.gz 2026-03-29 01:43:07,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:43:07,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8829ef41e4fb6a3785bfc HTTP/1.1" 200 None 2026-03-29 01:43:07,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:43:07,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:43:07,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:43:07,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8829ef41e4fb6a3785bfc HTTP/1.1" 200 None 2026-03-29 01:43:07,693 - lava_callback - INFO - Completed processing callback for node 69c8829ef41e4fb6a3785bfc INFO: 172.20.0.1:53514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53522 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:43:21,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:43:21,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46730 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:43:46,255 - lava_callback - INFO - Received callback for node 69c5e0cc2cb62322ad968cb6 INFO: 172.20.0.1:49062 - "POST /node/69c5e0cc2cb62322ad968cb6 HTTP/1.0" 202 Accepted 2026-03-29 01:43:46,261 - lava_callback - INFO - Starting async job submit for node 69c5e0cc2cb62322ad968cb6 2026-03-29 01:43:46,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:43:46,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e0cc2cb62322ad968cb6 HTTP/1.1" 200 None 2026-03-29 01:43:46,276 - lava_callback - DEBUG - Processing node 69c5e0cc2cb62322ad968cb6: baseline-arm64-clabbe 2026-03-29 01:43:46,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e0cc2cb62322ad968cb6 2026-03-29 01:43:46,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:43:46,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e0cc2cb62322ad968cb6 HTTP/1.1" 200 None 2026-03-29 01:43:46,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:43:46,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 01:43:46,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:43:46,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e0cc2cb62322ad968cb6 HTTP/1.1" 200 None 2026-03-29 01:43:46,335 - lava_callback - ERROR - Error processing callback for node 69c5e0cc2cb62322ad968cb6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50184 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:44:11,727 - lava_callback - INFO - Received callback for node 69c8829cf41e4fb6a3785bf6 INFO: 172.20.0.1:41838 - "POST /node/69c8829cf41e4fb6a3785bf6 HTTP/1.0" 202 Accepted 2026-03-29 01:44:11,734 - lava_callback - INFO - Starting async job submit for node 69c8829cf41e4fb6a3785bf6 2026-03-29 01:44:11,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:44:11,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8829cf41e4fb6a3785bf6 HTTP/1.1" 200 None 2026-03-29 01:44:11,809 - lava_callback - DEBUG - Processing node 69c8829cf41e4fb6a3785bf6: baseline-arm64 2026-03-29 01:44:12,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:44:12,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:44:12,340 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8829cf41e4fb6a3785bf6/log.txt.gz 2026-03-29 01:44:12,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:44:12,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:44:12,801 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8829cf41e4fb6a3785bf6/lava-logs.yaml 2026-03-29 01:44:12,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:44:12,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:44:12,994 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8829cf41e4fb6a3785bf6/lava_callback.json.gz DEBUG: Setup failed for 69c8829cf41e4fb6a3785bf6. Transitting job node result: incomplete -> fail 2026-03-29 01:44:13,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:44:13,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8829cf41e4fb6a3785bf6 HTTP/1.1" 200 None 2026-03-29 01:44:13,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:44:13,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:44:13,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:44:13,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8829cf41e4fb6a3785bf6 HTTP/1.1" 200 None 2026-03-29 01:44:13,224 - lava_callback - INFO - Completed processing callback for node 69c8829cf41e4fb6a3785bf6 INFO: 172.20.0.1:41846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41854 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:44:21,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:44:21,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55638 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:45:10,206 - lava_callback - INFO - Received callback for node 69c883e2f41e4fb6a3785ddc INFO: 172.20.0.1:43632 - "POST /node/69c883e2f41e4fb6a3785ddc HTTP/1.0" 202 Accepted 2026-03-29 01:45:10,213 - lava_callback - INFO - Starting async job submit for node 69c883e2f41e4fb6a3785ddc 2026-03-29 01:45:10,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:45:10,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c883e2f41e4fb6a3785ddc HTTP/1.1" 200 None 2026-03-29 01:45:10,258 - lava_callback - DEBUG - Processing node 69c883e2f41e4fb6a3785ddc: baseline-arm64 2026-03-29 01:45:10,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:45:10,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:45:10,583 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c883e2f41e4fb6a3785ddc/log.txt.gz 2026-03-29 01:45:10,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:45:10,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:45:10,831 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c883e2f41e4fb6a3785ddc/lava-logs.yaml 2026-03-29 01:45:10,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:45:10,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:45:10,973 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c883e2f41e4fb6a3785ddc/lava_callback.json.gz 2026-03-29 01:45:11,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:45:11,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c883e2f41e4fb6a3785ddc HTTP/1.1" 200 None 2026-03-29 01:45:11,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:45:11,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:45:11,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:45:11,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c883e2f41e4fb6a3785ddc HTTP/1.1" 200 None 2026-03-29 01:45:11,142 - lava_callback - INFO - Completed processing callback for node 69c883e2f41e4fb6a3785ddc INFO: 172.20.0.1:43642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43652 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:45:21,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:45:21,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56372 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:46:25,324 - lava_callback - INFO - Received callback for node 69c8842cf41e4fb6a3785df1 INFO: 172.20.0.1:37306 - "POST /node/69c8842cf41e4fb6a3785df1 HTTP/1.0" 202 Accepted 2026-03-29 01:46:25,331 - lava_callback - INFO - Starting async job submit for node 69c8842cf41e4fb6a3785df1 2026-03-29 01:46:25,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:46:25,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8842cf41e4fb6a3785df1 HTTP/1.1" 200 None 2026-03-29 01:46:25,385 - lava_callback - DEBUG - Processing node 69c8842cf41e4fb6a3785df1: baseline-arm64 2026-03-29 01:46:25,519 - lava_callback - WARNING - Failed to upload log 2026-03-29 01:46:25,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:46:25,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:46:25,794 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8842cf41e4fb6a3785df1/lava-logs.yaml 2026-03-29 01:46:25,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:46:25,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:46:25,939 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8842cf41e4fb6a3785df1/lava_callback.json.gz 2026-03-29 01:46:25,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:46:25,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8842cf41e4fb6a3785df1 HTTP/1.1" 200 None 2026-03-29 01:46:25,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:46:26,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:46:26,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:46:26,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8842cf41e4fb6a3785df1 HTTP/1.1" 200 None 2026-03-29 01:46:26,108 - lava_callback - INFO - Completed processing callback for node 69c8842cf41e4fb6a3785df1 INFO: 172.20.0.1:37320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34816 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:46:51,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:46:51,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55952 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:47:58,033 - lava_callback - INFO - Received callback for node 69c8847af41e4fb6a3785e07 INFO: 172.20.0.1:55960 - "POST /node/69c8847af41e4fb6a3785e07 HTTP/1.0" 202 Accepted 2026-03-29 01:47:58,039 - lava_callback - INFO - Starting async job submit for node 69c8847af41e4fb6a3785e07 2026-03-29 01:47:58,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:47:58,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8847af41e4fb6a3785e07 HTTP/1.1" 200 None 2026-03-29 01:47:58,076 - lava_callback - DEBUG - Processing node 69c8847af41e4fb6a3785e07: baseline-arm64 2026-03-29 01:47:58,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:47:58,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:47:58,676 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8847af41e4fb6a3785e07/log.txt.gz 2026-03-29 01:47:59,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:47:59,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:47:59,351 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8847af41e4fb6a3785e07/lava-logs.yaml 2026-03-29 01:47:59,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:47:59,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:47:59,575 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c8847af41e4fb6a3785e07/lava_callback.json.gz 2026-03-29 01:47:59,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:47:59,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8847af41e4fb6a3785e07 HTTP/1.1" 200 None 2026-03-29 01:47:59,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:47:59,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:47:59,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:47:59,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8847af41e4fb6a3785e07 HTTP/1.1" 200 None 2026-03-29 01:47:59,735 - lava_callback - INFO - Completed processing callback for node 69c8847af41e4fb6a3785e07 INFO: 172.20.0.1:36444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36894 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:48:21,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:48:21,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41358 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:49:32,998 - lava_callback - INFO - Received callback for node 69c88297f41e4fb6a3785bea INFO: 172.20.0.1:44736 - "POST /node/69c88297f41e4fb6a3785bea HTTP/1.0" 202 Accepted 2026-03-29 01:49:33,004 - lava_callback - INFO - Starting async job submit for node 69c88297f41e4fb6a3785bea 2026-03-29 01:49:33,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:49:33,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88297f41e4fb6a3785bea HTTP/1.1" 200 None 2026-03-29 01:49:33,074 - lava_callback - DEBUG - Processing node 69c88297f41e4fb6a3785bea: baseline-arm64 2026-03-29 01:49:33,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:49:33,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:49:33,733 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88297f41e4fb6a3785bea/log.txt.gz 2026-03-29 01:49:34,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:49:34,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:49:34,248 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88297f41e4fb6a3785bea/lava-logs.yaml 2026-03-29 01:49:34,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:49:34,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:49:34,461 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88297f41e4fb6a3785bea/lava_callback.json.gz 2026-03-29 01:49:34,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:49:34,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88297f41e4fb6a3785bea HTTP/1.1" 200 None 2026-03-29 01:49:34,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:49:34,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:49:34,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:49:34,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88297f41e4fb6a3785bea HTTP/1.1" 200 None 2026-03-29 01:49:34,606 - lava_callback - INFO - Completed processing callback for node 69c88297f41e4fb6a3785bea INFO: 172.20.0.1:44740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36418 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:49:51,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:49:51,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38230 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:50:38,519 - lava_callback - INFO - Received callback for node 69c88561f41e4fb6a3785e44 INFO: 172.20.0.1:38236 - "POST /node/69c88561f41e4fb6a3785e44 HTTP/1.0" 202 Accepted 2026-03-29 01:50:38,524 - lava_callback - INFO - Starting async job submit for node 69c88561f41e4fb6a3785e44 2026-03-29 01:50:38,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:50:38,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88561f41e4fb6a3785e44 HTTP/1.1" 200 None 2026-03-29 01:50:38,554 - lava_callback - DEBUG - Processing node 69c88561f41e4fb6a3785e44: baseline-nfs-arm64 2026-03-29 01:50:38,555 - lava_callback - WARNING - Failed to upload log 2026-03-29 01:50:38,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:50:38,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:50:38,850 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c88561f41e4fb6a3785e44/lava-logs.yaml 2026-03-29 01:50:38,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:50:39,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:50:39,020 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c88561f41e4fb6a3785e44/lava_callback.json.gz 2026-03-29 01:50:39,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:50:39,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88561f41e4fb6a3785e44 HTTP/1.1" 200 None 2026-03-29 01:50:39,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:50:39,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:50:39,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:50:39,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88561f41e4fb6a3785e44 HTTP/1.1" 200 None 2026-03-29 01:50:39,078 - lava_callback - INFO - Completed processing callback for node 69c88561f41e4fb6a3785e44 INFO: 172.20.0.1:41716 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:50:51,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:50:51,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33242 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:53:09,354 - lava_callback - INFO - Received callback for node 69c881cdf41e4fb6a3785b6d INFO: 172.20.0.1:33250 - "POST /node/69c881cdf41e4fb6a3785b6d HTTP/1.0" 202 Accepted 2026-03-29 01:53:09,361 - lava_callback - INFO - Starting async job submit for node 69c881cdf41e4fb6a3785b6d 2026-03-29 01:53:09,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:53:09,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c881cdf41e4fb6a3785b6d HTTP/1.1" 200 None 2026-03-29 01:53:09,429 - lava_callback - DEBUG - Processing node 69c881cdf41e4fb6a3785b6d: baseline-arm 2026-03-29 01:53:10,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:53:10,234 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:53:10,235 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c881cdf41e4fb6a3785b6d/log.txt.gz 2026-03-29 01:53:10,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:53:10,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:53:10,857 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c881cdf41e4fb6a3785b6d/lava-logs.yaml 2026-03-29 01:53:10,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:53:11,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:53:11,053 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c881cdf41e4fb6a3785b6d/lava_callback.json.gz DEBUG: Setup failed for 69c881cdf41e4fb6a3785b6d. Transitting job node result: incomplete -> fail 2026-03-29 01:53:11,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:53:11,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c881cdf41e4fb6a3785b6d HTTP/1.1" 200 None 2026-03-29 01:53:11,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:53:11,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 01:53:11,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:53:11,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c881cdf41e4fb6a3785b6d HTTP/1.1" 200 None 2026-03-29 01:53:11,340 - lava_callback - INFO - Completed processing callback for node 69c881cdf41e4fb6a3785b6d INFO: 172.20.0.1:45936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45946 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:53:21,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:53:21,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46744 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:54:08,116 - lava_callback - INFO - Received callback for node 69c5e3392cb62322ad968d7e INFO: 172.20.0.1:46746 - "POST /node/69c5e3392cb62322ad968d7e HTTP/1.0" 202 Accepted 2026-03-29 01:54:08,122 - lava_callback - INFO - Starting async job submit for node 69c5e3392cb62322ad968d7e 2026-03-29 01:54:08,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:54:08,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e3392cb62322ad968d7e HTTP/1.1" 200 None 2026-03-29 01:54:08,139 - lava_callback - DEBUG - Processing node 69c5e3392cb62322ad968d7e: baseline-arm64-baylibre 2026-03-29 01:54:08,139 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e3392cb62322ad968d7e 2026-03-29 01:54:08,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:54:08,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e3392cb62322ad968d7e HTTP/1.1" 200 None 2026-03-29 01:54:08,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:54:08,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 01:54:08,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:54:08,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e3392cb62322ad968d7e HTTP/1.1" 200 None 2026-03-29 01:54:08,198 - lava_callback - ERROR - Error processing callback for node 69c5e3392cb62322ad968d7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34972 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:54:32,979 - lava_callback - INFO - Received callback for node 69c882c3f41e4fb6a3785c6c INFO: 172.20.0.1:35524 - "POST /node/69c882c3f41e4fb6a3785c6c HTTP/1.0" 202 Accepted 2026-03-29 01:54:32,986 - lava_callback - INFO - Starting async job submit for node 69c882c3f41e4fb6a3785c6c 2026-03-29 01:54:33,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:54:33,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c3f41e4fb6a3785c6c HTTP/1.1" 200 None 2026-03-29 01:54:33,048 - lava_callback - DEBUG - Processing node 69c882c3f41e4fb6a3785c6c: baseline-arm64-clabbe 2026-03-29 01:54:33,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:54:33,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:54:33,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c882c3f41e4fb6a3785c6c/log.txt.gz 2026-03-29 01:54:34,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:54:34,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:54:34,713 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c882c3f41e4fb6a3785c6c/lava-logs.yaml 2026-03-29 01:54:34,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:54:34,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:54:34,908 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c882c3f41e4fb6a3785c6c/lava_callback.json.gz 2026-03-29 01:54:34,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:54:34,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882c3f41e4fb6a3785c6c HTTP/1.1" 200 None 2026-03-29 01:54:34,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:54:34,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:54:34,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:54:35,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882c3f41e4fb6a3785c6c HTTP/1.1" 200 None 2026-03-29 01:54:35,009 - lava_callback - INFO - Completed processing callback for node 69c882c3f41e4fb6a3785c6c INFO: 172.20.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41408 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:54:51,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:54:51,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49484 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:56:31,592 - lava_callback - INFO - Received callback for node 69c88686f41e4fb6a3785e8d INFO: 172.20.0.1:52910 - "POST /node/69c88686f41e4fb6a3785e8d HTTP/1.0" 202 Accepted 2026-03-29 01:56:31,598 - lava_callback - INFO - Starting async job submit for node 69c88686f41e4fb6a3785e8d 2026-03-29 01:56:31,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:56:31,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88686f41e4fb6a3785e8d HTTP/1.1" 200 None 2026-03-29 01:56:31,644 - lava_callback - DEBUG - Processing node 69c88686f41e4fb6a3785e8d: baseline-arm64 2026-03-29 01:56:31,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:56:32,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:56:32,159 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88686f41e4fb6a3785e8d/log.txt.gz 2026-03-29 01:56:32,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:56:32,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:56:32,617 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88686f41e4fb6a3785e8d/lava-logs.yaml 2026-03-29 01:56:32,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:56:32,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:56:32,761 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88686f41e4fb6a3785e8d/lava_callback.json.gz 2026-03-29 01:56:32,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:56:32,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88686f41e4fb6a3785e8d HTTP/1.1" 200 None 2026-03-29 01:56:32,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:56:32,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:56:32,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:56:32,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88686f41e4fb6a3785e8d HTTP/1.1" 200 None 2026-03-29 01:56:32,918 - lava_callback - INFO - Received callback for node 69c88248f41e4fb6a3785b90 2026-03-29 01:56:32,932 - lava_callback - INFO - Completed processing callback for node 69c88686f41e4fb6a3785e8d INFO: 172.20.0.1:52916 - "POST /node/69c88248f41e4fb6a3785b90 HTTP/1.0" 202 Accepted 2026-03-29 01:56:32,969 - lava_callback - INFO - Starting async job submit for node 69c88248f41e4fb6a3785b90 2026-03-29 01:56:33,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:56:33,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88248f41e4fb6a3785b90 HTTP/1.1" 200 None 2026-03-29 01:56:33,025 - lava_callback - DEBUG - Processing node 69c88248f41e4fb6a3785b90: baseline-arm-pengutronix 2026-03-29 01:56:33,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:56:33,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:56:33,334 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c88248f41e4fb6a3785b90/log.txt.gz 2026-03-29 01:56:33,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:56:33,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:56:33,581 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c88248f41e4fb6a3785b90/lava-logs.yaml 2026-03-29 01:56:33,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:56:33,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:56:33,696 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c88248f41e4fb6a3785b90/lava_callback.json.gz DEBUG: Setup failed for 69c88248f41e4fb6a3785b90. Transitting job node result: incomplete -> fail 2026-03-29 01:56:33,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:56:33,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88248f41e4fb6a3785b90 HTTP/1.1" 200 None 2026-03-29 01:56:33,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:56:33,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 01:56:33,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:56:33,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88248f41e4fb6a3785b90 HTTP/1.1" 200 None 2026-03-29 01:56:33,867 - lava_callback - INFO - Completed processing callback for node 69c88248f41e4fb6a3785b90 INFO: 172.20.0.1:52922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40668 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:56:51,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:56:51,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38388 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:57:08,239 - lava_callback - INFO - Received callback for node 69c5e3e42cb62322ad968da5 INFO: 172.20.0.1:38390 - "POST /node/69c5e3e42cb62322ad968da5 HTTP/1.0" 202 Accepted 2026-03-29 01:57:08,245 - lava_callback - INFO - Starting async job submit for node 69c5e3e42cb62322ad968da5 2026-03-29 01:57:08,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:57:08,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e3e42cb62322ad968da5 HTTP/1.1" 200 None 2026-03-29 01:57:08,267 - lava_callback - DEBUG - Processing node 69c5e3e42cb62322ad968da5: baseline-arm64-baylibre 2026-03-29 01:57:08,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e3e42cb62322ad968da5 2026-03-29 01:57:08,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:57:08,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e3e42cb62322ad968da5 HTTP/1.1" 200 None 2026-03-29 01:57:08,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:57:08,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 01:57:08,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:57:08,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e3e42cb62322ad968da5 HTTP/1.1" 200 None 2026-03-29 01:57:08,338 - lava_callback - ERROR - Error processing callback for node 69c5e3e42cb62322ad968da5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60908 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:57:25,464 - lava_callback - INFO - Received callback for node 69c886d0f41e4fb6a3785e99 INFO: 172.20.0.1:34260 - "POST /node/69c886d0f41e4fb6a3785e99 HTTP/1.0" 202 Accepted 2026-03-29 01:57:25,471 - lava_callback - INFO - Starting async job submit for node 69c886d0f41e4fb6a3785e99 2026-03-29 01:57:25,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:57:25,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c886d0f41e4fb6a3785e99 HTTP/1.1" 200 None 2026-03-29 01:57:25,516 - lava_callback - DEBUG - Processing node 69c886d0f41e4fb6a3785e99: baseline-arm64 2026-03-29 01:57:25,676 - lava_callback - WARNING - Failed to upload log 2026-03-29 01:57:25,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:57:25,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:57:25,909 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c886d0f41e4fb6a3785e99/lava-logs.yaml 2026-03-29 01:57:25,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:57:26,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:57:26,062 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c886d0f41e4fb6a3785e99/lava_callback.json.gz 2026-03-29 01:57:26,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:57:26,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c886d0f41e4fb6a3785e99 HTTP/1.1" 200 None 2026-03-29 01:57:26,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:57:26,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:57:26,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:57:26,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c886d0f41e4fb6a3785e99 HTTP/1.1" 200 None 2026-03-29 01:57:26,191 - lava_callback - INFO - Completed processing callback for node 69c886d0f41e4fb6a3785e99 INFO: 172.20.0.1:34264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57498 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:57:51,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:57:51,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50842 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:58:28,513 - lava_callback - INFO - Received callback for node 69c5e43b2cb62322ad968dca INFO: 172.20.0.1:50858 - "POST /node/69c5e43b2cb62322ad968dca HTTP/1.0" 202 Accepted 2026-03-29 01:58:28,518 - lava_callback - INFO - Starting async job submit for node 69c5e43b2cb62322ad968dca 2026-03-29 01:58:28,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:58:28,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e43b2cb62322ad968dca HTTP/1.1" 200 None 2026-03-29 01:58:28,535 - lava_callback - DEBUG - Processing node 69c5e43b2cb62322ad968dca: baseline-arm64-baylibre 2026-03-29 01:58:28,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e43b2cb62322ad968dca 2026-03-29 01:58:28,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:58:28,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e43b2cb62322ad968dca HTTP/1.1" 200 None 2026-03-29 01:58:28,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:58:28,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 01:58:28,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:58:28,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e43b2cb62322ad968dca HTTP/1.1" 200 None 2026-03-29 01:58:28,595 - lava_callback - ERROR - Error processing callback for node 69c5e43b2cb62322ad968dca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36906 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:58:52,777 - lava_callback - INFO - Received callback for node 69c882a1f41e4fb6a3785c02 INFO: 172.20.0.1:51560 - "POST /node/69c882a1f41e4fb6a3785c02 HTTP/1.0" 202 Accepted 2026-03-29 01:58:52,786 - lava_callback - INFO - Starting async job submit for node 69c882a1f41e4fb6a3785c02 2026-03-29 01:58:52,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:58:52,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a1f41e4fb6a3785c02 HTTP/1.1" 200 None 2026-03-29 01:58:52,861 - lava_callback - DEBUG - Processing node 69c882a1f41e4fb6a3785c02: baseline-nfs-arm64 2026-03-29 01:58:54,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:58:54,469 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:58:54,471 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a1f41e4fb6a3785c02/log.txt.gz 2026-03-29 01:58:55,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:58:55,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:58:55,392 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a1f41e4fb6a3785c02/lava-logs.yaml 2026-03-29 01:58:55,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 01:58:55,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 01:58:55,629 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a1f41e4fb6a3785c02/lava_callback.json.gz DEBUG: Setup failed for 69c882a1f41e4fb6a3785c02. Transitting job node result: incomplete -> fail 2026-03-29 01:58:55,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:58:55,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a1f41e4fb6a3785c02 HTTP/1.1" 200 None 2026-03-29 01:58:55,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:58:55,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 01:58:55,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:58:55,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882a1f41e4fb6a3785c02 HTTP/1.1" 200 None 2026-03-29 01:58:55,880 - lava_callback - INFO - Completed processing callback for node 69c882a1f41e4fb6a3785c02 INFO: 172.20.0.1:51570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35096 - "GET / HTTP/1.0" 200 OK 2026-03-29 01:59:21,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 01:59:21,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38058 - "GET /node/69ae1c008ef4424e487b3bcf%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48328 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:01:00,412 - lava_callback - INFO - Received callback for node 69c88764f41e4fb6a3785ec3 INFO: 172.20.0.1:52686 - "POST /node/69c88764f41e4fb6a3785ec3 HTTP/1.0" 202 Accepted 2026-03-29 02:01:00,418 - lava_callback - INFO - Starting async job submit for node 69c88764f41e4fb6a3785ec3 2026-03-29 02:01:00,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:00,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88764f41e4fb6a3785ec3 HTTP/1.1" 200 None 2026-03-29 02:01:00,459 - lava_callback - DEBUG - Processing node 69c88764f41e4fb6a3785ec3: baseline-arm64 2026-03-29 02:01:00,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:01,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:01,079 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88764f41e4fb6a3785ec3/log.txt.gz 2026-03-29 02:01:01,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:01,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:01,610 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88764f41e4fb6a3785ec3/lava-logs.yaml 2026-03-29 02:01:01,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:01,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:01,815 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88764f41e4fb6a3785ec3/lava_callback.json.gz 2026-03-29 02:01:01,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:01,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88764f41e4fb6a3785ec3 HTTP/1.1" 200 None 2026-03-29 02:01:01,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:01,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:01:01,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:01,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88764f41e4fb6a3785ec3 HTTP/1.1" 200 None 2026-03-29 02:01:01,963 - lava_callback - INFO - Completed processing callback for node 69c88764f41e4fb6a3785ec3 INFO: 172.20.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44176 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:01:21,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:21,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51894 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:01:38,870 - lava_callback - INFO - Received callback for node 69c887fdf41e4fb6a3785ee3 INFO: 172.20.0.1:51910 - "POST /node/69c887fdf41e4fb6a3785ee3 HTTP/1.0" 202 Accepted 2026-03-29 02:01:38,876 - lava_callback - INFO - Starting async job submit for node 69c887fdf41e4fb6a3785ee3 2026-03-29 02:01:38,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:38,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c887fdf41e4fb6a3785ee3 HTTP/1.1" 200 None 2026-03-29 02:01:38,901 - lava_callback - DEBUG - Processing node 69c887fdf41e4fb6a3785ee3: baseline-nfs-arm64 2026-03-29 02:01:38,902 - lava_callback - WARNING - Failed to upload log 2026-03-29 02:01:38,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:39,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:39,054 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c887fdf41e4fb6a3785ee3/lava-logs.yaml 2026-03-29 02:01:39,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:39,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:39,198 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c887fdf41e4fb6a3785ee3/lava_callback.json.gz 2026-03-29 02:01:39,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:39,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c887fdf41e4fb6a3785ee3 HTTP/1.1" 200 None 2026-03-29 02:01:39,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:39,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:01:39,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:39,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c887fdf41e4fb6a3785ee3 HTTP/1.1" 200 None 2026-03-29 02:01:39,253 - lava_callback - INFO - Completed processing callback for node 69c887fdf41e4fb6a3785ee3 2026-03-29 02:01:41,691 - lava_callback - INFO - Received callback for node 69c882a7f41e4fb6a3785c11 INFO: 172.20.0.1:34250 - "POST /node/69c882a7f41e4fb6a3785c11 HTTP/1.0" 202 Accepted 2026-03-29 02:01:41,698 - lava_callback - INFO - Starting async job submit for node 69c882a7f41e4fb6a3785c11 2026-03-29 02:01:41,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:41,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a7f41e4fb6a3785c11 HTTP/1.1" 200 None 2026-03-29 02:01:41,739 - lava_callback - DEBUG - Processing node 69c882a7f41e4fb6a3785c11: baseline-nfs-arm64 2026-03-29 02:01:42,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:42,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:42,404 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a7f41e4fb6a3785c11/log.txt.gz 2026-03-29 02:01:42,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:42,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:42,861 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a7f41e4fb6a3785c11/lava-logs.yaml 2026-03-29 02:01:42,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:43,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:43,027 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a7f41e4fb6a3785c11/lava_callback.json.gz 2026-03-29 02:01:43,077 - lava_callback - INFO - Received callback for node 69c88641f41e4fb6a3785e78 INFO: 172.20.0.1:34260 - "POST /node/69c88641f41e4fb6a3785e78 HTTP/1.0" 202 Accepted 2026-03-29 02:01:43,088 - lava_callback - INFO - Starting async job submit for node 69c88641f41e4fb6a3785e78 2026-03-29 02:01:43,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:43,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:43,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a7f41e4fb6a3785c11 HTTP/1.1" 200 None 2026-03-29 02:01:43,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:43,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88641f41e4fb6a3785e78 HTTP/1.1" 200 None 2026-03-29 02:01:43,146 - lava_callback - DEBUG - Processing node 69c88641f41e4fb6a3785e78: baseline-arm64 2026-03-29 02:01:43,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:01:43,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:43,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882a7f41e4fb6a3785c11 HTTP/1.1" 200 None 2026-03-29 02:01:43,429 - lava_callback - INFO - Completed processing callback for node 69c882a7f41e4fb6a3785c11 2026-03-29 02:01:43,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:43,799 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:43,801 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88641f41e4fb6a3785e78/log.txt.gz 2026-03-29 02:01:44,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:44,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:44,238 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88641f41e4fb6a3785e78/lava-logs.yaml 2026-03-29 02:01:44,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:01:44,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:01:44,408 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88641f41e4fb6a3785e78/lava_callback.json.gz DEBUG: Setup failed for 69c88641f41e4fb6a3785e78. Transitting job node result: incomplete -> fail 2026-03-29 02:01:44,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:44,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88641f41e4fb6a3785e78 HTTP/1.1" 200 None 2026-03-29 02:01:44,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:44,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:01:44,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:44,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88641f41e4fb6a3785e78 HTTP/1.1" 200 None 2026-03-29 02:01:44,626 - lava_callback - INFO - Completed processing callback for node 69c88641f41e4fb6a3785e78 INFO: 172.20.0.1:34270 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:01:48,482 - lava_callback - INFO - Received callback for node 69c5e5082cb62322ad968e18 INFO: 172.20.0.1:34286 - "POST /node/69c5e5082cb62322ad968e18 HTTP/1.0" 202 Accepted 2026-03-29 02:01:48,487 - lava_callback - INFO - Starting async job submit for node 69c5e5082cb62322ad968e18 2026-03-29 02:01:48,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:48,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e5082cb62322ad968e18 HTTP/1.1" 200 None 2026-03-29 02:01:48,503 - lava_callback - DEBUG - Processing node 69c5e5082cb62322ad968e18: baseline-arm64-baylibre 2026-03-29 02:01:48,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e5082cb62322ad968e18 2026-03-29 02:01:48,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:48,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e5082cb62322ad968e18 HTTP/1.1" 200 None 2026-03-29 02:01:48,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:48,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 02:01:48,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:48,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e5082cb62322ad968e18 HTTP/1.1" 200 None 2026-03-29 02:01:48,570 - lava_callback - ERROR - Error processing callback for node 69c5e5082cb62322ad968e18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 02:01:51,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:01:51,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48198 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:03:28,598 - lava_callback - INFO - Received callback for node 69c5e5602cb62322ad968e27 INFO: 172.20.0.1:48214 - "POST /node/69c5e5602cb62322ad968e27 HTTP/1.0" 202 Accepted 2026-03-29 02:03:28,604 - lava_callback - INFO - Starting async job submit for node 69c5e5602cb62322ad968e27 2026-03-29 02:03:28,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:03:28,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e5602cb62322ad968e27 HTTP/1.1" 200 None 2026-03-29 02:03:28,619 - lava_callback - DEBUG - Processing node 69c5e5602cb62322ad968e27: baseline-arm64-baylibre 2026-03-29 02:03:28,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e5602cb62322ad968e27 2026-03-29 02:03:28,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:03:28,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e5602cb62322ad968e27 HTTP/1.1" 200 None 2026-03-29 02:03:28,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:03:28,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 02:03:28,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:03:28,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e5602cb62322ad968e27 HTTP/1.1" 200 None 2026-03-29 02:03:28,673 - lava_callback - ERROR - Error processing callback for node 69c5e5602cb62322ad968e27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35712 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:04:12,607 - lava_callback - INFO - Received callback for node 69c88862f41e4fb6a3785f05 INFO: 172.20.0.1:37474 - "POST /node/69c88862f41e4fb6a3785f05 HTTP/1.0" 202 Accepted 2026-03-29 02:04:12,614 - lava_callback - INFO - Starting async job submit for node 69c88862f41e4fb6a3785f05 2026-03-29 02:04:12,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:04:12,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88862f41e4fb6a3785f05 HTTP/1.1" 200 None 2026-03-29 02:04:12,666 - lava_callback - DEBUG - Processing node 69c88862f41e4fb6a3785f05: baseline-arm64 2026-03-29 02:04:12,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:04:13,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:04:13,142 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88862f41e4fb6a3785f05/log.txt.gz 2026-03-29 02:04:13,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:04:13,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:04:13,431 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88862f41e4fb6a3785f05/lava-logs.yaml 2026-03-29 02:04:13,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:04:13,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:04:13,580 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88862f41e4fb6a3785f05/lava_callback.json.gz 2026-03-29 02:04:13,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:04:13,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88862f41e4fb6a3785f05 HTTP/1.1" 200 None 2026-03-29 02:04:13,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:04:13,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:04:13,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:04:13,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88862f41e4fb6a3785f05 HTTP/1.1" 200 None 2026-03-29 02:04:13,756 - lava_callback - INFO - Completed processing callback for node 69c88862f41e4fb6a3785f05 INFO: 172.20.0.1:37482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37496 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:04:21,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:04:21,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-29 02:04:26,478 - lava_callback - INFO - Received callback for node 69c88475f41e4fb6a3785e04 INFO: 172.20.0.1:40576 - "POST /node/69c88475f41e4fb6a3785e04 HTTP/1.0" 202 Accepted 2026-03-29 02:04:26,485 - lava_callback - INFO - Starting async job submit for node 69c88475f41e4fb6a3785e04 INFO: 172.20.0.1:40590 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:04:26,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:04:26,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88475f41e4fb6a3785e04 HTTP/1.1" 200 None 2026-03-29 02:04:26,560 - lava_callback - DEBUG - Processing node 69c88475f41e4fb6a3785e04: ltp-smoke 2026-03-29 02:04:27,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:04:27,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:04:27,218 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c88475f41e4fb6a3785e04/log.txt.gz 2026-03-29 02:04:27,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:04:27,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:04:27,731 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c88475f41e4fb6a3785e04/lava-logs.yaml 2026-03-29 02:04:27,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:04:27,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:04:27,890 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c88475f41e4fb6a3785e04/lava_callback.json.gz 2026-03-29 02:04:28,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:04:28,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88475f41e4fb6a3785e04 HTTP/1.1" 200 None 2026-03-29 02:04:28,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:04:28,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f76f41e4fb6a3782f44 HTTP/1.1" 200 None 2026-03-29 02:04:28,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:04:28,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88475f41e4fb6a3785e04 HTTP/1.1" 200 None 2026-03-29 02:04:28,214 - lava_callback - INFO - Completed processing callback for node 69c88475f41e4fb6a3785e04 INFO: 172.20.0.1:43910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57844 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:04:51,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:04:51,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60602 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:06:07,642 - lava_callback - INFO - Received callback for node 69c888d0f41e4fb6a3785f26 INFO: 172.20.0.1:60610 - "POST /node/69c888d0f41e4fb6a3785f26 HTTP/1.0" 202 Accepted 2026-03-29 02:06:07,649 - lava_callback - INFO - Starting async job submit for node 69c888d0f41e4fb6a3785f26 2026-03-29 02:06:07,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:06:07,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c888d0f41e4fb6a3785f26 HTTP/1.1" 200 None 2026-03-29 02:06:07,692 - lava_callback - DEBUG - Processing node 69c888d0f41e4fb6a3785f26: baseline-arm64 2026-03-29 02:06:07,826 - lava_callback - WARNING - Failed to upload log 2026-03-29 02:06:07,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:06:08,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:06:08,090 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c888d0f41e4fb6a3785f26/lava-logs.yaml 2026-03-29 02:06:08,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:06:08,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:06:08,235 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c888d0f41e4fb6a3785f26/lava_callback.json.gz 2026-03-29 02:06:08,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:06:08,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c888d0f41e4fb6a3785f26 HTTP/1.1" 200 None 2026-03-29 02:06:08,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:06:08,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:06:08,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:06:08,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c888d0f41e4fb6a3785f26 HTTP/1.1" 200 None 2026-03-29 02:06:08,403 - lava_callback - INFO - Completed processing callback for node 69c888d0f41e4fb6a3785f26 2026-03-29 02:06:08,767 - lava_callback - INFO - Received callback for node 69c5e60b2cb62322ad968e5b INFO: 172.20.0.1:60618 - "POST /node/69c5e60b2cb62322ad968e5b HTTP/1.0" 202 Accepted 2026-03-29 02:06:08,772 - lava_callback - INFO - Starting async job submit for node 69c5e60b2cb62322ad968e5b 2026-03-29 02:06:08,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:06:08,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e60b2cb62322ad968e5b HTTP/1.1" 200 None 2026-03-29 02:06:08,789 - lava_callback - DEBUG - Processing node 69c5e60b2cb62322ad968e5b: baseline-arm64-baylibre 2026-03-29 02:06:08,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e60b2cb62322ad968e5b 2026-03-29 02:06:08,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:06:08,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e60b2cb62322ad968e5b HTTP/1.1" 200 None 2026-03-29 02:06:08,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:06:08,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 02:06:08,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:06:08,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e60b2cb62322ad968e5b HTTP/1.1" 200 None 2026-03-29 02:06:08,855 - lava_callback - ERROR - Error processing callback for node 69c5e60b2cb62322ad968e5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52824 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:06:21,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:06:21,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42766 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:07:47,285 - lava_callback - INFO - Received callback for node 69c5e6652cb62322ad968e7d INFO: 172.20.0.1:42768 - "POST /node/69c5e6652cb62322ad968e7d HTTP/1.0" 202 Accepted 2026-03-29 02:07:47,291 - lava_callback - INFO - Starting async job submit for node 69c5e6652cb62322ad968e7d 2026-03-29 02:07:47,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:07:47,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e6652cb62322ad968e7d HTTP/1.1" 200 None 2026-03-29 02:07:47,306 - lava_callback - DEBUG - Processing node 69c5e6652cb62322ad968e7d: baseline-arm64-clabbe 2026-03-29 02:07:47,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e6652cb62322ad968e7d 2026-03-29 02:07:47,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:07:47,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e6652cb62322ad968e7d HTTP/1.1" 200 None 2026-03-29 02:07:47,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:07:47,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 02:07:47,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:07:47,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e6652cb62322ad968e7d HTTP/1.1" 200 None 2026-03-29 02:07:47,389 - lava_callback - ERROR - Error processing callback for node 69c5e6652cb62322ad968e7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56338 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:09:42,150 - lava_callback - INFO - Received callback for node 69c88967f41e4fb6a3785f39 INFO: 172.20.0.1:39170 - "POST /node/69c88967f41e4fb6a3785f39 HTTP/1.0" 202 Accepted 2026-03-29 02:09:42,156 - lava_callback - INFO - Starting async job submit for node 69c88967f41e4fb6a3785f39 2026-03-29 02:09:42,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:09:42,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88967f41e4fb6a3785f39 HTTP/1.1" 200 None 2026-03-29 02:09:42,193 - lava_callback - DEBUG - Processing node 69c88967f41e4fb6a3785f39: baseline-arm64 2026-03-29 02:09:42,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:09:42,799 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:09:42,800 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88967f41e4fb6a3785f39/log.txt.gz 2026-03-29 02:09:43,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:09:43,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:09:43,303 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88967f41e4fb6a3785f39/lava-logs.yaml 2026-03-29 02:09:43,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:09:43,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:09:43,494 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88967f41e4fb6a3785f39/lava_callback.json.gz 2026-03-29 02:09:43,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:09:43,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88967f41e4fb6a3785f39 HTTP/1.1" 200 None 2026-03-29 02:09:43,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:09:43,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:09:43,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:09:43,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88967f41e4fb6a3785f39 HTTP/1.1" 200 None 2026-03-29 02:09:43,659 - lava_callback - INFO - Completed processing callback for node 69c88967f41e4fb6a3785f39 INFO: 172.20.0.1:39184 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:09:51,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:09:51,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50894 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:10:47,431 - lava_callback - INFO - Received callback for node 69c5e7162cb62322ad968ebe INFO: 172.20.0.1:50906 - "POST /node/69c5e7162cb62322ad968ebe HTTP/1.0" 202 Accepted 2026-03-29 02:10:47,436 - lava_callback - INFO - Starting async job submit for node 69c5e7162cb62322ad968ebe 2026-03-29 02:10:47,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:10:47,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e7162cb62322ad968ebe HTTP/1.1" 200 None 2026-03-29 02:10:47,451 - lava_callback - DEBUG - Processing node 69c5e7162cb62322ad968ebe: baseline-arm64-clabbe 2026-03-29 02:10:47,452 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e7162cb62322ad968ebe 2026-03-29 02:10:47,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:10:47,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e7162cb62322ad968ebe HTTP/1.1" 200 None 2026-03-29 02:10:47,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:10:47,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 02:10:47,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:10:47,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e7162cb62322ad968ebe HTTP/1.1" 200 None 2026-03-29 02:10:47,514 - lava_callback - ERROR - Error processing callback for node 69c5e7162cb62322ad968ebe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57284 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:12:07,466 - lava_callback - INFO - Received callback for node 69c5e76c2cb62322ad968ec9 INFO: 172.20.0.1:57296 - "POST /node/69c5e76c2cb62322ad968ec9 HTTP/1.0" 202 Accepted 2026-03-29 02:12:07,472 - lava_callback - INFO - Starting async job submit for node 69c5e76c2cb62322ad968ec9 2026-03-29 02:12:07,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:12:07,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e76c2cb62322ad968ec9 HTTP/1.1" 200 None 2026-03-29 02:12:07,505 - lava_callback - DEBUG - Processing node 69c5e76c2cb62322ad968ec9: baseline-arm64-clabbe 2026-03-29 02:12:07,506 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e76c2cb62322ad968ec9 2026-03-29 02:12:07,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:12:07,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e76c2cb62322ad968ec9 HTTP/1.1" 200 None 2026-03-29 02:12:07,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:12:07,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 02:12:07,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:12:07,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e76c2cb62322ad968ec9 HTTP/1.1" 200 None 2026-03-29 02:12:07,570 - lava_callback - ERROR - Error processing callback for node 69c5e76c2cb62322ad968ec9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45604 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:13:27,536 - lava_callback - INFO - Received callback for node 69c5e7c52cb62322ad968edf INFO: 172.20.0.1:45608 - "POST /node/69c5e7c52cb62322ad968edf HTTP/1.0" 202 Accepted 2026-03-29 02:13:27,542 - lava_callback - INFO - Starting async job submit for node 69c5e7c52cb62322ad968edf 2026-03-29 02:13:27,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:13:27,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e7c52cb62322ad968edf HTTP/1.1" 200 None 2026-03-29 02:13:27,559 - lava_callback - DEBUG - Processing node 69c5e7c52cb62322ad968edf: baseline-arm64-clabbe 2026-03-29 02:13:27,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e7c52cb62322ad968edf 2026-03-29 02:13:27,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:13:27,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e7c52cb62322ad968edf HTTP/1.1" 200 None 2026-03-29 02:13:27,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:13:27,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 02:13:27,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:13:27,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e7c52cb62322ad968edf HTTP/1.1" 200 None 2026-03-29 02:13:27,617 - lava_callback - ERROR - Error processing callback for node 69c5e7c52cb62322ad968edf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-29 02:13:35,548 - lava_callback - INFO - Received callback for node 69c88295f41e4fb6a3785be7 INFO: 172.20.0.1:34110 - "POST /node/69c88295f41e4fb6a3785be7 HTTP/1.0" 202 Accepted 2026-03-29 02:13:35,554 - lava_callback - INFO - Starting async job submit for node 69c88295f41e4fb6a3785be7 2026-03-29 02:13:35,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:13:35,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88295f41e4fb6a3785be7 HTTP/1.1" 200 None 2026-03-29 02:13:35,598 - lava_callback - DEBUG - Processing node 69c88295f41e4fb6a3785be7: baseline-arm64 2026-03-29 02:13:35,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:13:36,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:13:36,107 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88295f41e4fb6a3785be7/log.txt.gz 2026-03-29 02:13:36,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:13:36,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:13:36,480 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88295f41e4fb6a3785be7/lava-logs.yaml 2026-03-29 02:13:36,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:34124 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:13:36,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:13:36,644 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88295f41e4fb6a3785be7/lava_callback.json.gz 2026-03-29 02:13:36,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:13:36,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88295f41e4fb6a3785be7 HTTP/1.1" 200 None 2026-03-29 02:13:36,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:13:36,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:13:36,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:13:36,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88295f41e4fb6a3785be7 HTTP/1.1" 200 None 2026-03-29 02:13:36,732 - lava_callback - INFO - Completed processing callback for node 69c88295f41e4fb6a3785be7 INFO: 172.20.0.1:42358 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:13:51,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:13:51,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47390 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:14:10,309 - lava_callback - INFO - Received callback for node 69c887aff41e4fb6a3785ecf INFO: 172.20.0.1:55400 - "POST /node/69c887aff41e4fb6a3785ecf HTTP/1.0" 202 Accepted 2026-03-29 02:14:10,316 - lava_callback - INFO - Starting async job submit for node 69c887aff41e4fb6a3785ecf 2026-03-29 02:14:10,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:14:10,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c887aff41e4fb6a3785ecf HTTP/1.1" 200 None 2026-03-29 02:14:10,387 - lava_callback - DEBUG - Processing node 69c887aff41e4fb6a3785ecf: baseline-arm64 2026-03-29 02:14:10,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:14:11,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:14:11,030 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c887aff41e4fb6a3785ecf/log.txt.gz 2026-03-29 02:14:11,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:14:11,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:14:11,519 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c887aff41e4fb6a3785ecf/lava-logs.yaml 2026-03-29 02:14:11,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:14:11,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:14:11,691 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c887aff41e4fb6a3785ecf/lava_callback.json.gz 2026-03-29 02:14:11,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:14:11,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c887aff41e4fb6a3785ecf HTTP/1.1" 200 None 2026-03-29 02:14:11,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:14:11,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:14:11,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:14:11,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c887aff41e4fb6a3785ecf HTTP/1.1" 200 None 2026-03-29 02:14:11,857 - lava_callback - INFO - Completed processing callback for node 69c887aff41e4fb6a3785ecf INFO: 172.20.0.1:55406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55416 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:14:21,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:14:21,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51756 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:15:30,545 - lava_callback - INFO - Received callback for node 69c889fdf41e4fb6a3785f51 INFO: 172.20.0.1:43204 - "POST /node/69c889fdf41e4fb6a3785f51 HTTP/1.0" 202 Accepted 2026-03-29 02:15:30,554 - lava_callback - INFO - Starting async job submit for node 69c889fdf41e4fb6a3785f51 2026-03-29 02:15:30,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:15:30,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c889fdf41e4fb6a3785f51 HTTP/1.1" 200 None 2026-03-29 02:15:30,607 - lava_callback - DEBUG - Processing node 69c889fdf41e4fb6a3785f51: baseline-arm64 2026-03-29 02:15:31,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:15:31,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:15:31,188 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c889fdf41e4fb6a3785f51/log.txt.gz 2026-03-29 02:15:31,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:15:31,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:15:31,636 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c889fdf41e4fb6a3785f51/lava-logs.yaml 2026-03-29 02:15:31,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:15:31,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:15:31,821 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c889fdf41e4fb6a3785f51/lava_callback.json.gz DEBUG: Setup failed for 69c889fdf41e4fb6a3785f51. Transitting job node result: incomplete -> fail 2026-03-29 02:15:31,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:15:31,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c889fdf41e4fb6a3785f51 HTTP/1.1" 200 None 2026-03-29 02:15:31,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:15:31,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:15:31,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:15:31,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c889fdf41e4fb6a3785f51 HTTP/1.1" 200 None 2026-03-29 02:15:32,017 - lava_callback - INFO - Completed processing callback for node 69c889fdf41e4fb6a3785f51 INFO: 172.20.0.1:43216 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:15:44,395 - lava_callback - INFO - Received callback for node 69c882a5f41e4fb6a3785c0e INFO: 172.20.0.1:43408 - "POST /node/69c882a5f41e4fb6a3785c0e HTTP/1.0" 202 Accepted 2026-03-29 02:15:44,401 - lava_callback - INFO - Starting async job submit for node 69c882a5f41e4fb6a3785c0e 2026-03-29 02:15:44,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:15:44,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a5f41e4fb6a3785c0e HTTP/1.1" 200 None 2026-03-29 02:15:44,445 - lava_callback - DEBUG - Processing node 69c882a5f41e4fb6a3785c0e: baseline-nfs-arm64 2026-03-29 02:15:44,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:15:44,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:15:44,907 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a5f41e4fb6a3785c0e/log.txt.gz 2026-03-29 02:15:45,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:15:45,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:15:45,299 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a5f41e4fb6a3785c0e/lava-logs.yaml 2026-03-29 02:15:45,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:15:45,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:15:45,445 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c882a5f41e4fb6a3785c0e/lava_callback.json.gz 2026-03-29 02:15:45,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:15:45,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c882a5f41e4fb6a3785c0e HTTP/1.1" 200 None 2026-03-29 02:15:45,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:15:45,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:15:45,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:15:45,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c882a5f41e4fb6a3785c0e HTTP/1.1" 200 None 2026-03-29 02:15:45,540 - lava_callback - INFO - Completed processing callback for node 69c882a5f41e4fb6a3785c0e INFO: 172.20.0.1:43416 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:15:51,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:15:51,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42508 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:16:27,651 - lava_callback - INFO - Received callback for node 69c5e8712cb62322ad968f17 INFO: 172.20.0.1:42510 - "POST /node/69c5e8712cb62322ad968f17 HTTP/1.0" 202 Accepted 2026-03-29 02:16:27,656 - lava_callback - INFO - Starting async job submit for node 69c5e8712cb62322ad968f17 2026-03-29 02:16:27,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:16:27,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e8712cb62322ad968f17 HTTP/1.1" 200 None 2026-03-29 02:16:27,680 - lava_callback - DEBUG - Processing node 69c5e8712cb62322ad968f17: baseline-arm64-clabbe 2026-03-29 02:16:27,680 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e8712cb62322ad968f17 2026-03-29 02:16:27,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:16:27,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e8712cb62322ad968f17 HTTP/1.1" 200 None 2026-03-29 02:16:27,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:16:27,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 02:16:27,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:16:27,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e8712cb62322ad968f17 HTTP/1.1" 200 None 2026-03-29 02:16:27,758 - lava_callback - ERROR - Error processing callback for node 69c5e8712cb62322ad968f17: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34738 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:17:47,762 - lava_callback - INFO - Received callback for node 69c5e8c82cb62322ad968f2e INFO: 172.20.0.1:34746 - "POST /node/69c5e8c82cb62322ad968f2e HTTP/1.0" 202 Accepted 2026-03-29 02:17:47,768 - lava_callback - INFO - Starting async job submit for node 69c5e8c82cb62322ad968f2e 2026-03-29 02:17:47,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:17:47,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e8c82cb62322ad968f2e HTTP/1.1" 200 None 2026-03-29 02:17:47,784 - lava_callback - DEBUG - Processing node 69c5e8c82cb62322ad968f2e: baseline-arm64-clabbe 2026-03-29 02:17:47,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e8c82cb62322ad968f2e 2026-03-29 02:17:47,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:17:47,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e8c82cb62322ad968f2e HTTP/1.1" 200 None 2026-03-29 02:17:47,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:17:47,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-29 02:17:47,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:17:47,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e8c82cb62322ad968f2e HTTP/1.1" 200 None 2026-03-29 02:17:47,900 - lava_callback - ERROR - Error processing callback for node 69c5e8c82cb62322ad968f2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52188 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:19:09,527 - lava_callback - INFO - Received callback for node 69c5e9142cb62322ad968f41 INFO: 172.20.0.1:52192 - "POST /node/69c5e9142cb62322ad968f41 HTTP/1.0" 202 Accepted 2026-03-29 02:19:09,532 - lava_callback - INFO - Starting async job submit for node 69c5e9142cb62322ad968f41 2026-03-29 02:19:09,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:19:09,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e9142cb62322ad968f41 HTTP/1.1" 200 None 2026-03-29 02:19:09,570 - lava_callback - DEBUG - Processing node 69c5e9142cb62322ad968f41: baseline-arm64-baylibre 2026-03-29 02:19:09,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e9142cb62322ad968f41 2026-03-29 02:19:09,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:19:09,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e9142cb62322ad968f41 HTTP/1.1" 200 None 2026-03-29 02:19:09,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:19:09,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:19:09,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:19:09,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e9142cb62322ad968f41 HTTP/1.1" 200 None 2026-03-29 02:19:09,669 - lava_callback - ERROR - Error processing callback for node 69c5e9142cb62322ad968f41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53160 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:20:17,828 - lava_callback - INFO - Received callback for node 69c8883ef41e4fb6a3785ef9 INFO: 172.20.0.1:53166 - "POST /node/69c8883ef41e4fb6a3785ef9 HTTP/1.0" 202 Accepted 2026-03-29 02:20:17,834 - lava_callback - INFO - Starting async job submit for node 69c8883ef41e4fb6a3785ef9 2026-03-29 02:20:17,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:20:17,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8883ef41e4fb6a3785ef9 HTTP/1.1" 200 None 2026-03-29 02:20:17,904 - lava_callback - DEBUG - Processing node 69c8883ef41e4fb6a3785ef9: baseline-arm 2026-03-29 02:20:18,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:20:18,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:20:18,663 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c8883ef41e4fb6a3785ef9/log.txt.gz 2026-03-29 02:20:19,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:20:19,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:20:19,267 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c8883ef41e4fb6a3785ef9/lava-logs.yaml 2026-03-29 02:20:19,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:20:19,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:20:19,467 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c8883ef41e4fb6a3785ef9/lava_callback.json.gz DEBUG: Setup failed for 69c8883ef41e4fb6a3785ef9. Transitting job node result: incomplete -> fail 2026-03-29 02:20:19,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:20:19,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8883ef41e4fb6a3785ef9 HTTP/1.1" 200 None 2026-03-29 02:20:19,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:20:19,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 02:20:19,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:20:19,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8883ef41e4fb6a3785ef9 HTTP/1.1" 200 None 2026-03-29 02:20:19,784 - lava_callback - INFO - Completed processing callback for node 69c8883ef41e4fb6a3785ef9 2026-03-29 02:20:21,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:20:21,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39154 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:20:29,582 - lava_callback - INFO - Received callback for node 69c5e96b2cb62322ad968f63 INFO: 172.20.0.1:39162 - "POST /node/69c5e96b2cb62322ad968f63 HTTP/1.0" 202 Accepted 2026-03-29 02:20:29,587 - lava_callback - INFO - Starting async job submit for node 69c5e96b2cb62322ad968f63 2026-03-29 02:20:29,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:20:29,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e96b2cb62322ad968f63 HTTP/1.1" 200 None 2026-03-29 02:20:29,603 - lava_callback - DEBUG - Processing node 69c5e96b2cb62322ad968f63: baseline-arm64-baylibre 2026-03-29 02:20:29,603 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e96b2cb62322ad968f63 2026-03-29 02:20:29,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:20:29,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e96b2cb62322ad968f63 HTTP/1.1" 200 None 2026-03-29 02:20:29,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:20:29,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:20:29,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:20:29,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e96b2cb62322ad968f63 HTTP/1.1" 200 None 2026-03-29 02:20:29,660 - lava_callback - ERROR - Error processing callback for node 69c5e96b2cb62322ad968f63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55984 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:22:09,678 - lava_callback - INFO - Received callback for node 69c5e9c12cb62322ad968f6d INFO: 172.20.0.1:56000 - "POST /node/69c5e9c12cb62322ad968f6d HTTP/1.0" 202 Accepted 2026-03-29 02:22:09,683 - lava_callback - INFO - Starting async job submit for node 69c5e9c12cb62322ad968f6d 2026-03-29 02:22:09,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:22:09,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e9c12cb62322ad968f6d HTTP/1.1" 200 None 2026-03-29 02:22:09,715 - lava_callback - DEBUG - Processing node 69c5e9c12cb62322ad968f6d: baseline-arm64-baylibre 2026-03-29 02:22:09,715 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5e9c12cb62322ad968f6d 2026-03-29 02:22:09,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:22:09,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5e9c12cb62322ad968f6d HTTP/1.1" 200 None 2026-03-29 02:22:09,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:22:09,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:22:09,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:22:09,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5e9c12cb62322ad968f6d HTTP/1.1" 200 None 2026-03-29 02:22:09,787 - lava_callback - ERROR - Error processing callback for node 69c5e9c12cb62322ad968f6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60348 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:23:11,832 - lava_callback - INFO - Received callback for node 69c88b4af41e4fb6a3785f6f INFO: 172.20.0.1:35608 - "POST /node/69c88b4af41e4fb6a3785f6f HTTP/1.0" 202 Accepted 2026-03-29 02:23:11,839 - lava_callback - INFO - Starting async job submit for node 69c88b4af41e4fb6a3785f6f 2026-03-29 02:23:11,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:23:11,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88b4af41e4fb6a3785f6f HTTP/1.1" 200 None 2026-03-29 02:23:11,890 - lava_callback - DEBUG - Processing node 69c88b4af41e4fb6a3785f6f: baseline-arm64 2026-03-29 02:23:12,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:23:12,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:23:12,547 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88b4af41e4fb6a3785f6f/log.txt.gz 2026-03-29 02:23:12,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:23:12,998 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:23:13,001 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88b4af41e4fb6a3785f6f/lava-logs.yaml 2026-03-29 02:23:13,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:23:13,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:23:13,200 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88b4af41e4fb6a3785f6f/lava_callback.json.gz DEBUG: Setup failed for 69c88b4af41e4fb6a3785f6f. Transitting job node result: incomplete -> fail 2026-03-29 02:23:13,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:23:13,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88b4af41e4fb6a3785f6f HTTP/1.1" 200 None 2026-03-29 02:23:13,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:23:13,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:23:13,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:23:13,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88b4af41e4fb6a3785f6f HTTP/1.1" 200 None 2026-03-29 02:23:13,480 - lava_callback - INFO - Completed processing callback for node 69c88b4af41e4fb6a3785f6f INFO: 172.20.0.1:35622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35634 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:23:21,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:23:21,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52446 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:23:54,536 - lava_callback - INFO - Received callback for node 69c888b6f41e4fb6a3785f17 INFO: 172.20.0.1:58008 - "POST /node/69c888b6f41e4fb6a3785f17 HTTP/1.0" 202 Accepted 2026-03-29 02:23:54,585 - lava_callback - INFO - Starting async job submit for node 69c888b6f41e4fb6a3785f17 2026-03-29 02:23:54,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:23:54,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c888b6f41e4fb6a3785f17 HTTP/1.1" 200 None 2026-03-29 02:23:54,626 - lava_callback - DEBUG - Processing node 69c888b6f41e4fb6a3785f17: baseline-arm-pengutronix 2026-03-29 02:23:54,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:23:54,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:23:54,891 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c888b6f41e4fb6a3785f17/log.txt.gz 2026-03-29 02:23:54,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:23:55,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:23:55,127 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c888b6f41e4fb6a3785f17/lava-logs.yaml 2026-03-29 02:23:55,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:23:55,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:23:55,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c888b6f41e4fb6a3785f17/lava_callback.json.gz DEBUG: Setup failed for 69c888b6f41e4fb6a3785f17. Transitting job node result: incomplete -> fail 2026-03-29 02:23:55,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:23:55,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c888b6f41e4fb6a3785f17 HTTP/1.1" 200 None 2026-03-29 02:23:55,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:23:55,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-03-29 02:23:55,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:23:55,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c888b6f41e4fb6a3785f17 HTTP/1.1" 200 None 2026-03-29 02:23:55,441 - lava_callback - INFO - Completed processing callback for node 69c888b6f41e4fb6a3785f17 INFO: 172.20.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55618 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:24:21,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:24:21,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58692 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:24:49,871 - lava_callback - INFO - Received callback for node 69c5ea6a2cb62322ad968f97 INFO: 172.20.0.1:58272 - "POST /node/69c5ea6a2cb62322ad968f97 HTTP/1.0" 202 Accepted 2026-03-29 02:24:49,877 - lava_callback - INFO - Starting async job submit for node 69c5ea6a2cb62322ad968f97 2026-03-29 02:24:49,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:24:49,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ea6a2cb62322ad968f97 HTTP/1.1" 200 None 2026-03-29 02:24:49,900 - lava_callback - DEBUG - Processing node 69c5ea6a2cb62322ad968f97: baseline-arm64-baylibre 2026-03-29 02:24:49,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5ea6a2cb62322ad968f97 2026-03-29 02:24:49,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:24:49,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ea6a2cb62322ad968f97 HTTP/1.1" 200 None 2026-03-29 02:24:49,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:24:49,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:24:49,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:24:49,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ea6a2cb62322ad968f97 HTTP/1.1" 200 None 2026-03-29 02:24:49,955 - lava_callback - ERROR - Error processing callback for node 69c5ea6a2cb62322ad968f97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60926 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:26:20,814 - lava_callback - INFO - Received callback for node 69c88921f41e4fb6a3785f33 INFO: 172.20.0.1:43612 - "POST /node/69c88921f41e4fb6a3785f33 HTTP/1.0" 202 Accepted 2026-03-29 02:26:20,822 - lava_callback - INFO - Starting async job submit for node 69c88921f41e4fb6a3785f33 2026-03-29 02:26:20,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:26:20,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88921f41e4fb6a3785f33 HTTP/1.1" 200 None 2026-03-29 02:26:20,884 - lava_callback - DEBUG - Processing node 69c88921f41e4fb6a3785f33: baseline-nfs-arm64 2026-03-29 02:26:22,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:26:22,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:26:22,237 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c88921f41e4fb6a3785f33/log.txt.gz 2026-03-29 02:26:22,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:26:23,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:26:23,311 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c88921f41e4fb6a3785f33/lava-logs.yaml 2026-03-29 02:26:23,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:26:23,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:26:23,696 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c88921f41e4fb6a3785f33/lava_callback.json.gz DEBUG: Setup failed for 69c88921f41e4fb6a3785f33. Transitting job node result: incomplete -> fail 2026-03-29 02:26:23,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:26:23,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88921f41e4fb6a3785f33 HTTP/1.1" 200 None 2026-03-29 02:26:23,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:26:23,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:26:23,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:26:23,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88921f41e4fb6a3785f33 HTTP/1.1" 200 None 2026-03-29 02:26:23,925 - lava_callback - INFO - Completed processing callback for node 69c88921f41e4fb6a3785f33 INFO: 172.20.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32876 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:26:46,248 - lava_callback - INFO - Received callback for node 69c889b7f41e4fb6a3785f45 INFO: 172.20.0.1:35266 - "POST /node/69c889b7f41e4fb6a3785f45 HTTP/1.0" 202 Accepted 2026-03-29 02:26:46,255 - lava_callback - INFO - Starting async job submit for node 69c889b7f41e4fb6a3785f45 2026-03-29 02:26:46,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:26:46,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c889b7f41e4fb6a3785f45 HTTP/1.1" 200 None 2026-03-29 02:26:46,295 - lava_callback - DEBUG - Processing node 69c889b7f41e4fb6a3785f45: baseline-nfs-arm64 INFO: 172.20.0.1:35280 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:26:46,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:26:46,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:26:46,966 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c889b7f41e4fb6a3785f45/log.txt.gz 2026-03-29 02:26:47,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:26:47,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:26:47,500 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c889b7f41e4fb6a3785f45/lava-logs.yaml 2026-03-29 02:26:47,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:26:47,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:26:47,672 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c889b7f41e4fb6a3785f45/lava_callback.json.gz 2026-03-29 02:26:47,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:26:47,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c889b7f41e4fb6a3785f45 HTTP/1.1" 200 None 2026-03-29 02:26:47,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:26:47,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:26:47,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:26:47,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c889b7f41e4fb6a3785f45 HTTP/1.1" 200 None 2026-03-29 02:26:47,827 - lava_callback - INFO - Completed processing callback for node 69c889b7f41e4fb6a3785f45 2026-03-29 02:26:51,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:26:51,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46802 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:27:49,979 - lava_callback - INFO - Received callback for node 69c5eb142cb62322ad968fec INFO: 172.20.0.1:55208 - "POST /node/69c5eb142cb62322ad968fec HTTP/1.0" 202 Accepted 2026-03-29 02:27:49,984 - lava_callback - INFO - Starting async job submit for node 69c5eb142cb62322ad968fec 2026-03-29 02:27:49,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:27:49,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eb142cb62322ad968fec HTTP/1.1" 200 None 2026-03-29 02:27:50,000 - lava_callback - DEBUG - Processing node 69c5eb142cb62322ad968fec: baseline-arm64-baylibre 2026-03-29 02:27:50,000 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5eb142cb62322ad968fec 2026-03-29 02:27:50,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:27:50,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eb142cb62322ad968fec HTTP/1.1" 200 None 2026-03-29 02:27:50,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:27:50,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:27:50,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:27:50,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5eb142cb62322ad968fec HTTP/1.1" 200 None 2026-03-29 02:27:50,066 - lava_callback - ERROR - Error processing callback for node 69c5eb142cb62322ad968fec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43794 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:29:10,121 - lava_callback - INFO - Received callback for node 69c5eb692cb62322ad96900f INFO: 172.20.0.1:52716 - "POST /node/69c5eb692cb62322ad96900f HTTP/1.0" 202 Accepted 2026-03-29 02:29:10,127 - lava_callback - INFO - Starting async job submit for node 69c5eb692cb62322ad96900f 2026-03-29 02:29:10,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:29:10,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eb692cb62322ad96900f HTTP/1.1" 200 None 2026-03-29 02:29:10,143 - lava_callback - DEBUG - Processing node 69c5eb692cb62322ad96900f: baseline-arm64-baylibre 2026-03-29 02:29:10,143 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5eb692cb62322ad96900f 2026-03-29 02:29:10,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:29:10,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eb692cb62322ad96900f HTTP/1.1" 200 None 2026-03-29 02:29:10,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:29:10,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:29:10,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:29:10,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5eb692cb62322ad96900f HTTP/1.1" 200 None 2026-03-29 02:29:10,197 - lava_callback - ERROR - Error processing callback for node 69c5eb692cb62322ad96900f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35266 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:30:48,239 - lava_callback - INFO - Received callback for node 69c5ebc62cb62322ad969024 INFO: 172.20.0.1:35274 - "POST /node/69c5ebc62cb62322ad969024 HTTP/1.0" 202 Accepted 2026-03-29 02:30:48,245 - lava_callback - INFO - Starting async job submit for node 69c5ebc62cb62322ad969024 2026-03-29 02:30:48,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:30:48,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ebc62cb62322ad969024 HTTP/1.1" 200 None 2026-03-29 02:30:48,260 - lava_callback - DEBUG - Processing node 69c5ebc62cb62322ad969024: baseline-arm64-clabbe 2026-03-29 02:30:48,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5ebc62cb62322ad969024 2026-03-29 02:30:48,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:30:48,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ebc62cb62322ad969024 HTTP/1.1" 200 None 2026-03-29 02:30:48,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:30:48,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:30:48,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:30:48,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ebc62cb62322ad969024 HTTP/1.1" 200 None 2026-03-29 02:30:48,320 - lava_callback - ERROR - Error processing callback for node 69c5ebc62cb62322ad969024: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33090 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:32:08,312 - lava_callback - INFO - Received callback for node 69c5ec1a2cb62322ad969034 INFO: 172.20.0.1:33094 - "POST /node/69c5ec1a2cb62322ad969034 HTTP/1.0" 202 Accepted 2026-03-29 02:32:08,318 - lava_callback - INFO - Starting async job submit for node 69c5ec1a2cb62322ad969034 2026-03-29 02:32:08,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:32:08,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ec1a2cb62322ad969034 HTTP/1.1" 200 None 2026-03-29 02:32:08,335 - lava_callback - DEBUG - Processing node 69c5ec1a2cb62322ad969034: baseline-arm64-clabbe 2026-03-29 02:32:08,335 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5ec1a2cb62322ad969034 2026-03-29 02:32:08,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:32:08,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ec1a2cb62322ad969034 HTTP/1.1" 200 None 2026-03-29 02:32:08,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:32:08,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:32:08,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:32:08,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ec1a2cb62322ad969034 HTTP/1.1" 200 None 2026-03-29 02:32:08,386 - lava_callback - ERROR - Error processing callback for node 69c5ec1a2cb62322ad969034: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36842 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:33:28,369 - lava_callback - INFO - Received callback for node 69c5ec6f2cb62322ad969044 INFO: 172.20.0.1:36854 - "POST /node/69c5ec6f2cb62322ad969044 HTTP/1.0" 202 Accepted 2026-03-29 02:33:28,374 - lava_callback - INFO - Starting async job submit for node 69c5ec6f2cb62322ad969044 2026-03-29 02:33:28,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:33:28,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ec6f2cb62322ad969044 HTTP/1.1" 200 None 2026-03-29 02:33:28,391 - lava_callback - DEBUG - Processing node 69c5ec6f2cb62322ad969044: baseline-arm64-clabbe 2026-03-29 02:33:28,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5ec6f2cb62322ad969044 2026-03-29 02:33:28,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:33:28,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ec6f2cb62322ad969044 HTTP/1.1" 200 None 2026-03-29 02:33:28,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:33:28,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:33:28,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:33:28,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ec6f2cb62322ad969044 HTTP/1.1" 200 None 2026-03-29 02:33:28,455 - lava_callback - ERROR - Error processing callback for node 69c5ec6f2cb62322ad969044: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52468 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:34:48,452 - lava_callback - INFO - Received callback for node 69c5ecc22cb62322ad969064 INFO: 172.20.0.1:52482 - "POST /node/69c5ecc22cb62322ad969064 HTTP/1.0" 202 Accepted 2026-03-29 02:34:48,457 - lava_callback - INFO - Starting async job submit for node 69c5ecc22cb62322ad969064 2026-03-29 02:34:48,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:34:48,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ecc22cb62322ad969064 HTTP/1.1" 200 None 2026-03-29 02:34:48,481 - lava_callback - DEBUG - Processing node 69c5ecc22cb62322ad969064: baseline-arm64-clabbe 2026-03-29 02:34:48,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5ecc22cb62322ad969064 2026-03-29 02:34:48,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:34:48,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ecc22cb62322ad969064 HTTP/1.1" 200 None 2026-03-29 02:34:48,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:34:48,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:34:48,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:34:48,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ecc22cb62322ad969064 HTTP/1.1" 200 None 2026-03-29 02:34:48,537 - lava_callback - ERROR - Error processing callback for node 69c5ecc22cb62322ad969064: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36112 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:36:28,507 - lava_callback - INFO - Received callback for node 69c5ed162cb62322ad969089 INFO: 172.20.0.1:36114 - "POST /node/69c5ed162cb62322ad969089 HTTP/1.0" 202 Accepted 2026-03-29 02:36:28,513 - lava_callback - INFO - Starting async job submit for node 69c5ed162cb62322ad969089 2026-03-29 02:36:28,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:36:28,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ed162cb62322ad969089 HTTP/1.1" 200 None 2026-03-29 02:36:28,530 - lava_callback - DEBUG - Processing node 69c5ed162cb62322ad969089: baseline-arm64-clabbe 2026-03-29 02:36:28,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5ed162cb62322ad969089 2026-03-29 02:36:28,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:36:28,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ed162cb62322ad969089 HTTP/1.1" 200 None 2026-03-29 02:36:28,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:36:28,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:36:28,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:36:28,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ed162cb62322ad969089 HTTP/1.1" 200 None 2026-03-29 02:36:28,610 - lava_callback - ERROR - Error processing callback for node 69c5ed162cb62322ad969089: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39530 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:39:08,625 - lava_callback - INFO - Received callback for node 69c5edbd2cb62322ad9690cd INFO: 172.20.0.1:39536 - "POST /node/69c5edbd2cb62322ad9690cd HTTP/1.0" 202 Accepted 2026-03-29 02:39:08,630 - lava_callback - INFO - Starting async job submit for node 69c5edbd2cb62322ad9690cd 2026-03-29 02:39:08,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:39:08,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5edbd2cb62322ad9690cd HTTP/1.1" 200 None 2026-03-29 02:39:08,669 - lava_callback - DEBUG - Processing node 69c5edbd2cb62322ad9690cd: baseline-arm64-clabbe 2026-03-29 02:39:08,670 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5edbd2cb62322ad9690cd 2026-03-29 02:39:08,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:39:08,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5edbd2cb62322ad9690cd HTTP/1.1" 200 None 2026-03-29 02:39:08,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:39:08,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-29 02:39:08,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:39:08,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5edbd2cb62322ad9690cd HTTP/1.1" 200 None 2026-03-29 02:39:08,730 - lava_callback - ERROR - Error processing callback for node 69c5edbd2cb62322ad9690cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59446 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 02:46:16,153 - lava_callback - INFO - Received callback for node 69c88dd8f41e4fb6a3785fbf INFO: 172.20.0.1:59450 - "POST /node/69c88dd8f41e4fb6a3785fbf HTTP/1.0" 202 Accepted 2026-03-29 02:46:16,162 - lava_callback - INFO - Starting async job submit for node 69c88dd8f41e4fb6a3785fbf 2026-03-29 02:46:16,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:46:16,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88dd8f41e4fb6a3785fbf HTTP/1.1" 200 None 2026-03-29 02:46:16,225 - lava_callback - DEBUG - Processing node 69c88dd8f41e4fb6a3785fbf: baseline-nfs-arm64 INFO: 172.20.0.1:59462 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:46:17,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:46:17,566 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:46:17,568 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c88dd8f41e4fb6a3785fbf/log.txt.gz 2026-03-29 02:46:18,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:46:18,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:46:18,485 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c88dd8f41e4fb6a3785fbf/lava-logs.yaml 2026-03-29 02:46:18,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 02:46:18,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 02:46:18,689 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c88dd8f41e4fb6a3785fbf/lava_callback.json.gz DEBUG: Setup failed for 69c88dd8f41e4fb6a3785fbf. Transitting job node result: incomplete -> fail 2026-03-29 02:46:18,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:46:18,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88dd8f41e4fb6a3785fbf HTTP/1.1" 200 None 2026-03-29 02:46:18,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:46:18,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 02:46:18,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:46:18,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88dd8f41e4fb6a3785fbf HTTP/1.1" 200 None 2026-03-29 02:46:18,941 - lava_callback - INFO - Completed processing callback for node 69c88dd8f41e4fb6a3785fbf 2026-03-29 02:46:21,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:46:21,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54212 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:47:29,074 - lava_callback - INFO - Received callback for node 69c5efb22cb62322ad96915d INFO: 172.20.0.1:54218 - "POST /node/69c5efb22cb62322ad96915d HTTP/1.0" 202 Accepted 2026-03-29 02:47:29,080 - lava_callback - INFO - Starting async job submit for node 69c5efb22cb62322ad96915d 2026-03-29 02:47:29,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:47:29,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5efb22cb62322ad96915d HTTP/1.1" 200 None 2026-03-29 02:47:29,097 - lava_callback - DEBUG - Processing node 69c5efb22cb62322ad96915d: baseline-arm-clabbe 2026-03-29 02:47:29,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5efb22cb62322ad96915d 2026-03-29 02:47:29,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:47:29,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5efb22cb62322ad96915d HTTP/1.1" 200 None 2026-03-29 02:47:29,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:47:29,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-29 02:47:29,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:47:29,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5efb22cb62322ad96915d HTTP/1.1" 200 None 2026-03-29 02:47:29,177 - lava_callback - ERROR - Error processing callback for node 69c5efb22cb62322ad96915d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47210 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:48:09,048 - lava_callback - INFO - Received callback for node 69c5efd72cb62322ad969168 INFO: 172.20.0.1:47212 - "POST /node/69c5efd72cb62322ad969168 HTTP/1.0" 202 Accepted 2026-03-29 02:48:09,054 - lava_callback - INFO - Starting async job submit for node 69c5efd72cb62322ad969168 2026-03-29 02:48:09,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:48:09,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5efd72cb62322ad969168 HTTP/1.1" 200 None 2026-03-29 02:48:09,070 - lava_callback - DEBUG - Processing node 69c5efd72cb62322ad969168: baseline-arm-clabbe 2026-03-29 02:48:09,070 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5efd72cb62322ad969168 2026-03-29 02:48:09,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:48:09,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5efd72cb62322ad969168 HTTP/1.1" 200 None 2026-03-29 02:48:09,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:48:09,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-29 02:48:09,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:48:09,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5efd72cb62322ad969168 HTTP/1.1" 200 None 2026-03-29 02:48:09,122 - lava_callback - ERROR - Error processing callback for node 69c5efd72cb62322ad969168: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41948 - "GET /node/6983f28c969566dcb0d2a89f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41956 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:50:09,047 - lava_callback - INFO - Received callback for node 69c5f04e2cb62322ad969184 INFO: 172.20.0.1:41972 - "POST /node/69c5f04e2cb62322ad969184 HTTP/1.0" 202 Accepted 2026-03-29 02:50:09,052 - lava_callback - INFO - Starting async job submit for node 69c5f04e2cb62322ad969184 2026-03-29 02:50:09,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:50:09,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f04e2cb62322ad969184 HTTP/1.1" 200 None 2026-03-29 02:50:09,069 - lava_callback - DEBUG - Processing node 69c5f04e2cb62322ad969184: baseline-arm-clabbe 2026-03-29 02:50:09,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5f04e2cb62322ad969184 2026-03-29 02:50:09,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:50:09,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f04e2cb62322ad969184 HTTP/1.1" 200 None 2026-03-29 02:50:09,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:50:09,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-29 02:50:09,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:50:09,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f04e2cb62322ad969184 HTTP/1.1" 200 None 2026-03-29 02:50:09,130 - lava_callback - ERROR - Error processing callback for node 69c5f04e2cb62322ad969184: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34940 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:50:49,142 - lava_callback - INFO - Received callback for node 69c5f0732cb62322ad969190 INFO: 172.20.0.1:34948 - "POST /node/69c5f0732cb62322ad969190 HTTP/1.0" 202 Accepted 2026-03-29 02:50:49,147 - lava_callback - INFO - Starting async job submit for node 69c5f0732cb62322ad969190 2026-03-29 02:50:49,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:50:49,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f0732cb62322ad969190 HTTP/1.1" 200 None 2026-03-29 02:50:49,164 - lava_callback - DEBUG - Processing node 69c5f0732cb62322ad969190: baseline-arm-clabbe 2026-03-29 02:50:49,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5f0732cb62322ad969190 2026-03-29 02:50:49,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:50:49,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f0732cb62322ad969190 HTTP/1.1" 200 None 2026-03-29 02:50:49,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:50:49,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-29 02:50:49,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:50:49,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f0732cb62322ad969190 HTTP/1.1" 200 None 2026-03-29 02:50:49,224 - lava_callback - ERROR - Error processing callback for node 69c5f0732cb62322ad969190: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36324 - "GET / HTTP/1.0" 200 OK 2026-03-29 02:51:29,200 - lava_callback - INFO - Received callback for node 69c5f0a92cb62322ad96919d INFO: 172.20.0.1:36336 - "POST /node/69c5f0a92cb62322ad96919d HTTP/1.0" 202 Accepted 2026-03-29 02:51:29,206 - lava_callback - INFO - Starting async job submit for node 69c5f0a92cb62322ad96919d 2026-03-29 02:51:29,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:51:29,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f0a92cb62322ad96919d HTTP/1.1" 200 None 2026-03-29 02:51:29,222 - lava_callback - DEBUG - Processing node 69c5f0a92cb62322ad96919d: baseline-arm-clabbe 2026-03-29 02:51:29,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5f0a92cb62322ad96919d 2026-03-29 02:51:29,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:51:29,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f0a92cb62322ad96919d HTTP/1.1" 200 None 2026-03-29 02:51:29,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:51:29,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-03-29 02:51:29,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 02:51:29,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f0a92cb62322ad96919d HTTP/1.1" 200 None 2026-03-29 02:51:29,278 - lava_callback - ERROR - Error processing callback for node 69c5f0a92cb62322ad96919d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47646 - "GET /node/698af5e83b3beb19e7616ef6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58172 - "GET /node/6949243febaed71d43807203%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47204 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:02:34,745 - lava_callback - INFO - Received callback for node 69c88289f41e4fb6a3785bbe INFO: 172.20.0.1:51174 - "POST /node/69c88289f41e4fb6a3785bbe HTTP/1.0" 202 Accepted 2026-03-29 03:02:34,751 - lava_callback - INFO - Starting async job submit for node 69c88289f41e4fb6a3785bbe 2026-03-29 03:02:34,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:02:34,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88289f41e4fb6a3785bbe HTTP/1.1" 200 None 2026-03-29 03:02:34,788 - lava_callback - DEBUG - Processing node 69c88289f41e4fb6a3785bbe: baseline-arm64 2026-03-29 03:02:35,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 03:02:35,188 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 03:02:35,190 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88289f41e4fb6a3785bbe/log.txt.gz 2026-03-29 03:02:35,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 03:02:35,537 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 03:02:35,539 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88289f41e4fb6a3785bbe/lava-logs.yaml 2026-03-29 03:02:35,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 03:02:35,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 03:02:35,701 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c88289f41e4fb6a3785bbe/lava_callback.json.gz 2026-03-29 03:02:35,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:02:35,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c88289f41e4fb6a3785bbe HTTP/1.1" 200 None 2026-03-29 03:02:35,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:02:35,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 03:02:35,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:02:35,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c88289f41e4fb6a3785bbe HTTP/1.1" 200 None 2026-03-29 03:02:35,786 - lava_callback - INFO - Completed processing callback for node 69c88289f41e4fb6a3785bbe INFO: 172.20.0.1:51186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52886 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:02:51,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:02:51,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46346 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:03:19,546 - lava_callback - INFO - Received callback for node 69c8829df41e4fb6a3785bf9 INFO: 172.20.0.1:46356 - "POST /node/69c8829df41e4fb6a3785bf9 HTTP/1.0" 202 Accepted 2026-03-29 03:03:19,552 - lava_callback - INFO - Starting async job submit for node 69c8829df41e4fb6a3785bf9 2026-03-29 03:03:19,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:03:19,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8829df41e4fb6a3785bf9 HTTP/1.1" 200 None 2026-03-29 03:03:19,590 - lava_callback - DEBUG - Processing node 69c8829df41e4fb6a3785bf9: baseline-nfs-arm64 2026-03-29 03:03:19,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 03:03:20,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 03:03:20,035 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8829df41e4fb6a3785bf9/log.txt.gz 2026-03-29 03:03:20,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 03:03:20,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 03:03:20,441 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8829df41e4fb6a3785bf9/lava-logs.yaml 2026-03-29 03:03:20,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 03:03:20,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 03:03:20,598 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c8829df41e4fb6a3785bf9/lava_callback.json.gz 2026-03-29 03:03:20,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:03:20,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c8829df41e4fb6a3785bf9 HTTP/1.1" 200 None 2026-03-29 03:03:20,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:03:20,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 03:03:20,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:03:20,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c8829df41e4fb6a3785bf9 HTTP/1.1" 200 None 2026-03-29 03:03:20,689 - lava_callback - INFO - Completed processing callback for node 69c8829df41e4fb6a3785bf9 2026-03-29 03:03:21,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:03:21,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45660 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:06:36,980 - lava_callback - INFO - Received callback for node 69c89298f41e4fb6a37860a3 INFO: 172.20.0.1:45672 - "POST /node/69c89298f41e4fb6a37860a3 HTTP/1.0" 202 Accepted 2026-03-29 03:06:36,988 - lava_callback - INFO - Starting async job submit for node 69c89298f41e4fb6a37860a3 2026-03-29 03:06:37,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:06:37,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89298f41e4fb6a37860a3 HTTP/1.1" 200 None 2026-03-29 03:06:37,049 - lava_callback - DEBUG - Processing node 69c89298f41e4fb6a37860a3: baseline-nfs-arm64 2026-03-29 03:06:38,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 03:06:38,397 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 03:06:38,399 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c89298f41e4fb6a37860a3/log.txt.gz 2026-03-29 03:06:39,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 03:06:39,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 03:06:39,321 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c89298f41e4fb6a37860a3/lava-logs.yaml 2026-03-29 03:06:39,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-29 03:06:39,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-29 03:06:39,545 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c89298f41e4fb6a37860a3/lava_callback.json.gz DEBUG: Setup failed for 69c89298f41e4fb6a37860a3. Transitting job node result: incomplete -> fail 2026-03-29 03:06:39,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:06:39,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c89298f41e4fb6a37860a3 HTTP/1.1" 200 None 2026-03-29 03:06:39,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:06:39,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-03-29 03:06:39,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:06:39,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c89298f41e4fb6a37860a3 HTTP/1.1" 200 None 2026-03-29 03:06:39,810 - lava_callback - INFO - Completed processing callback for node 69c89298f41e4fb6a37860a3 INFO: 172.20.0.1:45788 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:06:51,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:06:51,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37792 - "GET /node/690fc5748dbb8b8effa96acf%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53474 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:18:53,102 - lava_callback - INFO - Received callback for node 69c5f7102cb62322ad96923c INFO: 172.20.0.1:57810 - "POST /node/69c5f7102cb62322ad96923c HTTP/1.0" 202 Accepted 2026-03-29 03:18:53,108 - lava_callback - INFO - Starting async job submit for node 69c5f7102cb62322ad96923c 2026-03-29 03:18:53,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:18:53,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f7102cb62322ad96923c HTTP/1.1" 200 None 2026-03-29 03:18:53,124 - lava_callback - DEBUG - Processing node 69c5f7102cb62322ad96923c: baseline-arm64-baylibre 2026-03-29 03:18:53,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5f7102cb62322ad96923c 2026-03-29 03:18:53,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:18:53,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f7102cb62322ad96923c HTTP/1.1" 200 None 2026-03-29 03:18:53,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:18:53,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:18:53,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:18:53,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f7102cb62322ad96923c HTTP/1.1" 200 None 2026-03-29 03:18:53,190 - lava_callback - ERROR - Error processing callback for node 69c5f7102cb62322ad96923c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34806 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:20:12,926 - lava_callback - INFO - Received callback for node 69c5f7632cb62322ad969244 INFO: 172.20.0.1:45832 - "POST /node/69c5f7632cb62322ad969244 HTTP/1.0" 202 Accepted 2026-03-29 03:20:12,931 - lava_callback - INFO - Starting async job submit for node 69c5f7632cb62322ad969244 2026-03-29 03:20:12,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:20:12,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f7632cb62322ad969244 HTTP/1.1" 200 None 2026-03-29 03:20:12,950 - lava_callback - DEBUG - Processing node 69c5f7632cb62322ad969244: baseline-arm64-baylibre 2026-03-29 03:20:12,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5f7632cb62322ad969244 2026-03-29 03:20:12,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:20:12,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f7632cb62322ad969244 HTTP/1.1" 200 None 2026-03-29 03:20:12,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:20:12,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:20:12,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:20:13,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f7632cb62322ad969244 HTTP/1.1" 200 None 2026-03-29 03:20:13,026 - lava_callback - ERROR - Error processing callback for node 69c5f7632cb62322ad969244: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39312 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:24:53,287 - lava_callback - INFO - Received callback for node 69c5f8702cb62322ad969255 INFO: 172.20.0.1:39116 - "POST /node/69c5f8702cb62322ad969255 HTTP/1.0" 202 Accepted 2026-03-29 03:24:53,293 - lava_callback - INFO - Starting async job submit for node 69c5f8702cb62322ad969255 2026-03-29 03:24:53,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:24:53,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f8702cb62322ad969255 HTTP/1.1" 200 None 2026-03-29 03:24:53,317 - lava_callback - DEBUG - Processing node 69c5f8702cb62322ad969255: baseline-arm64-baylibre 2026-03-29 03:24:53,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5f8702cb62322ad969255 2026-03-29 03:24:53,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:24:53,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f8702cb62322ad969255 HTTP/1.1" 200 None 2026-03-29 03:24:53,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:24:53,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:24:53,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:24:53,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f8702cb62322ad969255 HTTP/1.1" 200 None 2026-03-29 03:24:53,426 - lava_callback - ERROR - Error processing callback for node 69c5f8702cb62322ad969255: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46986 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:26:13,130 - lava_callback - INFO - Received callback for node 69c5f8c42cb62322ad969261 INFO: 172.20.0.1:58510 - "POST /node/69c5f8c42cb62322ad969261 HTTP/1.0" 202 Accepted 2026-03-29 03:26:13,136 - lava_callback - INFO - Starting async job submit for node 69c5f8c42cb62322ad969261 2026-03-29 03:26:13,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:26:13,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f8c42cb62322ad969261 HTTP/1.1" 200 None 2026-03-29 03:26:13,152 - lava_callback - DEBUG - Processing node 69c5f8c42cb62322ad969261: baseline-arm64-baylibre 2026-03-29 03:26:13,152 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5f8c42cb62322ad969261 2026-03-29 03:26:13,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:26:13,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f8c42cb62322ad969261 HTTP/1.1" 200 None 2026-03-29 03:26:13,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:26:13,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:26:13,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:26:13,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f8c42cb62322ad969261 HTTP/1.1" 200 None 2026-03-29 03:26:13,209 - lava_callback - ERROR - Error processing callback for node 69c5f8c42cb62322ad969261: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36656 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:27:33,198 - lava_callback - INFO - Received callback for node 69c5f9172cb62322ad96926d INFO: 172.20.0.1:48500 - "POST /node/69c5f9172cb62322ad96926d HTTP/1.0" 202 Accepted 2026-03-29 03:27:33,204 - lava_callback - INFO - Starting async job submit for node 69c5f9172cb62322ad96926d 2026-03-29 03:27:33,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:27:33,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f9172cb62322ad96926d HTTP/1.1" 200 None 2026-03-29 03:27:33,220 - lava_callback - DEBUG - Processing node 69c5f9172cb62322ad96926d: baseline-arm64-baylibre 2026-03-29 03:27:33,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5f9172cb62322ad96926d 2026-03-29 03:27:33,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:27:33,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f9172cb62322ad96926d HTTP/1.1" 200 None 2026-03-29 03:27:33,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:27:33,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:27:33,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:27:33,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f9172cb62322ad96926d HTTP/1.1" 200 None 2026-03-29 03:27:33,298 - lava_callback - ERROR - Error processing callback for node 69c5f9172cb62322ad96926d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35744 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:28:53,245 - lava_callback - INFO - Received callback for node 69c5f96f2cb62322ad96927b INFO: 172.20.0.1:44342 - "POST /node/69c5f96f2cb62322ad96927b HTTP/1.0" 202 Accepted 2026-03-29 03:28:53,251 - lava_callback - INFO - Starting async job submit for node 69c5f96f2cb62322ad96927b 2026-03-29 03:28:53,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:28:53,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f96f2cb62322ad96927b HTTP/1.1" 200 None 2026-03-29 03:28:53,267 - lava_callback - DEBUG - Processing node 69c5f96f2cb62322ad96927b: baseline-arm64-baylibre 2026-03-29 03:28:53,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5f96f2cb62322ad96927b 2026-03-29 03:28:53,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:28:53,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f96f2cb62322ad96927b HTTP/1.1" 200 None 2026-03-29 03:28:53,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:28:53,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:28:53,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:28:53,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f96f2cb62322ad96927b HTTP/1.1" 200 None 2026-03-29 03:28:53,336 - lava_callback - ERROR - Error processing callback for node 69c5f96f2cb62322ad96927b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49332 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:30:30,763 - lava_callback - INFO - Received callback for node 69c5f9c92cb62322ad969289 INFO: 172.20.0.1:39928 - "POST /node/69c5f9c92cb62322ad969289 HTTP/1.0" 202 Accepted 2026-03-29 03:30:30,769 - lava_callback - INFO - Starting async job submit for node 69c5f9c92cb62322ad969289 2026-03-29 03:30:30,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:30:30,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f9c92cb62322ad969289 HTTP/1.1" 200 None 2026-03-29 03:30:30,788 - lava_callback - DEBUG - Processing node 69c5f9c92cb62322ad969289: baseline-arm64-clabbe 2026-03-29 03:30:30,788 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5f9c92cb62322ad969289 2026-03-29 03:30:30,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:30:30,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f9c92cb62322ad969289 HTTP/1.1" 200 None 2026-03-29 03:30:30,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:30:30,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:30:30,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:30:30,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f9c92cb62322ad969289 HTTP/1.1" 200 None 2026-03-29 03:30:30,898 - lava_callback - ERROR - Error processing callback for node 69c5f9c92cb62322ad969289: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58336 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:31:50,804 - lava_callback - INFO - Received callback for node 69c5fa1e2cb62322ad969296 INFO: 172.20.0.1:37438 - "POST /node/69c5fa1e2cb62322ad969296 HTTP/1.0" 202 Accepted 2026-03-29 03:31:50,810 - lava_callback - INFO - Starting async job submit for node 69c5fa1e2cb62322ad969296 2026-03-29 03:31:50,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:31:50,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fa1e2cb62322ad969296 HTTP/1.1" 200 None 2026-03-29 03:31:50,855 - lava_callback - DEBUG - Processing node 69c5fa1e2cb62322ad969296: baseline-arm64-clabbe 2026-03-29 03:31:50,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5fa1e2cb62322ad969296 2026-03-29 03:31:50,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:31:50,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fa1e2cb62322ad969296 HTTP/1.1" 200 None 2026-03-29 03:31:50,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:31:50,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:31:50,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:31:50,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5fa1e2cb62322ad969296 HTTP/1.1" 200 None 2026-03-29 03:31:50,913 - lava_callback - ERROR - Error processing callback for node 69c5fa1e2cb62322ad969296: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37574 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:35:50,876 - lava_callback - INFO - Received callback for node 69c5fb0d2cb62322ad9692b9 INFO: 172.20.0.1:42538 - "POST /node/69c5fb0d2cb62322ad9692b9 HTTP/1.0" 202 Accepted 2026-03-29 03:35:50,882 - lava_callback - INFO - Starting async job submit for node 69c5fb0d2cb62322ad9692b9 2026-03-29 03:35:50,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:35:50,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fb0d2cb62322ad9692b9 HTTP/1.1" 200 None 2026-03-29 03:35:50,898 - lava_callback - DEBUG - Processing node 69c5fb0d2cb62322ad9692b9: baseline-arm64-clabbe 2026-03-29 03:35:50,898 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5fb0d2cb62322ad9692b9 2026-03-29 03:35:50,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:35:50,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fb0d2cb62322ad9692b9 HTTP/1.1" 200 None 2026-03-29 03:35:50,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:35:50,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:35:50,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:35:50,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5fb0d2cb62322ad9692b9 HTTP/1.1" 200 None 2026-03-29 03:35:50,956 - lava_callback - ERROR - Error processing callback for node 69c5fb0d2cb62322ad9692b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54184 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:38:31,019 - lava_callback - INFO - Received callback for node 69c5fbb22cb62322ad9692cd INFO: 172.20.0.1:48600 - "POST /node/69c5fbb22cb62322ad9692cd HTTP/1.0" 202 Accepted 2026-03-29 03:38:31,024 - lava_callback - INFO - Starting async job submit for node 69c5fbb22cb62322ad9692cd 2026-03-29 03:38:31,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:38:31,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fbb22cb62322ad9692cd HTTP/1.1" 200 None 2026-03-29 03:38:31,052 - lava_callback - DEBUG - Processing node 69c5fbb22cb62322ad9692cd: baseline-arm64-clabbe 2026-03-29 03:38:31,053 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5fbb22cb62322ad9692cd 2026-03-29 03:38:31,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:38:31,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fbb22cb62322ad9692cd HTTP/1.1" 200 None 2026-03-29 03:38:31,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:38:31,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:38:31,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:38:31,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5fbb22cb62322ad9692cd HTTP/1.1" 200 None 2026-03-29 03:38:31,168 - lava_callback - ERROR - Error processing callback for node 69c5fbb22cb62322ad9692cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33564 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:40:11,066 - lava_callback - INFO - Received callback for node 69c5fc072cb62322ad9692da INFO: 172.20.0.1:58910 - "POST /node/69c5fc072cb62322ad9692da HTTP/1.0" 202 Accepted 2026-03-29 03:40:11,071 - lava_callback - INFO - Starting async job submit for node 69c5fc072cb62322ad9692da 2026-03-29 03:40:11,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:40:11,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fc072cb62322ad9692da HTTP/1.1" 200 None 2026-03-29 03:40:11,087 - lava_callback - DEBUG - Processing node 69c5fc072cb62322ad9692da: baseline-arm64-clabbe 2026-03-29 03:40:11,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5fc072cb62322ad9692da 2026-03-29 03:40:11,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:40:11,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fc072cb62322ad9692da HTTP/1.1" 200 None 2026-03-29 03:40:11,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:40:11,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:40:11,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:40:11,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5fc072cb62322ad9692da HTTP/1.1" 200 None 2026-03-29 03:40:11,200 - lava_callback - ERROR - Error processing callback for node 69c5fc072cb62322ad9692da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49796 - "GET / HTTP/1.0" 200 OK 2026-03-29 03:41:51,173 - lava_callback - INFO - Received callback for node 69c5fc762cb62322ad9692ff INFO: 172.20.0.1:38214 - "POST /node/69c5fc762cb62322ad9692ff HTTP/1.0" 202 Accepted 2026-03-29 03:41:51,179 - lava_callback - INFO - Starting async job submit for node 69c5fc762cb62322ad9692ff 2026-03-29 03:41:51,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:41:51,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fc762cb62322ad9692ff HTTP/1.1" 200 None 2026-03-29 03:41:51,215 - lava_callback - DEBUG - Processing node 69c5fc762cb62322ad9692ff: baseline-arm64-clabbe 2026-03-29 03:41:51,215 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c5fc762cb62322ad9692ff 2026-03-29 03:41:51,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:41:51,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fc762cb62322ad9692ff HTTP/1.1" 200 None 2026-03-29 03:41:51,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:41:51,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-29 03:41:51,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 03:41:51,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5fc762cb62322ad9692ff HTTP/1.1" 200 None 2026-03-29 03:41:51,309 - lava_callback - ERROR - Error processing callback for node 69c5fc762cb62322ad9692ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45088 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:43:37,399 - lava_callback - INFO - Received callback for node 69c60af02cb62322ad96933b INFO: 172.20.0.1:45090 - "POST /node/69c60af02cb62322ad96933b HTTP/1.0" 202 Accepted 2026-03-29 04:43:37,405 - lava_callback - INFO - Starting async job submit for node 69c60af02cb62322ad96933b 2026-03-29 04:43:37,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:43:37,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60af02cb62322ad96933b HTTP/1.1" 200 None 2026-03-29 04:43:37,445 - lava_callback - DEBUG - Processing node 69c60af02cb62322ad96933b: baseline-arm-baylibre 2026-03-29 04:43:37,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60af02cb62322ad96933b 2026-03-29 04:43:37,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:43:37,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60af02cb62322ad96933b HTTP/1.1" 200 None 2026-03-29 04:43:37,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:43:37,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:43:37,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:43:37,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60af02cb62322ad96933b HTTP/1.1" 200 None 2026-03-29 04:43:37,512 - lava_callback - ERROR - Error processing callback for node 69c60af02cb62322ad96933b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38128 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:44:17,282 - lava_callback - INFO - Received callback for node 69c60b162cb62322ad969347 INFO: 172.20.0.1:38140 - "POST /node/69c60b162cb62322ad969347 HTTP/1.0" 202 Accepted 2026-03-29 04:44:17,288 - lava_callback - INFO - Starting async job submit for node 69c60b162cb62322ad969347 2026-03-29 04:44:17,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:44:17,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60b162cb62322ad969347 HTTP/1.1" 200 None 2026-03-29 04:44:17,304 - lava_callback - DEBUG - Processing node 69c60b162cb62322ad969347: baseline-arm-baylibre 2026-03-29 04:44:17,304 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60b162cb62322ad969347 2026-03-29 04:44:17,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:44:17,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60b162cb62322ad969347 HTTP/1.1" 200 None 2026-03-29 04:44:17,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:44:17,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:44:17,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:44:17,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60b162cb62322ad969347 HTTP/1.1" 200 None 2026-03-29 04:44:17,364 - lava_callback - ERROR - Error processing callback for node 69c60b162cb62322ad969347: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58208 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:44:57,386 - lava_callback - INFO - Received callback for node 69c60b3b2cb62322ad969354 INFO: 172.20.0.1:58216 - "POST /node/69c60b3b2cb62322ad969354 HTTP/1.0" 202 Accepted 2026-03-29 04:44:57,392 - lava_callback - INFO - Starting async job submit for node 69c60b3b2cb62322ad969354 2026-03-29 04:44:57,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:44:57,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60b3b2cb62322ad969354 HTTP/1.1" 200 None 2026-03-29 04:44:57,416 - lava_callback - DEBUG - Processing node 69c60b3b2cb62322ad969354: baseline-arm-baylibre 2026-03-29 04:44:57,416 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60b3b2cb62322ad969354 2026-03-29 04:44:57,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:44:57,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60b3b2cb62322ad969354 HTTP/1.1" 200 None 2026-03-29 04:44:57,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:44:57,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:44:57,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:44:57,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60b3b2cb62322ad969354 HTTP/1.1" 200 None 2026-03-29 04:44:57,469 - lava_callback - ERROR - Error processing callback for node 69c60b3b2cb62322ad969354: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55888 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:45:37,344 - lava_callback - INFO - Received callback for node 69c60b5f2cb62322ad96935f INFO: 172.20.0.1:55894 - "POST /node/69c60b5f2cb62322ad96935f HTTP/1.0" 202 Accepted 2026-03-29 04:45:37,349 - lava_callback - INFO - Starting async job submit for node 69c60b5f2cb62322ad96935f 2026-03-29 04:45:37,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:45:37,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60b5f2cb62322ad96935f HTTP/1.1" 200 None 2026-03-29 04:45:37,365 - lava_callback - DEBUG - Processing node 69c60b5f2cb62322ad96935f: baseline-arm-baylibre 2026-03-29 04:45:37,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60b5f2cb62322ad96935f 2026-03-29 04:45:37,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:45:37,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60b5f2cb62322ad96935f HTTP/1.1" 200 None 2026-03-29 04:45:37,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:45:37,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:45:37,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:45:37,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60b5f2cb62322ad96935f HTTP/1.1" 200 None 2026-03-29 04:45:37,433 - lava_callback - ERROR - Error processing callback for node 69c60b5f2cb62322ad96935f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58208 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:45:57,354 - lava_callback - INFO - Received callback for node 69c60b842cb62322ad96936a INFO: 172.20.0.1:58220 - "POST /node/69c60b842cb62322ad96936a HTTP/1.0" 202 Accepted 2026-03-29 04:45:57,361 - lava_callback - INFO - Starting async job submit for node 69c60b842cb62322ad96936a 2026-03-29 04:45:57,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:45:57,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60b842cb62322ad96936a HTTP/1.1" 200 None 2026-03-29 04:45:57,376 - lava_callback - DEBUG - Processing node 69c60b842cb62322ad96936a: baseline-arm-baylibre 2026-03-29 04:45:57,377 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60b842cb62322ad96936a 2026-03-29 04:45:57,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:45:57,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60b842cb62322ad96936a HTTP/1.1" 200 None 2026-03-29 04:45:57,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:45:57,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:45:57,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:45:57,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60b842cb62322ad96936a HTTP/1.1" 200 None 2026-03-29 04:45:57,451 - lava_callback - ERROR - Error processing callback for node 69c60b842cb62322ad96936a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59150 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:46:37,448 - lava_callback - INFO - Received callback for node 69c60ba82cb62322ad96937a INFO: 172.20.0.1:59156 - "POST /node/69c60ba82cb62322ad96937a HTTP/1.0" 202 Accepted 2026-03-29 04:46:37,454 - lava_callback - INFO - Starting async job submit for node 69c60ba82cb62322ad96937a 2026-03-29 04:46:37,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:46:37,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60ba82cb62322ad96937a HTTP/1.1" 200 None 2026-03-29 04:46:37,469 - lava_callback - DEBUG - Processing node 69c60ba82cb62322ad96937a: baseline-arm-baylibre 2026-03-29 04:46:37,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60ba82cb62322ad96937a 2026-03-29 04:46:37,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:46:37,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60ba82cb62322ad96937a HTTP/1.1" 200 None 2026-03-29 04:46:37,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:46:37,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:46:37,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:46:37,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60ba82cb62322ad96937a HTTP/1.1" 200 None 2026-03-29 04:46:37,524 - lava_callback - ERROR - Error processing callback for node 69c60ba82cb62322ad96937a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37198 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:47:17,433 - lava_callback - INFO - Received callback for node 69c60bce2cb62322ad969386 INFO: 172.20.0.1:37210 - "POST /node/69c60bce2cb62322ad969386 HTTP/1.0" 202 Accepted 2026-03-29 04:47:17,439 - lava_callback - INFO - Starting async job submit for node 69c60bce2cb62322ad969386 2026-03-29 04:47:17,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:47:17,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60bce2cb62322ad969386 HTTP/1.1" 200 None 2026-03-29 04:47:17,455 - lava_callback - DEBUG - Processing node 69c60bce2cb62322ad969386: baseline-arm-baylibre 2026-03-29 04:47:17,455 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60bce2cb62322ad969386 2026-03-29 04:47:17,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:47:17,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60bce2cb62322ad969386 HTTP/1.1" 200 None 2026-03-29 04:47:17,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:47:17,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:47:17,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:47:17,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60bce2cb62322ad969386 HTTP/1.1" 200 None 2026-03-29 04:47:17,516 - lava_callback - ERROR - Error processing callback for node 69c60bce2cb62322ad969386: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48834 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:47:57,512 - lava_callback - INFO - Received callback for node 69c60bf02cb62322ad969392 INFO: 172.20.0.1:48836 - "POST /node/69c60bf02cb62322ad969392 HTTP/1.0" 202 Accepted 2026-03-29 04:47:57,517 - lava_callback - INFO - Starting async job submit for node 69c60bf02cb62322ad969392 2026-03-29 04:47:57,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:47:57,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60bf02cb62322ad969392 HTTP/1.1" 200 None 2026-03-29 04:47:57,533 - lava_callback - DEBUG - Processing node 69c60bf02cb62322ad969392: baseline-arm-baylibre 2026-03-29 04:47:57,533 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60bf02cb62322ad969392 2026-03-29 04:47:57,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:47:57,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60bf02cb62322ad969392 HTTP/1.1" 200 None 2026-03-29 04:47:57,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:47:57,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:47:57,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:47:57,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60bf02cb62322ad969392 HTTP/1.1" 200 None 2026-03-29 04:47:57,593 - lava_callback - ERROR - Error processing callback for node 69c60bf02cb62322ad969392: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48418 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:49:37,879 - lava_callback - INFO - Received callback for node 69c60c592cb62322ad9693aa INFO: 172.20.0.1:48434 - "POST /node/69c60c592cb62322ad9693aa HTTP/1.0" 202 Accepted 2026-03-29 04:49:37,885 - lava_callback - INFO - Starting async job submit for node 69c60c592cb62322ad9693aa 2026-03-29 04:49:37,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:49:37,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60c592cb62322ad9693aa HTTP/1.1" 200 None 2026-03-29 04:49:37,932 - lava_callback - DEBUG - Processing node 69c60c592cb62322ad9693aa: baseline-arm-baylibre 2026-03-29 04:49:37,932 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60c592cb62322ad9693aa 2026-03-29 04:49:37,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:49:37,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60c592cb62322ad9693aa HTTP/1.1" 200 None 2026-03-29 04:49:37,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:49:37,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:49:37,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:49:37,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60c592cb62322ad9693aa HTTP/1.1" 200 None 2026-03-29 04:49:37,996 - lava_callback - ERROR - Error processing callback for node 69c60c592cb62322ad9693aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33046 - "GET /node/698a99d4d9a1cfa777a24879%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:33052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33056 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:50:17,621 - lava_callback - INFO - Received callback for node 69c60c7c2cb62322ad9693b8 INFO: 172.20.0.1:33070 - "POST /node/69c60c7c2cb62322ad9693b8 HTTP/1.0" 202 Accepted 2026-03-29 04:50:17,626 - lava_callback - INFO - Starting async job submit for node 69c60c7c2cb62322ad9693b8 2026-03-29 04:50:17,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:50:17,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60c7c2cb62322ad9693b8 HTTP/1.1" 200 None 2026-03-29 04:50:17,651 - lava_callback - DEBUG - Processing node 69c60c7c2cb62322ad9693b8: baseline-arm-baylibre 2026-03-29 04:50:17,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60c7c2cb62322ad9693b8 2026-03-29 04:50:17,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:50:17,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60c7c2cb62322ad9693b8 HTTP/1.1" 200 None 2026-03-29 04:50:17,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:50:17,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:50:17,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:50:17,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60c7c2cb62322ad9693b8 HTTP/1.1" 200 None 2026-03-29 04:50:17,724 - lava_callback - ERROR - Error processing callback for node 69c60c7c2cb62322ad9693b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51192 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:50:57,668 - lava_callback - INFO - Received callback for node 69c60ca02cb62322ad9693c9 INFO: 172.20.0.1:51196 - "POST /node/69c60ca02cb62322ad9693c9 HTTP/1.0" 202 Accepted 2026-03-29 04:50:57,673 - lava_callback - INFO - Starting async job submit for node 69c60ca02cb62322ad9693c9 2026-03-29 04:50:57,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:50:57,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60ca02cb62322ad9693c9 HTTP/1.1" 200 None 2026-03-29 04:50:57,690 - lava_callback - DEBUG - Processing node 69c60ca02cb62322ad9693c9: baseline-arm-baylibre 2026-03-29 04:50:57,690 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60ca02cb62322ad9693c9 2026-03-29 04:50:57,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:50:57,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60ca02cb62322ad9693c9 HTTP/1.1" 200 None 2026-03-29 04:50:57,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:50:57,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:50:57,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:50:57,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60ca02cb62322ad9693c9 HTTP/1.1" 200 None 2026-03-29 04:50:57,756 - lava_callback - ERROR - Error processing callback for node 69c60ca02cb62322ad9693c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48470 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:51:37,661 - lava_callback - INFO - Received callback for node 69c60cc52cb62322ad9693d4 INFO: 172.20.0.1:48474 - "POST /node/69c60cc52cb62322ad9693d4 HTTP/1.0" 202 Accepted 2026-03-29 04:51:37,666 - lava_callback - INFO - Starting async job submit for node 69c60cc52cb62322ad9693d4 2026-03-29 04:51:37,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:51:37,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60cc52cb62322ad9693d4 HTTP/1.1" 200 None 2026-03-29 04:51:37,696 - lava_callback - DEBUG - Processing node 69c60cc52cb62322ad9693d4: baseline-arm-baylibre 2026-03-29 04:51:37,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60cc52cb62322ad9693d4 2026-03-29 04:51:37,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:51:37,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60cc52cb62322ad9693d4 HTTP/1.1" 200 None 2026-03-29 04:51:37,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:51:37,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:51:37,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:51:37,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60cc52cb62322ad9693d4 HTTP/1.1" 200 None 2026-03-29 04:51:37,752 - lava_callback - ERROR - Error processing callback for node 69c60cc52cb62322ad9693d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40708 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:52:14,140 - lava_callback - INFO - Received callback for node 69c60cec2cb62322ad9693ee INFO: 172.20.0.1:38276 - "POST /node/69c60cec2cb62322ad9693ee HTTP/1.0" 202 Accepted 2026-03-29 04:52:14,145 - lava_callback - INFO - Starting async job submit for node 69c60cec2cb62322ad9693ee 2026-03-29 04:52:14,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:52:14,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60cec2cb62322ad9693ee HTTP/1.1" 200 None 2026-03-29 04:52:14,161 - lava_callback - DEBUG - Processing node 69c60cec2cb62322ad9693ee: baseline-arm-clabbe 2026-03-29 04:52:14,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60cec2cb62322ad9693ee 2026-03-29 04:52:14,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:52:14,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60cec2cb62322ad9693ee HTTP/1.1" 200 None 2026-03-29 04:52:14,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:52:14,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:52:14,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:52:14,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60cec2cb62322ad9693ee HTTP/1.1" 200 None 2026-03-29 04:52:14,239 - lava_callback - ERROR - Error processing callback for node 69c60cec2cb62322ad9693ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47164 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:52:37,787 - lava_callback - INFO - Received callback for node 69c60d0f2cb62322ad969423 INFO: 172.20.0.1:47168 - "POST /node/69c60d0f2cb62322ad969423 HTTP/1.0" 202 Accepted 2026-03-29 04:52:37,793 - lava_callback - INFO - Starting async job submit for node 69c60d0f2cb62322ad969423 2026-03-29 04:52:37,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:52:37,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60d0f2cb62322ad969423 HTTP/1.1" 200 None 2026-03-29 04:52:37,827 - lava_callback - DEBUG - Processing node 69c60d0f2cb62322ad969423: baseline-arm-baylibre 2026-03-29 04:52:37,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60d0f2cb62322ad969423 2026-03-29 04:52:37,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:52:38,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60d0f2cb62322ad969423 HTTP/1.1" 200 None 2026-03-29 04:52:38,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:52:38,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:52:38,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:52:38,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60d0f2cb62322ad969423 HTTP/1.1" 200 None 2026-03-29 04:52:38,083 - lava_callback - ERROR - Error processing callback for node 69c60d0f2cb62322ad969423: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44682 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:53:14,127 - lava_callback - INFO - Received callback for node 69c60d352cb62322ad969436 INFO: 172.20.0.1:50704 - "POST /node/69c60d352cb62322ad969436 HTTP/1.0" 202 Accepted 2026-03-29 04:53:14,132 - lava_callback - INFO - Starting async job submit for node 69c60d352cb62322ad969436 2026-03-29 04:53:14,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:53:14,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60d352cb62322ad969436 HTTP/1.1" 200 None 2026-03-29 04:53:14,152 - lava_callback - DEBUG - Processing node 69c60d352cb62322ad969436: baseline-arm-clabbe 2026-03-29 04:53:14,152 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60d352cb62322ad969436 2026-03-29 04:53:14,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:53:14,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60d352cb62322ad969436 HTTP/1.1" 200 None 2026-03-29 04:53:14,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:53:14,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:53:14,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:53:14,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60d352cb62322ad969436 HTTP/1.1" 200 None 2026-03-29 04:53:14,228 - lava_callback - ERROR - Error processing callback for node 69c60d352cb62322ad969436: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58610 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:53:57,865 - lava_callback - INFO - Received callback for node 69c60d582cb62322ad969447 INFO: 172.20.0.1:58626 - "POST /node/69c60d582cb62322ad969447 HTTP/1.0" 202 Accepted 2026-03-29 04:53:57,870 - lava_callback - INFO - Starting async job submit for node 69c60d582cb62322ad969447 2026-03-29 04:53:57,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:53:57,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60d582cb62322ad969447 HTTP/1.1" 200 None 2026-03-29 04:53:57,887 - lava_callback - DEBUG - Processing node 69c60d582cb62322ad969447: baseline-arm-baylibre 2026-03-29 04:53:57,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60d582cb62322ad969447 2026-03-29 04:53:57,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:53:57,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60d582cb62322ad969447 HTTP/1.1" 200 None 2026-03-29 04:53:57,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:53:57,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:53:57,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:53:57,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60d582cb62322ad969447 HTTP/1.1" 200 None 2026-03-29 04:53:57,949 - lava_callback - ERROR - Error processing callback for node 69c60d582cb62322ad969447: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55892 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:54:37,856 - lava_callback - INFO - Received callback for node 69c60d7d2cb62322ad969455 INFO: 172.20.0.1:55894 - "POST /node/69c60d7d2cb62322ad969455 HTTP/1.0" 202 Accepted 2026-03-29 04:54:37,861 - lava_callback - INFO - Starting async job submit for node 69c60d7d2cb62322ad969455 2026-03-29 04:54:37,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:54:37,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60d7d2cb62322ad969455 HTTP/1.1" 200 None 2026-03-29 04:54:37,877 - lava_callback - DEBUG - Processing node 69c60d7d2cb62322ad969455: baseline-arm-baylibre 2026-03-29 04:54:37,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60d7d2cb62322ad969455 2026-03-29 04:54:37,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:54:37,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60d7d2cb62322ad969455 HTTP/1.1" 200 None 2026-03-29 04:54:37,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:54:37,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:54:37,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:54:37,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60d7d2cb62322ad969455 HTTP/1.1" 200 None 2026-03-29 04:54:37,928 - lava_callback - ERROR - Error processing callback for node 69c60d7d2cb62322ad969455: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60626 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:55:14,200 - lava_callback - INFO - Received callback for node 69c60da62cb62322ad969469 INFO: 172.20.0.1:57620 - "POST /node/69c60da62cb62322ad969469 HTTP/1.0" 202 Accepted 2026-03-29 04:55:14,206 - lava_callback - INFO - Starting async job submit for node 69c60da62cb62322ad969469 2026-03-29 04:55:14,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:55:14,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60da62cb62322ad969469 HTTP/1.1" 200 None 2026-03-29 04:55:14,227 - lava_callback - DEBUG - Processing node 69c60da62cb62322ad969469: baseline-arm-clabbe 2026-03-29 04:55:14,227 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60da62cb62322ad969469 2026-03-29 04:55:14,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:55:14,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60da62cb62322ad969469 HTTP/1.1" 200 None 2026-03-29 04:55:14,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:55:14,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:55:14,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:55:14,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60da62cb62322ad969469 HTTP/1.1" 200 None 2026-03-29 04:55:14,302 - lava_callback - ERROR - Error processing callback for node 69c60da62cb62322ad969469: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41980 - "GET / HTTP/1.0" 200 OK 2026-03-29 04:55:54,217 - lava_callback - INFO - Received callback for node 69c60dcb2cb62322ad969477 INFO: 172.20.0.1:50216 - "POST /node/69c60dcb2cb62322ad969477 HTTP/1.0" 202 Accepted 2026-03-29 04:55:54,223 - lava_callback - INFO - Starting async job submit for node 69c60dcb2cb62322ad969477 2026-03-29 04:55:54,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:55:54,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60dcb2cb62322ad969477 HTTP/1.1" 200 None 2026-03-29 04:55:54,268 - lava_callback - DEBUG - Processing node 69c60dcb2cb62322ad969477: baseline-arm-clabbe 2026-03-29 04:55:54,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60dcb2cb62322ad969477 2026-03-29 04:55:54,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:55:54,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60dcb2cb62322ad969477 HTTP/1.1" 200 None 2026-03-29 04:55:54,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:55:54,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 04:55:54,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 04:55:54,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60dcb2cb62322ad969477 HTTP/1.1" 200 None 2026-03-29 04:55:54,348 - lava_callback - ERROR - Error processing callback for node 69c60dcb2cb62322ad969477: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41270 - "GET /node/69936ab2cebe99dd8e067176%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52780 - "GET /node/6993c6cb3e9ba4930770ebde%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52782 - "GET /node/69b913120ec88e58bad0d8b0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49372 - "GET /node/694bc94fc19e09af58d46e68%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-29 05:01:54,537 - lava_callback - INFO - Received callback for node 69c60f382cb62322ad9694ca INFO: 172.20.0.1:49378 - "POST /node/69c60f382cb62322ad9694ca HTTP/1.0" 202 Accepted 2026-03-29 05:01:54,542 - lava_callback - INFO - Starting async job submit for node 69c60f382cb62322ad9694ca 2026-03-29 05:01:54,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:01:54,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60f382cb62322ad9694ca HTTP/1.1" 200 None 2026-03-29 05:01:54,559 - lava_callback - DEBUG - Processing node 69c60f382cb62322ad9694ca: baseline-arm-clabbe 2026-03-29 05:01:54,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c60f382cb62322ad9694ca 2026-03-29 05:01:54,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:01:54,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60f382cb62322ad9694ca HTTP/1.1" 200 None 2026-03-29 05:01:54,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:01:54,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:01:54,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:01:54,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60f382cb62322ad9694ca HTTP/1.1" 200 None 2026-03-29 05:01:54,619 - lava_callback - ERROR - Error processing callback for node 69c60f382cb62322ad9694ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53592 - "GET /node/6949e6ec2060d3d84791d128%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53616 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:05:18,658 - lava_callback - INFO - Received callback for node 69c610092cb62322ad9694fd INFO: 172.20.0.1:53622 - "POST /node/69c610092cb62322ad9694fd HTTP/1.0" 202 Accepted 2026-03-29 05:05:18,664 - lava_callback - INFO - Starting async job submit for node 69c610092cb62322ad9694fd 2026-03-29 05:05:18,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:05:18,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c610092cb62322ad9694fd HTTP/1.1" 200 None 2026-03-29 05:05:18,679 - lava_callback - DEBUG - Processing node 69c610092cb62322ad9694fd: baseline-arm64-baylibre 2026-03-29 05:05:18,679 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c610092cb62322ad9694fd 2026-03-29 05:05:18,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:05:18,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c610092cb62322ad9694fd HTTP/1.1" 200 None 2026-03-29 05:05:18,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:05:18,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 05:05:18,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:05:18,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c610092cb62322ad9694fd HTTP/1.1" 200 None 2026-03-29 05:05:18,729 - lava_callback - ERROR - Error processing callback for node 69c610092cb62322ad9694fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35994 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:06:38,564 - lava_callback - INFO - Received callback for node 69c6105b2cb62322ad96950f INFO: 172.20.0.1:36002 - "POST /node/69c6105b2cb62322ad96950f HTTP/1.0" 202 Accepted 2026-03-29 05:06:38,570 - lava_callback - INFO - Starting async job submit for node 69c6105b2cb62322ad96950f 2026-03-29 05:06:38,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:06:38,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6105b2cb62322ad96950f HTTP/1.1" 200 None 2026-03-29 05:06:38,586 - lava_callback - DEBUG - Processing node 69c6105b2cb62322ad96950f: baseline-arm64-baylibre 2026-03-29 05:06:38,586 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6105b2cb62322ad96950f 2026-03-29 05:06:38,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:06:38,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6105b2cb62322ad96950f HTTP/1.1" 200 None 2026-03-29 05:06:38,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:06:38,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 05:06:38,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:06:38,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6105b2cb62322ad96950f HTTP/1.1" 200 None 2026-03-29 05:06:38,677 - lava_callback - ERROR - Error processing callback for node 69c6105b2cb62322ad96950f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35312 - "GET /node/6988c8b09fc35f969e1eefa7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50558 - "GET /node/6993a3d9cebe99dd8e07751b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50562 - "GET /node/694bdbf0c19e09af58d4a889%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34222 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:09:58,742 - lava_callback - INFO - Received callback for node 69c611232cb62322ad969529 INFO: 172.20.0.1:34232 - "POST /node/69c611232cb62322ad969529 HTTP/1.0" 202 Accepted 2026-03-29 05:09:58,747 - lava_callback - INFO - Starting async job submit for node 69c611232cb62322ad969529 2026-03-29 05:09:58,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:09:58,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c611232cb62322ad969529 HTTP/1.1" 200 None 2026-03-29 05:09:58,763 - lava_callback - DEBUG - Processing node 69c611232cb62322ad969529: baseline-arm64-baylibre 2026-03-29 05:09:58,764 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c611232cb62322ad969529 2026-03-29 05:09:58,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:09:58,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c611232cb62322ad969529 HTTP/1.1" 200 None 2026-03-29 05:09:58,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:09:58,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 05:09:58,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:09:58,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c611232cb62322ad969529 HTTP/1.1" 200 None 2026-03-29 05:09:58,825 - lava_callback - ERROR - Error processing callback for node 69c611232cb62322ad969529: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42388 - "GET /node/6941fd1fbb27571da24afd94%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51190 - "GET /node/69903563985e4efca431c613%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51202 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:14:15,132 - lava_callback - INFO - Received callback for node 69c612242cb62322ad96954f INFO: 172.20.0.1:34148 - "POST /node/69c612242cb62322ad96954f HTTP/1.0" 202 Accepted 2026-03-29 05:14:15,138 - lava_callback - INFO - Starting async job submit for node 69c612242cb62322ad96954f 2026-03-29 05:14:15,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:14:15,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c612242cb62322ad96954f HTTP/1.1" 200 None 2026-03-29 05:14:15,154 - lava_callback - DEBUG - Processing node 69c612242cb62322ad96954f: baseline-arm64-clabbe 2026-03-29 05:14:15,155 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c612242cb62322ad96954f 2026-03-29 05:14:15,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:34154 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 05:14:15,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c612242cb62322ad96954f HTTP/1.1" 200 None 2026-03-29 05:14:15,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:14:15,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 05:14:15,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:14:15,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c612242cb62322ad96954f HTTP/1.1" 200 None 2026-03-29 05:14:15,208 - lava_callback - ERROR - Error processing callback for node 69c612242cb62322ad96954f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41742 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:15:55,124 - lava_callback - INFO - Received callback for node 69c612762cb62322ad969555 INFO: 172.20.0.1:53160 - "POST /node/69c612762cb62322ad969555 HTTP/1.0" 202 Accepted 2026-03-29 05:15:55,129 - lava_callback - INFO - Starting async job submit for node 69c612762cb62322ad969555 2026-03-29 05:15:55,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:15:55,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c612762cb62322ad969555 HTTP/1.1" 200 None 2026-03-29 05:15:55,144 - lava_callback - DEBUG - Processing node 69c612762cb62322ad969555: baseline-arm64-clabbe 2026-03-29 05:15:55,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c612762cb62322ad969555 2026-03-29 05:15:55,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:15:55,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c612762cb62322ad969555 HTTP/1.1" 200 None 2026-03-29 05:15:55,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:15:55,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 05:15:55,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:15:55,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c612762cb62322ad969555 HTTP/1.1" 200 None 2026-03-29 05:15:55,197 - lava_callback - ERROR - Error processing callback for node 69c612762cb62322ad969555: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42284 - "GET /node/69935ac2523507f3806a9afc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33330 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 05:17:15,174 - lava_callback - INFO - Received callback for node 69c612c92cb62322ad969562 INFO: 172.20.0.1:33342 - "POST /node/69c612c92cb62322ad969562 HTTP/1.0" 202 Accepted 2026-03-29 05:17:15,179 - lava_callback - INFO - Starting async job submit for node 69c612c92cb62322ad969562 2026-03-29 05:17:15,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:17:15,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c612c92cb62322ad969562 HTTP/1.1" 200 None 2026-03-29 05:17:15,195 - lava_callback - DEBUG - Processing node 69c612c92cb62322ad969562: baseline-arm64-clabbe 2026-03-29 05:17:15,196 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c612c92cb62322ad969562 2026-03-29 05:17:15,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:17:15,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c612c92cb62322ad969562 HTTP/1.1" 200 None 2026-03-29 05:17:15,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:17:15,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 05:17:15,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:17:15,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c612c92cb62322ad969562 HTTP/1.1" 200 None 2026-03-29 05:17:15,254 - lava_callback - ERROR - Error processing callback for node 69c612c92cb62322ad969562: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41816 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:18:35,266 - lava_callback - INFO - Received callback for node 69c6131c2cb62322ad96956f INFO: 172.20.0.1:51744 - "POST /node/69c6131c2cb62322ad96956f HTTP/1.0" 202 Accepted 2026-03-29 05:18:35,272 - lava_callback - INFO - Starting async job submit for node 69c6131c2cb62322ad96956f 2026-03-29 05:18:35,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:18:35,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6131c2cb62322ad96956f HTTP/1.1" 200 None 2026-03-29 05:18:35,287 - lava_callback - DEBUG - Processing node 69c6131c2cb62322ad96956f: baseline-arm64-clabbe 2026-03-29 05:18:35,287 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6131c2cb62322ad96956f 2026-03-29 05:18:35,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:18:35,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6131c2cb62322ad96956f HTTP/1.1" 200 None 2026-03-29 05:18:35,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:18:35,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 05:18:35,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:18:35,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6131c2cb62322ad96956f HTTP/1.1" 200 None 2026-03-29 05:18:35,356 - lava_callback - ERROR - Error processing callback for node 69c6131c2cb62322ad96956f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58492 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:19:55,373 - lava_callback - INFO - Received callback for node 69c6136d2cb62322ad969583 INFO: 172.20.0.1:35132 - "POST /node/69c6136d2cb62322ad969583 HTTP/1.0" 202 Accepted 2026-03-29 05:19:55,379 - lava_callback - INFO - Starting async job submit for node 69c6136d2cb62322ad969583 2026-03-29 05:19:55,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:19:55,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6136d2cb62322ad969583 HTTP/1.1" 200 None 2026-03-29 05:19:55,395 - lava_callback - DEBUG - Processing node 69c6136d2cb62322ad969583: baseline-arm64-clabbe 2026-03-29 05:19:55,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6136d2cb62322ad969583 2026-03-29 05:19:55,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:19:55,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6136d2cb62322ad969583 HTTP/1.1" 200 None 2026-03-29 05:19:55,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:19:55,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 05:19:55,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:19:55,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6136d2cb62322ad969583 HTTP/1.1" 200 None 2026-03-29 05:19:55,462 - lava_callback - ERROR - Error processing callback for node 69c6136d2cb62322ad969583: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49834 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 05:21:15,373 - lava_callback - INFO - Received callback for node 69c613bf2cb62322ad969590 INFO: 172.20.0.1:49848 - "POST /node/69c613bf2cb62322ad969590 HTTP/1.0" 202 Accepted 2026-03-29 05:21:15,379 - lava_callback - INFO - Starting async job submit for node 69c613bf2cb62322ad969590 2026-03-29 05:21:15,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:21:15,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c613bf2cb62322ad969590 HTTP/1.1" 200 None 2026-03-29 05:21:15,394 - lava_callback - DEBUG - Processing node 69c613bf2cb62322ad969590: baseline-arm64-clabbe 2026-03-29 05:21:15,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c613bf2cb62322ad969590 2026-03-29 05:21:15,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:21:15,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c613bf2cb62322ad969590 HTTP/1.1" 200 None 2026-03-29 05:21:15,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:21:15,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-29 05:21:15,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:21:15,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c613bf2cb62322ad969590 HTTP/1.1" 200 None 2026-03-29 05:21:15,462 - lava_callback - ERROR - Error processing callback for node 69c613bf2cb62322ad969590: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59702 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 05:22:15,449 - lava_callback - INFO - Received callback for node 69c614002cb62322ad96959d INFO: 172.20.0.1:59708 - "POST /node/69c614002cb62322ad96959d HTTP/1.0" 202 Accepted 2026-03-29 05:22:15,455 - lava_callback - INFO - Starting async job submit for node 69c614002cb62322ad96959d 2026-03-29 05:22:15,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:22:15,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c614002cb62322ad96959d HTTP/1.1" 200 None 2026-03-29 05:22:15,471 - lava_callback - DEBUG - Processing node 69c614002cb62322ad96959d: baseline-arm-clabbe 2026-03-29 05:22:15,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c614002cb62322ad96959d 2026-03-29 05:22:15,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:22:15,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c614002cb62322ad96959d HTTP/1.1" 200 None 2026-03-29 05:22:15,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:22:15,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:22:15,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:22:15,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c614002cb62322ad96959d HTTP/1.1" 200 None 2026-03-29 05:22:15,538 - lava_callback - ERROR - Error processing callback for node 69c614002cb62322ad96959d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:22:55,422 - lava_callback - INFO - Received callback for node 69c614262cb62322ad9695a6 INFO: 172.20.0.1:34352 - "POST /node/69c614262cb62322ad9695a6 HTTP/1.0" 202 Accepted 2026-03-29 05:22:55,427 - lava_callback - INFO - Starting async job submit for node 69c614262cb62322ad9695a6 2026-03-29 05:22:55,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:22:55,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c614262cb62322ad9695a6 HTTP/1.1" 200 None 2026-03-29 05:22:55,443 - lava_callback - DEBUG - Processing node 69c614262cb62322ad9695a6: baseline-arm-clabbe 2026-03-29 05:22:55,443 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c614262cb62322ad9695a6 2026-03-29 05:22:55,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:22:55,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c614262cb62322ad9695a6 HTTP/1.1" 200 None 2026-03-29 05:22:55,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:22:55,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:22:55,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:22:55,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c614262cb62322ad9695a6 HTTP/1.1" 200 None 2026-03-29 05:22:55,511 - lava_callback - ERROR - Error processing callback for node 69c614262cb62322ad9695a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49022 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:23:59,619 - lava_callback - INFO - Received callback for node 69c614692cb62322ad9695b9 INFO: 172.20.0.1:49024 - "POST /node/69c614692cb62322ad9695b9 HTTP/1.0" 202 Accepted 2026-03-29 05:23:59,624 - lava_callback - INFO - Starting async job submit for node 69c614692cb62322ad9695b9 2026-03-29 05:23:59,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:23:59,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c614692cb62322ad9695b9 HTTP/1.1" 200 None 2026-03-29 05:23:59,648 - lava_callback - DEBUG - Processing node 69c614692cb62322ad9695b9: baseline-arm-baylibre 2026-03-29 05:23:59,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c614692cb62322ad9695b9 2026-03-29 05:23:59,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:23:59,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c614692cb62322ad9695b9 HTTP/1.1" 200 None 2026-03-29 05:23:59,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:23:59,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:23:59,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:23:59,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c614692cb62322ad9695b9 HTTP/1.1" 200 None 2026-03-29 05:23:59,705 - lava_callback - ERROR - Error processing callback for node 69c614692cb62322ad9695b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56070 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:25:59,664 - lava_callback - INFO - Received callback for node 69c614de2cb62322ad9695cd INFO: 172.20.0.1:56074 - "POST /node/69c614de2cb62322ad9695cd HTTP/1.0" 202 Accepted 2026-03-29 05:25:59,669 - lava_callback - INFO - Starting async job submit for node 69c614de2cb62322ad9695cd 2026-03-29 05:25:59,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:25:59,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c614de2cb62322ad9695cd HTTP/1.1" 200 None 2026-03-29 05:25:59,707 - lava_callback - DEBUG - Processing node 69c614de2cb62322ad9695cd: baseline-arm-baylibre 2026-03-29 05:25:59,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c614de2cb62322ad9695cd 2026-03-29 05:25:59,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:25:59,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c614de2cb62322ad9695cd HTTP/1.1" 200 None 2026-03-29 05:25:59,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:25:59,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:25:59,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:25:59,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c614de2cb62322ad9695cd HTTP/1.1" 200 None 2026-03-29 05:25:59,773 - lava_callback - ERROR - Error processing callback for node 69c614de2cb62322ad9695cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60392 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:26:39,608 - lava_callback - INFO - Received callback for node 69c615032cb62322ad9695d7 INFO: 172.20.0.1:60408 - "POST /node/69c615032cb62322ad9695d7 HTTP/1.0" 202 Accepted 2026-03-29 05:26:39,614 - lava_callback - INFO - Starting async job submit for node 69c615032cb62322ad9695d7 2026-03-29 05:26:39,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:26:39,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c615032cb62322ad9695d7 HTTP/1.1" 200 None 2026-03-29 05:26:39,630 - lava_callback - DEBUG - Processing node 69c615032cb62322ad9695d7: baseline-arm-baylibre 2026-03-29 05:26:39,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c615032cb62322ad9695d7 2026-03-29 05:26:39,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:26:39,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c615032cb62322ad9695d7 HTTP/1.1" 200 None 2026-03-29 05:26:39,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:26:39,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:26:39,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:26:39,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c615032cb62322ad9695d7 HTTP/1.1" 200 None 2026-03-29 05:26:39,698 - lava_callback - ERROR - Error processing callback for node 69c615032cb62322ad9695d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48346 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:28:39,787 - lava_callback - INFO - Received callback for node 69c615772cb62322ad9695ef INFO: 172.20.0.1:36530 - "POST /node/69c615772cb62322ad9695ef HTTP/1.0" 202 Accepted 2026-03-29 05:28:39,792 - lava_callback - INFO - Starting async job submit for node 69c615772cb62322ad9695ef 2026-03-29 05:28:39,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:28:39,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c615772cb62322ad9695ef HTTP/1.1" 200 None 2026-03-29 05:28:39,809 - lava_callback - DEBUG - Processing node 69c615772cb62322ad9695ef: baseline-arm-baylibre 2026-03-29 05:28:39,809 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c615772cb62322ad9695ef 2026-03-29 05:28:39,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:28:39,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c615772cb62322ad9695ef HTTP/1.1" 200 None 2026-03-29 05:28:39,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:28:40,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:28:40,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:28:40,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c615772cb62322ad9695ef HTTP/1.1" 200 None 2026-03-29 05:28:40,055 - lava_callback - ERROR - Error processing callback for node 69c615772cb62322ad9695ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46256 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:29:20,029 - lava_callback - INFO - Received callback for node 69c6159d2cb62322ad969601 INFO: 172.20.0.1:58388 - "POST /node/69c6159d2cb62322ad969601 HTTP/1.0" 202 Accepted 2026-03-29 05:29:20,062 - lava_callback - INFO - Starting async job submit for node 69c6159d2cb62322ad969601 2026-03-29 05:29:20,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:29:20,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6159d2cb62322ad969601 HTTP/1.1" 200 None 2026-03-29 05:29:20,097 - lava_callback - DEBUG - Processing node 69c6159d2cb62322ad969601: baseline-arm-baylibre 2026-03-29 05:29:20,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6159d2cb62322ad969601 2026-03-29 05:29:20,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:29:20,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6159d2cb62322ad969601 HTTP/1.1" 200 None 2026-03-29 05:29:20,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:29:20,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:29:20,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:29:20,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6159d2cb62322ad969601 HTTP/1.1" 200 None 2026-03-29 05:29:20,163 - lava_callback - ERROR - Error processing callback for node 69c6159d2cb62322ad969601: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51830 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:29:39,774 - lava_callback - INFO - Received callback for node 69c615c12cb62322ad969610 INFO: 172.20.0.1:40704 - "POST /node/69c615c12cb62322ad969610 HTTP/1.0" 202 Accepted 2026-03-29 05:29:39,780 - lava_callback - INFO - Starting async job submit for node 69c615c12cb62322ad969610 2026-03-29 05:29:39,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:29:39,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c615c12cb62322ad969610 HTTP/1.1" 200 None 2026-03-29 05:29:39,795 - lava_callback - DEBUG - Processing node 69c615c12cb62322ad969610: baseline-arm-baylibre 2026-03-29 05:29:39,795 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c615c12cb62322ad969610 2026-03-29 05:29:39,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:29:39,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c615c12cb62322ad969610 HTTP/1.1" 200 None 2026-03-29 05:29:39,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:29:39,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:29:39,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:29:39,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c615c12cb62322ad969610 HTTP/1.1" 200 None 2026-03-29 05:29:39,850 - lava_callback - ERROR - Error processing callback for node 69c615c12cb62322ad969610: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42940 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:31:35,800 - lava_callback - INFO - Received callback for node 69c6162e2cb62322ad969637 INFO: 172.20.0.1:54478 - "POST /node/69c6162e2cb62322ad969637 HTTP/1.0" 202 Accepted 2026-03-29 05:31:35,805 - lava_callback - INFO - Starting async job submit for node 69c6162e2cb62322ad969637 2026-03-29 05:31:35,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:31:35,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6162e2cb62322ad969637 HTTP/1.1" 200 None 2026-03-29 05:31:35,822 - lava_callback - DEBUG - Processing node 69c6162e2cb62322ad969637: baseline-arm-clabbe 2026-03-29 05:31:35,822 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6162e2cb62322ad969637 2026-03-29 05:31:35,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:31:35,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6162e2cb62322ad969637 HTTP/1.1" 200 None 2026-03-29 05:31:35,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:31:35,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:31:35,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:31:35,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6162e2cb62322ad969637 HTTP/1.1" 200 None 2026-03-29 05:31:35,878 - lava_callback - ERROR - Error processing callback for node 69c6162e2cb62322ad969637: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45378 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 05:32:15,784 - lava_callback - INFO - Received callback for node 69c616522cb62322ad969645 INFO: 172.20.0.1:45390 - "POST /node/69c616522cb62322ad969645 HTTP/1.0" 202 Accepted 2026-03-29 05:32:15,791 - lava_callback - INFO - Starting async job submit for node 69c616522cb62322ad969645 2026-03-29 05:32:15,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:32:15,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c616522cb62322ad969645 HTTP/1.1" 200 None 2026-03-29 05:32:15,808 - lava_callback - DEBUG - Processing node 69c616522cb62322ad969645: baseline-arm-clabbe 2026-03-29 05:32:15,808 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c616522cb62322ad969645 2026-03-29 05:32:15,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:32:15,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c616522cb62322ad969645 HTTP/1.1" 200 None 2026-03-29 05:32:15,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:32:15,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:32:15,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:32:15,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c616522cb62322ad969645 HTTP/1.1" 200 None 2026-03-29 05:32:15,865 - lava_callback - ERROR - Error processing callback for node 69c616522cb62322ad969645: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51876 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:32:55,841 - lava_callback - INFO - Received callback for node 69c616772cb62322ad96965a INFO: 172.20.0.1:59618 - "POST /node/69c616772cb62322ad96965a HTTP/1.0" 202 Accepted 2026-03-29 05:32:55,846 - lava_callback - INFO - Starting async job submit for node 69c616772cb62322ad96965a 2026-03-29 05:32:55,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:32:55,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c616772cb62322ad96965a HTTP/1.1" 200 None 2026-03-29 05:32:55,862 - lava_callback - DEBUG - Processing node 69c616772cb62322ad96965a: baseline-arm-clabbe 2026-03-29 05:32:55,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c616772cb62322ad96965a 2026-03-29 05:32:55,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:32:55,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c616772cb62322ad96965a HTTP/1.1" 200 None 2026-03-29 05:32:55,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:32:55,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:32:55,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:32:55,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c616772cb62322ad96965a HTTP/1.1" 200 None 2026-03-29 05:32:55,916 - lava_callback - ERROR - Error processing callback for node 69c616772cb62322ad96965a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60652 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:33:35,845 - lava_callback - INFO - Received callback for node 69c6169c2cb62322ad969666 INFO: 172.20.0.1:44752 - "POST /node/69c6169c2cb62322ad969666 HTTP/1.0" 202 Accepted 2026-03-29 05:33:35,850 - lava_callback - INFO - Starting async job submit for node 69c6169c2cb62322ad969666 2026-03-29 05:33:35,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:33:35,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6169c2cb62322ad969666 HTTP/1.1" 200 None 2026-03-29 05:33:35,870 - lava_callback - DEBUG - Processing node 69c6169c2cb62322ad969666: baseline-arm-clabbe 2026-03-29 05:33:35,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6169c2cb62322ad969666 2026-03-29 05:33:35,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:33:35,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6169c2cb62322ad969666 HTTP/1.1" 200 None 2026-03-29 05:33:35,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:33:35,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:33:35,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:33:35,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6169c2cb62322ad969666 HTTP/1.1" 200 None 2026-03-29 05:33:35,922 - lava_callback - ERROR - Error processing callback for node 69c6169c2cb62322ad969666: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39690 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:33:55,873 - lava_callback - INFO - Received callback for node 69c616c12cb62322ad969672 INFO: 172.20.0.1:38660 - "POST /node/69c616c12cb62322ad969672 HTTP/1.0" 202 Accepted 2026-03-29 05:33:55,879 - lava_callback - INFO - Starting async job submit for node 69c616c12cb62322ad969672 2026-03-29 05:33:55,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:33:55,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c616c12cb62322ad969672 HTTP/1.1" 200 None 2026-03-29 05:33:55,894 - lava_callback - DEBUG - Processing node 69c616c12cb62322ad969672: baseline-arm-clabbe 2026-03-29 05:33:55,895 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c616c12cb62322ad969672 2026-03-29 05:33:55,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:33:55,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c616c12cb62322ad969672 HTTP/1.1" 200 None 2026-03-29 05:33:55,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:33:55,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c55c70abf66a8f4d9b9655 HTTP/1.1" 200 None 2026-03-29 05:33:55,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:33:55,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c616c12cb62322ad969672 HTTP/1.1" 200 None 2026-03-29 05:33:55,957 - lava_callback - ERROR - Error processing callback for node 69c616c12cb62322ad969672: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41140 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:34:35,841 - lava_callback - INFO - Received callback for node 69c616e52cb62322ad96967e INFO: 172.20.0.1:43580 - "POST /node/69c616e52cb62322ad96967e HTTP/1.0" 202 Accepted 2026-03-29 05:34:35,846 - lava_callback - INFO - Starting async job submit for node 69c616e52cb62322ad96967e 2026-03-29 05:34:35,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:34:35,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c616e52cb62322ad96967e HTTP/1.1" 200 None 2026-03-29 05:34:35,862 - lava_callback - DEBUG - Processing node 69c616e52cb62322ad96967e: baseline-arm-clabbe 2026-03-29 05:34:35,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c616e52cb62322ad96967e 2026-03-29 05:34:35,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:34:35,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c616e52cb62322ad96967e HTTP/1.1" 200 None 2026-03-29 05:34:35,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:34:35,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:34:35,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:34:35,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c616e52cb62322ad96967e HTTP/1.1" 200 None 2026-03-29 05:34:35,912 - lava_callback - ERROR - Error processing callback for node 69c616e52cb62322ad96967e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55230 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 05:35:15,893 - lava_callback - INFO - Received callback for node 69c6170a2cb62322ad96968b INFO: 172.20.0.1:55234 - "POST /node/69c6170a2cb62322ad96968b HTTP/1.0" 202 Accepted 2026-03-29 05:35:15,899 - lava_callback - INFO - Starting async job submit for node 69c6170a2cb62322ad96968b 2026-03-29 05:35:15,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:35:15,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6170a2cb62322ad96968b HTTP/1.1" 200 None 2026-03-29 05:35:15,915 - lava_callback - DEBUG - Processing node 69c6170a2cb62322ad96968b: baseline-arm-clabbe 2026-03-29 05:35:15,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6170a2cb62322ad96968b 2026-03-29 05:35:15,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:35:15,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6170a2cb62322ad96968b HTTP/1.1" 200 None 2026-03-29 05:35:15,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:35:15,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:35:15,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:35:15,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6170a2cb62322ad96968b HTTP/1.1" 200 None 2026-03-29 05:35:15,979 - lava_callback - ERROR - Error processing callback for node 69c6170a2cb62322ad96968b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34578 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:35:55,943 - lava_callback - INFO - Received callback for node 69c6172f2cb62322ad969694 INFO: 172.20.0.1:58662 - "POST /node/69c6172f2cb62322ad969694 HTTP/1.0" 202 Accepted 2026-03-29 05:35:55,949 - lava_callback - INFO - Starting async job submit for node 69c6172f2cb62322ad969694 2026-03-29 05:35:55,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:35:55,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6172f2cb62322ad969694 HTTP/1.1" 200 None 2026-03-29 05:35:55,965 - lava_callback - DEBUG - Processing node 69c6172f2cb62322ad969694: baseline-arm-clabbe 2026-03-29 05:35:55,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6172f2cb62322ad969694 2026-03-29 05:35:55,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:35:55,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6172f2cb62322ad969694 HTTP/1.1" 200 None 2026-03-29 05:35:55,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:35:55,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:35:55,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:35:56,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6172f2cb62322ad969694 HTTP/1.1" 200 None 2026-03-29 05:35:56,022 - lava_callback - ERROR - Error processing callback for node 69c6172f2cb62322ad969694: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57842 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:36:35,998 - lava_callback - INFO - Received callback for node 69c617542cb62322ad96969e INFO: 172.20.0.1:33346 - "POST /node/69c617542cb62322ad96969e HTTP/1.0" 202 Accepted 2026-03-29 05:36:36,003 - lava_callback - INFO - Starting async job submit for node 69c617542cb62322ad96969e 2026-03-29 05:36:36,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:36:36,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c617542cb62322ad96969e HTTP/1.1" 200 None 2026-03-29 05:36:36,021 - lava_callback - DEBUG - Processing node 69c617542cb62322ad96969e: baseline-arm-clabbe 2026-03-29 05:36:36,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c617542cb62322ad96969e 2026-03-29 05:36:36,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:36:36,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c617542cb62322ad96969e HTTP/1.1" 200 None 2026-03-29 05:36:36,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:36:36,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:36:36,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:36:36,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c617542cb62322ad96969e HTTP/1.1" 200 None 2026-03-29 05:36:36,085 - lava_callback - ERROR - Error processing callback for node 69c617542cb62322ad96969e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42546 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 05:37:16,002 - lava_callback - INFO - Received callback for node 69c617782cb62322ad9696a8 INFO: 172.20.0.1:42556 - "POST /node/69c617782cb62322ad9696a8 HTTP/1.0" 202 Accepted 2026-03-29 05:37:16,008 - lava_callback - INFO - Starting async job submit for node 69c617782cb62322ad9696a8 2026-03-29 05:37:16,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:37:16,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c617782cb62322ad9696a8 HTTP/1.1" 200 None 2026-03-29 05:37:16,024 - lava_callback - DEBUG - Processing node 69c617782cb62322ad9696a8: baseline-arm-clabbe 2026-03-29 05:37:16,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c617782cb62322ad9696a8 2026-03-29 05:37:16,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:37:16,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c617782cb62322ad9696a8 HTTP/1.1" 200 None 2026-03-29 05:37:16,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:37:16,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:37:16,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:37:16,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c617782cb62322ad9696a8 HTTP/1.1" 200 None 2026-03-29 05:37:16,094 - lava_callback - ERROR - Error processing callback for node 69c617782cb62322ad9696a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32874 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:37:36,056 - lava_callback - INFO - Received callback for node 69c6179d2cb62322ad9696b2 INFO: 172.20.0.1:36400 - "POST /node/69c6179d2cb62322ad9696b2 HTTP/1.0" 202 Accepted 2026-03-29 05:37:36,062 - lava_callback - INFO - Starting async job submit for node 69c6179d2cb62322ad9696b2 2026-03-29 05:37:36,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:37:36,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6179d2cb62322ad9696b2 HTTP/1.1" 200 None 2026-03-29 05:37:36,086 - lava_callback - DEBUG - Processing node 69c6179d2cb62322ad9696b2: baseline-arm-clabbe 2026-03-29 05:37:36,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6179d2cb62322ad9696b2 2026-03-29 05:37:36,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:37:36,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6179d2cb62322ad9696b2 HTTP/1.1" 200 None 2026-03-29 05:37:36,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:37:36,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:37:36,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:37:36,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6179d2cb62322ad9696b2 HTTP/1.1" 200 None 2026-03-29 05:37:36,143 - lava_callback - ERROR - Error processing callback for node 69c6179d2cb62322ad9696b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41138 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-29 05:38:16,103 - lava_callback - INFO - Received callback for node 69c617c12cb62322ad9696bf INFO: 172.20.0.1:41150 - "POST /node/69c617c12cb62322ad9696bf HTTP/1.0" 202 Accepted 2026-03-29 05:38:16,108 - lava_callback - INFO - Starting async job submit for node 69c617c12cb62322ad9696bf 2026-03-29 05:38:16,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:38:16,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c617c12cb62322ad9696bf HTTP/1.1" 200 None 2026-03-29 05:38:16,132 - lava_callback - DEBUG - Processing node 69c617c12cb62322ad9696bf: baseline-arm-clabbe 2026-03-29 05:38:16,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c617c12cb62322ad9696bf 2026-03-29 05:38:16,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:38:16,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c617c12cb62322ad9696bf HTTP/1.1" 200 None 2026-03-29 05:38:16,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:38:16,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:38:16,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:38:16,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c617c12cb62322ad9696bf HTTP/1.1" 200 None 2026-03-29 05:38:16,199 - lava_callback - ERROR - Error processing callback for node 69c617c12cb62322ad9696bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60284 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:38:56,124 - lava_callback - INFO - Received callback for node 69c617e62cb62322ad9696c6 INFO: 172.20.0.1:33040 - "POST /node/69c617e62cb62322ad9696c6 HTTP/1.0" 202 Accepted 2026-03-29 05:38:56,130 - lava_callback - INFO - Starting async job submit for node 69c617e62cb62322ad9696c6 2026-03-29 05:38:56,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:38:56,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c617e62cb62322ad9696c6 HTTP/1.1" 200 None 2026-03-29 05:38:56,147 - lava_callback - DEBUG - Processing node 69c617e62cb62322ad9696c6: baseline-arm-clabbe 2026-03-29 05:38:56,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c617e62cb62322ad9696c6 2026-03-29 05:38:56,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:38:56,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c617e62cb62322ad9696c6 HTTP/1.1" 200 None 2026-03-29 05:38:56,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:38:56,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-29 05:38:56,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:38:56,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c617e62cb62322ad9696c6 HTTP/1.1" 200 None 2026-03-29 05:38:56,200 - lava_callback - ERROR - Error processing callback for node 69c617e62cb62322ad9696c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46512 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:39:40,467 - lava_callback - INFO - Received callback for node 69c618102cb62322ad9696d0 INFO: 172.20.0.1:57004 - "POST /node/69c618102cb62322ad9696d0 HTTP/1.0" 202 Accepted 2026-03-29 05:39:40,473 - lava_callback - INFO - Starting async job submit for node 69c618102cb62322ad9696d0 2026-03-29 05:39:40,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:39:40,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c618102cb62322ad9696d0 HTTP/1.1" 200 None 2026-03-29 05:39:40,495 - lava_callback - DEBUG - Processing node 69c618102cb62322ad9696d0: baseline-arm64-baylibre 2026-03-29 05:39:40,495 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c618102cb62322ad9696d0 2026-03-29 05:39:40,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:39:40,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c618102cb62322ad9696d0 HTTP/1.1" 200 None 2026-03-29 05:39:40,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:39:40,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:39:40,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:39:40,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c618102cb62322ad9696d0 HTTP/1.1" 200 None 2026-03-29 05:39:40,546 - lava_callback - ERROR - Error processing callback for node 69c618102cb62322ad9696d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58094 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:41:00,438 - lava_callback - INFO - Received callback for node 69c618632cb62322ad9696da INFO: 172.20.0.1:33322 - "POST /node/69c618632cb62322ad9696da HTTP/1.0" 202 Accepted 2026-03-29 05:41:00,443 - lava_callback - INFO - Starting async job submit for node 69c618632cb62322ad9696da 2026-03-29 05:41:00,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:41:00,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c618632cb62322ad9696da HTTP/1.1" 200 None 2026-03-29 05:41:00,464 - lava_callback - DEBUG - Processing node 69c618632cb62322ad9696da: baseline-arm64-baylibre 2026-03-29 05:41:00,464 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c618632cb62322ad9696da 2026-03-29 05:41:00,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:41:00,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c618632cb62322ad9696da HTTP/1.1" 200 None 2026-03-29 05:41:00,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:41:00,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:41:00,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:41:00,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c618632cb62322ad9696da HTTP/1.1" 200 None 2026-03-29 05:41:00,516 - lava_callback - ERROR - Error processing callback for node 69c618632cb62322ad9696da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38738 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:42:20,484 - lava_callback - INFO - Received callback for node 69c618b62cb62322ad9696e6 INFO: 172.20.0.1:58226 - "POST /node/69c618b62cb62322ad9696e6 HTTP/1.0" 202 Accepted 2026-03-29 05:42:20,490 - lava_callback - INFO - Starting async job submit for node 69c618b62cb62322ad9696e6 2026-03-29 05:42:20,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:42:20,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c618b62cb62322ad9696e6 HTTP/1.1" 200 None 2026-03-29 05:42:20,506 - lava_callback - DEBUG - Processing node 69c618b62cb62322ad9696e6: baseline-arm64-baylibre 2026-03-29 05:42:20,506 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c618b62cb62322ad9696e6 2026-03-29 05:42:20,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:42:20,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c618b62cb62322ad9696e6 HTTP/1.1" 200 None 2026-03-29 05:42:20,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:42:20,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:42:20,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:42:20,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c618b62cb62322ad9696e6 HTTP/1.1" 200 None 2026-03-29 05:42:20,597 - lava_callback - ERROR - Error processing callback for node 69c618b62cb62322ad9696e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49474 - "GET /node/699447d85ae459b9df3daaa9%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48232 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:45:00,838 - lava_callback - INFO - Received callback for node 69c6195a2cb62322ad9696fe INFO: 172.20.0.1:48112 - "POST /node/69c6195a2cb62322ad9696fe HTTP/1.0" 202 Accepted 2026-03-29 05:45:00,847 - lava_callback - INFO - Starting async job submit for node 69c6195a2cb62322ad9696fe 2026-03-29 05:45:00,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:45:00,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6195a2cb62322ad9696fe HTTP/1.1" 200 None 2026-03-29 05:45:00,864 - lava_callback - DEBUG - Processing node 69c6195a2cb62322ad9696fe: baseline-arm64-baylibre 2026-03-29 05:45:00,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6195a2cb62322ad9696fe 2026-03-29 05:45:00,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:45:00,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6195a2cb62322ad9696fe HTTP/1.1" 200 None 2026-03-29 05:45:00,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:45:00,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:45:00,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:45:00,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6195a2cb62322ad9696fe HTTP/1.1" 200 None 2026-03-29 05:45:00,933 - lava_callback - ERROR - Error processing callback for node 69c6195a2cb62322ad9696fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42074 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:46:20,744 - lava_callback - INFO - Received callback for node 69c619a92cb62322ad96970d INFO: 172.20.0.1:42692 - "POST /node/69c619a92cb62322ad96970d HTTP/1.0" 202 Accepted 2026-03-29 05:46:20,750 - lava_callback - INFO - Starting async job submit for node 69c619a92cb62322ad96970d 2026-03-29 05:46:20,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:46:20,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c619a92cb62322ad96970d HTTP/1.1" 200 None 2026-03-29 05:46:20,767 - lava_callback - DEBUG - Processing node 69c619a92cb62322ad96970d: baseline-arm64-baylibre 2026-03-29 05:46:20,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c619a92cb62322ad96970d 2026-03-29 05:46:20,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:46:20,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c619a92cb62322ad96970d HTTP/1.1" 200 None 2026-03-29 05:46:20,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:46:20,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:46:20,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:46:20,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c619a92cb62322ad96970d HTTP/1.1" 200 None 2026-03-29 05:46:20,830 - lava_callback - ERROR - Error processing callback for node 69c619a92cb62322ad96970d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33534 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:47:56,449 - lava_callback - INFO - Received callback for node 69c619ff2cb62322ad96971a INFO: 172.20.0.1:38666 - "POST /node/69c619ff2cb62322ad96971a HTTP/1.0" 202 Accepted 2026-03-29 05:47:56,455 - lava_callback - INFO - Starting async job submit for node 69c619ff2cb62322ad96971a 2026-03-29 05:47:56,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:47:56,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c619ff2cb62322ad96971a HTTP/1.1" 200 None 2026-03-29 05:47:56,471 - lava_callback - DEBUG - Processing node 69c619ff2cb62322ad96971a: baseline-arm64-clabbe 2026-03-29 05:47:56,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c619ff2cb62322ad96971a 2026-03-29 05:47:56,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:47:56,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c619ff2cb62322ad96971a HTTP/1.1" 200 None 2026-03-29 05:47:56,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:47:56,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:47:56,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:47:56,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c619ff2cb62322ad96971a HTTP/1.1" 200 None INFO: 172.20.0.1:38676 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:47:56,554 - lava_callback - ERROR - Error processing callback for node 69c619ff2cb62322ad96971a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45680 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:49:16,556 - lava_callback - INFO - Received callback for node 69c61a512cb62322ad969720 INFO: 172.20.0.1:45686 - "POST /node/69c61a512cb62322ad969720 HTTP/1.0" 202 Accepted 2026-03-29 05:49:16,561 - lava_callback - INFO - Starting async job submit for node 69c61a512cb62322ad969720 2026-03-29 05:49:16,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:49:16,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61a512cb62322ad969720 HTTP/1.1" 200 None 2026-03-29 05:49:16,577 - lava_callback - DEBUG - Processing node 69c61a512cb62322ad969720: baseline-arm64-clabbe 2026-03-29 05:49:16,577 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61a512cb62322ad969720 2026-03-29 05:49:16,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:49:16,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61a512cb62322ad969720 HTTP/1.1" 200 None 2026-03-29 05:49:16,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:49:16,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:49:16,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:49:16,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61a512cb62322ad969720 HTTP/1.1" 200 None 2026-03-29 05:49:16,629 - lava_callback - ERROR - Error processing callback for node 69c61a512cb62322ad969720: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60544 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:50:36,602 - lava_callback - INFO - Received callback for node 69c61aa42cb62322ad96972e INFO: 172.20.0.1:60560 - "POST /node/69c61aa42cb62322ad96972e HTTP/1.0" 202 Accepted 2026-03-29 05:50:36,608 - lava_callback - INFO - Starting async job submit for node 69c61aa42cb62322ad96972e 2026-03-29 05:50:36,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:50:36,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61aa42cb62322ad96972e HTTP/1.1" 200 None 2026-03-29 05:50:36,623 - lava_callback - DEBUG - Processing node 69c61aa42cb62322ad96972e: baseline-arm64-clabbe 2026-03-29 05:50:36,624 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61aa42cb62322ad96972e 2026-03-29 05:50:36,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:50:36,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61aa42cb62322ad96972e HTTP/1.1" 200 None 2026-03-29 05:50:36,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:50:36,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:50:36,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:50:36,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61aa42cb62322ad96972e HTTP/1.1" 200 None 2026-03-29 05:50:36,680 - lava_callback - ERROR - Error processing callback for node 69c61aa42cb62322ad96972e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55144 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:51:56,638 - lava_callback - INFO - Received callback for node 69c61af72cb62322ad96973a INFO: 172.20.0.1:55156 - "POST /node/69c61af72cb62322ad96973a HTTP/1.0" 202 Accepted 2026-03-29 05:51:56,643 - lava_callback - INFO - Starting async job submit for node 69c61af72cb62322ad96973a 2026-03-29 05:51:56,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:51:56,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61af72cb62322ad96973a HTTP/1.1" 200 None 2026-03-29 05:51:56,660 - lava_callback - DEBUG - Processing node 69c61af72cb62322ad96973a: baseline-arm64-clabbe 2026-03-29 05:51:56,660 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61af72cb62322ad96973a 2026-03-29 05:51:56,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:51:56,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61af72cb62322ad96973a HTTP/1.1" 200 None 2026-03-29 05:51:56,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:51:56,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:51:56,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:51:56,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61af72cb62322ad96973a HTTP/1.1" 200 None 2026-03-29 05:51:56,713 - lava_callback - ERROR - Error processing callback for node 69c61af72cb62322ad96973a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33840 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:53:16,730 - lava_callback - INFO - Received callback for node 69c61b4a2cb62322ad969749 INFO: 172.20.0.1:33846 - "POST /node/69c61b4a2cb62322ad969749 HTTP/1.0" 202 Accepted 2026-03-29 05:53:16,735 - lava_callback - INFO - Starting async job submit for node 69c61b4a2cb62322ad969749 2026-03-29 05:53:16,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:53:16,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61b4a2cb62322ad969749 HTTP/1.1" 200 None 2026-03-29 05:53:16,751 - lava_callback - DEBUG - Processing node 69c61b4a2cb62322ad969749: baseline-arm64-clabbe 2026-03-29 05:53:16,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61b4a2cb62322ad969749 2026-03-29 05:53:16,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:53:16,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61b4a2cb62322ad969749 HTTP/1.1" 200 None 2026-03-29 05:53:16,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:53:16,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:53:16,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:53:16,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61b4a2cb62322ad969749 HTTP/1.1" 200 None 2026-03-29 05:53:16,816 - lava_callback - ERROR - Error processing callback for node 69c61b4a2cb62322ad969749: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39210 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:54:56,795 - lava_callback - INFO - Received callback for node 69c61b9c2cb62322ad969750 INFO: 172.20.0.1:39224 - "POST /node/69c61b9c2cb62322ad969750 HTTP/1.0" 202 Accepted 2026-03-29 05:54:56,801 - lava_callback - INFO - Starting async job submit for node 69c61b9c2cb62322ad969750 2026-03-29 05:54:56,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:54:56,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61b9c2cb62322ad969750 HTTP/1.1" 200 None 2026-03-29 05:54:56,817 - lava_callback - DEBUG - Processing node 69c61b9c2cb62322ad969750: baseline-arm64-clabbe 2026-03-29 05:54:56,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61b9c2cb62322ad969750 2026-03-29 05:54:56,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:54:56,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61b9c2cb62322ad969750 HTTP/1.1" 200 None 2026-03-29 05:54:56,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:54:56,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:54:56,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:54:56,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61b9c2cb62322ad969750 HTTP/1.1" 200 None 2026-03-29 05:54:56,873 - lava_callback - ERROR - Error processing callback for node 69c61b9c2cb62322ad969750: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42900 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:56:01,318 - lava_callback - INFO - Received callback for node 69c61be62cb62322ad96975a INFO: 172.20.0.1:42916 - "POST /node/69c61be62cb62322ad96975a HTTP/1.0" 202 Accepted 2026-03-29 05:56:01,324 - lava_callback - INFO - Starting async job submit for node 69c61be62cb62322ad96975a 2026-03-29 05:56:01,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:56:01,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61be62cb62322ad96975a HTTP/1.1" 200 None 2026-03-29 05:56:01,369 - lava_callback - DEBUG - Processing node 69c61be62cb62322ad96975a: baseline-arm64-baylibre 2026-03-29 05:56:01,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61be62cb62322ad96975a 2026-03-29 05:56:01,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:56:01,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61be62cb62322ad96975a HTTP/1.1" 200 None 2026-03-29 05:56:01,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:56:01,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:56:01,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:56:01,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61be62cb62322ad96975a HTTP/1.1" 200 None 2026-03-29 05:56:01,422 - lava_callback - ERROR - Error processing callback for node 69c61be62cb62322ad96975a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58616 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:57:21,315 - lava_callback - INFO - Received callback for node 69c61c372cb62322ad969767 INFO: 172.20.0.1:53488 - "POST /node/69c61c372cb62322ad969767 HTTP/1.0" 202 Accepted 2026-03-29 05:57:21,321 - lava_callback - INFO - Starting async job submit for node 69c61c372cb62322ad969767 2026-03-29 05:57:21,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:57:21,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61c372cb62322ad969767 HTTP/1.1" 200 None 2026-03-29 05:57:21,346 - lava_callback - DEBUG - Processing node 69c61c372cb62322ad969767: baseline-arm64-baylibre 2026-03-29 05:57:21,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61c372cb62322ad969767 2026-03-29 05:57:21,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:57:21,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61c372cb62322ad969767 HTTP/1.1" 200 None 2026-03-29 05:57:21,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:57:21,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:57:21,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:57:21,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61c372cb62322ad969767 HTTP/1.1" 200 None 2026-03-29 05:57:21,428 - lava_callback - ERROR - Error processing callback for node 69c61c372cb62322ad969767: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59480 - "GET / HTTP/1.0" 200 OK 2026-03-29 05:58:41,408 - lava_callback - INFO - Received callback for node 69c61c872cb62322ad96976e INFO: 172.20.0.1:45562 - "POST /node/69c61c872cb62322ad96976e HTTP/1.0" 202 Accepted 2026-03-29 05:58:41,414 - lava_callback - INFO - Starting async job submit for node 69c61c872cb62322ad96976e 2026-03-29 05:58:41,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:58:41,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61c872cb62322ad96976e HTTP/1.1" 200 None 2026-03-29 05:58:41,439 - lava_callback - DEBUG - Processing node 69c61c872cb62322ad96976e: baseline-arm64-baylibre 2026-03-29 05:58:41,439 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61c872cb62322ad96976e 2026-03-29 05:58:41,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:58:41,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61c872cb62322ad96976e HTTP/1.1" 200 None 2026-03-29 05:58:41,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:58:41,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 05:58:41,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 05:58:41,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61c872cb62322ad96976e HTTP/1.1" 200 None 2026-03-29 05:58:41,509 - lava_callback - ERROR - Error processing callback for node 69c61c872cb62322ad96976e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55284 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:00:01,434 - lava_callback - INFO - Received callback for node 69c61cd72cb62322ad96977b INFO: 172.20.0.1:55296 - "POST /node/69c61cd72cb62322ad96977b HTTP/1.0" 202 Accepted 2026-03-29 06:00:01,439 - lava_callback - INFO - Starting async job submit for node 69c61cd72cb62322ad96977b 2026-03-29 06:00:01,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:00:01,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61cd72cb62322ad96977b HTTP/1.1" 200 None 2026-03-29 06:00:01,456 - lava_callback - DEBUG - Processing node 69c61cd72cb62322ad96977b: baseline-arm64-baylibre 2026-03-29 06:00:01,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61cd72cb62322ad96977b 2026-03-29 06:00:01,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:00:01,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61cd72cb62322ad96977b HTTP/1.1" 200 None 2026-03-29 06:00:01,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:00:01,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 06:00:01,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:00:01,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61cd72cb62322ad96977b HTTP/1.1" 200 None 2026-03-29 06:00:01,518 - lava_callback - ERROR - Error processing callback for node 69c61cd72cb62322ad96977b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36128 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:01:21,546 - lava_callback - INFO - Received callback for node 69c61d2c2cb62322ad96978b INFO: 172.20.0.1:57264 - "POST /node/69c61d2c2cb62322ad96978b HTTP/1.0" 202 Accepted 2026-03-29 06:01:21,551 - lava_callback - INFO - Starting async job submit for node 69c61d2c2cb62322ad96978b 2026-03-29 06:01:21,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:01:21,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61d2c2cb62322ad96978b HTTP/1.1" 200 None 2026-03-29 06:01:21,603 - lava_callback - DEBUG - Processing node 69c61d2c2cb62322ad96978b: baseline-arm64-baylibre 2026-03-29 06:01:21,603 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61d2c2cb62322ad96978b 2026-03-29 06:01:21,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:01:21,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61d2c2cb62322ad96978b HTTP/1.1" 200 None 2026-03-29 06:01:21,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:01:21,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 06:01:21,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:01:21,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61d2c2cb62322ad96978b HTTP/1.1" 200 None 2026-03-29 06:01:21,657 - lava_callback - ERROR - Error processing callback for node 69c61d2c2cb62322ad96978b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34328 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:02:41,569 - lava_callback - INFO - Received callback for node 69c61d7e2cb62322ad969798 INFO: 172.20.0.1:34940 - "POST /node/69c61d7e2cb62322ad969798 HTTP/1.0" 202 Accepted 2026-03-29 06:02:41,575 - lava_callback - INFO - Starting async job submit for node 69c61d7e2cb62322ad969798 2026-03-29 06:02:41,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:02:41,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61d7e2cb62322ad969798 HTTP/1.1" 200 None 2026-03-29 06:02:41,591 - lava_callback - DEBUG - Processing node 69c61d7e2cb62322ad969798: baseline-arm64-baylibre 2026-03-29 06:02:41,591 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61d7e2cb62322ad969798 2026-03-29 06:02:41,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:02:41,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61d7e2cb62322ad969798 HTTP/1.1" 200 None 2026-03-29 06:02:41,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:02:41,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 06:02:41,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:02:41,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61d7e2cb62322ad969798 HTTP/1.1" 200 None 2026-03-29 06:02:41,698 - lava_callback - ERROR - Error processing callback for node 69c61d7e2cb62322ad969798: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56222 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:04:21,696 - lava_callback - INFO - Received callback for node 69c61dd02cb62322ad9697a1 INFO: 172.20.0.1:56260 - "POST /node/69c61dd02cb62322ad9697a1 HTTP/1.0" 202 Accepted 2026-03-29 06:04:21,701 - lava_callback - INFO - Starting async job submit for node 69c61dd02cb62322ad9697a1 2026-03-29 06:04:21,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:04:21,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61dd02cb62322ad9697a1 HTTP/1.1" 200 None 2026-03-29 06:04:21,720 - lava_callback - DEBUG - Processing node 69c61dd02cb62322ad9697a1: baseline-arm64-baylibre 2026-03-29 06:04:21,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61dd02cb62322ad9697a1 2026-03-29 06:04:21,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:04:21,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61dd02cb62322ad9697a1 HTTP/1.1" 200 None 2026-03-29 06:04:21,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:04:21,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 06:04:21,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:04:21,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61dd02cb62322ad9697a1 HTTP/1.1" 200 None 2026-03-29 06:04:21,785 - lava_callback - ERROR - Error processing callback for node 69c61dd02cb62322ad9697a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49936 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:05:37,211 - lava_callback - INFO - Received callback for node 69c61e282cb62322ad9697ad INFO: 172.20.0.1:49940 - "POST /node/69c61e282cb62322ad9697ad HTTP/1.0" 202 Accepted 2026-03-29 06:05:37,217 - lava_callback - INFO - Starting async job submit for node 69c61e282cb62322ad9697ad 2026-03-29 06:05:37,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:05:37,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61e282cb62322ad9697ad HTTP/1.1" 200 None 2026-03-29 06:05:37,239 - lava_callback - DEBUG - Processing node 69c61e282cb62322ad9697ad: baseline-arm64-clabbe 2026-03-29 06:05:37,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61e282cb62322ad9697ad 2026-03-29 06:05:37,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:05:37,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61e282cb62322ad9697ad HTTP/1.1" 200 None 2026-03-29 06:05:37,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:05:37,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 06:05:37,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:05:37,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61e282cb62322ad9697ad HTTP/1.1" 200 None 2026-03-29 06:05:37,318 - lava_callback - ERROR - Error processing callback for node 69c61e282cb62322ad9697ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38848 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:06:57,217 - lava_callback - INFO - Received callback for node 69c61e7a2cb62322ad9697be INFO: 172.20.0.1:38852 - "POST /node/69c61e7a2cb62322ad9697be HTTP/1.0" 202 Accepted 2026-03-29 06:06:57,223 - lava_callback - INFO - Starting async job submit for node 69c61e7a2cb62322ad9697be 2026-03-29 06:06:57,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:06:57,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61e7a2cb62322ad9697be HTTP/1.1" 200 None 2026-03-29 06:06:57,242 - lava_callback - DEBUG - Processing node 69c61e7a2cb62322ad9697be: baseline-arm64-clabbe 2026-03-29 06:06:57,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61e7a2cb62322ad9697be 2026-03-29 06:06:57,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:06:57,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61e7a2cb62322ad9697be HTTP/1.1" 200 None 2026-03-29 06:06:57,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:06:57,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 06:06:57,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:06:57,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61e7a2cb62322ad9697be HTTP/1.1" 200 None 2026-03-29 06:06:57,321 - lava_callback - ERROR - Error processing callback for node 69c61e7a2cb62322ad9697be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51110 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:08:17,228 - lava_callback - INFO - Received callback for node 69c61ecc2cb62322ad9697ce INFO: 172.20.0.1:51118 - "POST /node/69c61ecc2cb62322ad9697ce HTTP/1.0" 202 Accepted 2026-03-29 06:08:17,234 - lava_callback - INFO - Starting async job submit for node 69c61ecc2cb62322ad9697ce 2026-03-29 06:08:17,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:08:17,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61ecc2cb62322ad9697ce HTTP/1.1" 200 None 2026-03-29 06:08:17,250 - lava_callback - DEBUG - Processing node 69c61ecc2cb62322ad9697ce: baseline-arm64-clabbe 2026-03-29 06:08:17,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61ecc2cb62322ad9697ce 2026-03-29 06:08:17,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:08:17,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61ecc2cb62322ad9697ce HTTP/1.1" 200 None 2026-03-29 06:08:17,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:08:17,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 06:08:17,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:08:17,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61ecc2cb62322ad9697ce HTTP/1.1" 200 None 2026-03-29 06:08:17,315 - lava_callback - ERROR - Error processing callback for node 69c61ecc2cb62322ad9697ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34406 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:10:17,293 - lava_callback - INFO - Received callback for node 69c61f342cb62322ad9697da INFO: 172.20.0.1:34422 - "POST /node/69c61f342cb62322ad9697da HTTP/1.0" 202 Accepted 2026-03-29 06:10:17,298 - lava_callback - INFO - Starting async job submit for node 69c61f342cb62322ad9697da 2026-03-29 06:10:17,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:10:17,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61f342cb62322ad9697da HTTP/1.1" 200 None 2026-03-29 06:10:17,320 - lava_callback - DEBUG - Processing node 69c61f342cb62322ad9697da: baseline-arm64-clabbe 2026-03-29 06:10:17,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61f342cb62322ad9697da 2026-03-29 06:10:17,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:10:17,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61f342cb62322ad9697da HTTP/1.1" 200 None 2026-03-29 06:10:17,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:10:17,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 06:10:17,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:10:17,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61f342cb62322ad9697da HTTP/1.1" 200 None 2026-03-29 06:10:17,390 - lava_callback - ERROR - Error processing callback for node 69c61f342cb62322ad9697da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45704 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:11:37,429 - lava_callback - INFO - Received callback for node 69c61f852cb62322ad9697ed INFO: 172.20.0.1:45706 - "POST /node/69c61f852cb62322ad9697ed HTTP/1.0" 202 Accepted 2026-03-29 06:11:37,435 - lava_callback - INFO - Starting async job submit for node 69c61f852cb62322ad9697ed 2026-03-29 06:11:37,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:11:37,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61f852cb62322ad9697ed HTTP/1.1" 200 None 2026-03-29 06:11:37,451 - lava_callback - DEBUG - Processing node 69c61f852cb62322ad9697ed: baseline-arm64-clabbe 2026-03-29 06:11:37,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c61f852cb62322ad9697ed 2026-03-29 06:11:37,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:11:37,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c61f852cb62322ad9697ed HTTP/1.1" 200 None 2026-03-29 06:11:37,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:11:37,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-29 06:11:37,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:11:37,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c61f852cb62322ad9697ed HTTP/1.1" 200 None 2026-03-29 06:11:37,533 - lava_callback - ERROR - Error processing callback for node 69c61f852cb62322ad9697ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36516 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:35:23,456 - lava_callback - INFO - Received callback for node 69c625192cb62322ad96985d INFO: 172.20.0.1:53510 - "POST /node/69c625192cb62322ad96985d HTTP/1.0" 202 Accepted 2026-03-29 06:35:23,462 - lava_callback - INFO - Starting async job submit for node 69c625192cb62322ad96985d 2026-03-29 06:35:23,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:35:23,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c625192cb62322ad96985d HTTP/1.1" 200 None 2026-03-29 06:35:23,484 - lava_callback - DEBUG - Processing node 69c625192cb62322ad96985d: baseline-arm-baylibre 2026-03-29 06:35:23,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c625192cb62322ad96985d 2026-03-29 06:35:23,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:35:23,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c625192cb62322ad96985d HTTP/1.1" 200 None 2026-03-29 06:35:23,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:35:23,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 06:35:23,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:35:23,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c625192cb62322ad96985d HTTP/1.1" 200 None 2026-03-29 06:35:23,538 - lava_callback - ERROR - Error processing callback for node 69c625192cb62322ad96985d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34356 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:36:03,383 - lava_callback - INFO - Received callback for node 69c6253e2cb62322ad969868 INFO: 172.20.0.1:34358 - "POST /node/69c6253e2cb62322ad969868 HTTP/1.0" 202 Accepted 2026-03-29 06:36:03,389 - lava_callback - INFO - Starting async job submit for node 69c6253e2cb62322ad969868 2026-03-29 06:36:03,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:36:03,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6253e2cb62322ad969868 HTTP/1.1" 200 None 2026-03-29 06:36:03,405 - lava_callback - DEBUG - Processing node 69c6253e2cb62322ad969868: baseline-arm-baylibre 2026-03-29 06:36:03,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6253e2cb62322ad969868 2026-03-29 06:36:03,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:36:03,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6253e2cb62322ad969868 HTTP/1.1" 200 None 2026-03-29 06:36:03,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:36:03,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 06:36:03,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:36:03,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6253e2cb62322ad969868 HTTP/1.1" 200 None 2026-03-29 06:36:03,475 - lava_callback - ERROR - Error processing callback for node 69c6253e2cb62322ad969868: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47242 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:36:23,392 - lava_callback - INFO - Received callback for node 69c625622cb62322ad969874 INFO: 172.20.0.1:40514 - "POST /node/69c625622cb62322ad969874 HTTP/1.0" 202 Accepted 2026-03-29 06:36:23,397 - lava_callback - INFO - Starting async job submit for node 69c625622cb62322ad969874 2026-03-29 06:36:23,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:36:23,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c625622cb62322ad969874 HTTP/1.1" 200 None 2026-03-29 06:36:23,413 - lava_callback - DEBUG - Processing node 69c625622cb62322ad969874: baseline-arm-baylibre 2026-03-29 06:36:23,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c625622cb62322ad969874 2026-03-29 06:36:23,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:36:23,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c625622cb62322ad969874 HTTP/1.1" 200 None 2026-03-29 06:36:23,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:36:23,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 06:36:23,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:36:23,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c625622cb62322ad969874 HTTP/1.1" 200 None 2026-03-29 06:36:23,474 - lava_callback - ERROR - Error processing callback for node 69c625622cb62322ad969874: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46052 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:37:43,476 - lava_callback - INFO - Received callback for node 69c625ac2cb62322ad969886 INFO: 172.20.0.1:60146 - "POST /node/69c625ac2cb62322ad969886 HTTP/1.0" 202 Accepted 2026-03-29 06:37:43,481 - lava_callback - INFO - Starting async job submit for node 69c625ac2cb62322ad969886 2026-03-29 06:37:43,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:37:43,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c625ac2cb62322ad969886 HTTP/1.1" 200 None 2026-03-29 06:37:43,497 - lava_callback - DEBUG - Processing node 69c625ac2cb62322ad969886: baseline-arm-baylibre 2026-03-29 06:37:43,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c625ac2cb62322ad969886 2026-03-29 06:37:43,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:37:43,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c625ac2cb62322ad969886 HTTP/1.1" 200 None 2026-03-29 06:37:43,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:37:43,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 06:37:43,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:37:43,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c625ac2cb62322ad969886 HTTP/1.1" 200 None 2026-03-29 06:37:43,548 - lava_callback - ERROR - Error processing callback for node 69c625ac2cb62322ad969886: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47372 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:38:23,496 - lava_callback - INFO - Received callback for node 69c625d12cb62322ad969898 INFO: 172.20.0.1:54872 - "POST /node/69c625d12cb62322ad969898 HTTP/1.0" 202 Accepted 2026-03-29 06:38:23,502 - lava_callback - INFO - Starting async job submit for node 69c625d12cb62322ad969898 2026-03-29 06:38:23,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:38:23,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c625d12cb62322ad969898 HTTP/1.1" 200 None 2026-03-29 06:38:23,518 - lava_callback - DEBUG - Processing node 69c625d12cb62322ad969898: baseline-arm-baylibre 2026-03-29 06:38:23,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c625d12cb62322ad969898 2026-03-29 06:38:23,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:38:23,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c625d12cb62322ad969898 HTTP/1.1" 200 None 2026-03-29 06:38:23,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:38:23,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 06:38:23,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:38:23,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c625d12cb62322ad969898 HTTP/1.1" 200 None 2026-03-29 06:38:23,571 - lava_callback - ERROR - Error processing callback for node 69c625d12cb62322ad969898: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58380 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:40:18,485 - lava_callback - INFO - Received callback for node 69c6263f2cb62322ad9698bd INFO: 172.20.0.1:58388 - "POST /node/69c6263f2cb62322ad9698bd HTTP/1.0" 202 Accepted 2026-03-29 06:40:18,490 - lava_callback - INFO - Starting async job submit for node 69c6263f2cb62322ad9698bd 2026-03-29 06:40:18,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:40:18,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6263f2cb62322ad9698bd HTTP/1.1" 200 None 2026-03-29 06:40:18,505 - lava_callback - DEBUG - Processing node 69c6263f2cb62322ad9698bd: baseline-arm-clabbe 2026-03-29 06:40:18,505 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6263f2cb62322ad9698bd 2026-03-29 06:40:18,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:40:18,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6263f2cb62322ad9698bd HTTP/1.1" 200 None 2026-03-29 06:40:18,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:40:18,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 06:40:18,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:40:18,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6263f2cb62322ad9698bd HTTP/1.1" 200 None 2026-03-29 06:40:18,573 - lava_callback - ERROR - Error processing callback for node 69c6263f2cb62322ad9698bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48292 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:40:58,563 - lava_callback - INFO - Received callback for node 69c626652cb62322ad9698ca INFO: 172.20.0.1:48294 - "POST /node/69c626652cb62322ad9698ca HTTP/1.0" 202 Accepted 2026-03-29 06:40:58,568 - lava_callback - INFO - Starting async job submit for node 69c626652cb62322ad9698ca 2026-03-29 06:40:58,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:40:58,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c626652cb62322ad9698ca HTTP/1.1" 200 None 2026-03-29 06:40:58,584 - lava_callback - DEBUG - Processing node 69c626652cb62322ad9698ca: baseline-arm-clabbe 2026-03-29 06:40:58,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c626652cb62322ad9698ca 2026-03-29 06:40:58,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:40:58,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c626652cb62322ad9698ca HTTP/1.1" 200 None 2026-03-29 06:40:58,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:40:58,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 06:40:58,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:40:58,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c626652cb62322ad9698ca HTTP/1.1" 200 None 2026-03-29 06:40:58,642 - lava_callback - ERROR - Error processing callback for node 69c626652cb62322ad9698ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58992 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:41:18,646 - lava_callback - INFO - Received callback for node 69c6268a2cb62322ad9698d8 INFO: 172.20.0.1:59008 - "POST /node/69c6268a2cb62322ad9698d8 HTTP/1.0" 202 Accepted 2026-03-29 06:41:18,652 - lava_callback - INFO - Starting async job submit for node 69c6268a2cb62322ad9698d8 2026-03-29 06:41:18,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:41:18,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6268a2cb62322ad9698d8 HTTP/1.1" 200 None 2026-03-29 06:41:18,679 - lava_callback - DEBUG - Processing node 69c6268a2cb62322ad9698d8: baseline-arm-clabbe 2026-03-29 06:41:18,679 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6268a2cb62322ad9698d8 2026-03-29 06:41:18,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:41:18,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6268a2cb62322ad9698d8 HTTP/1.1" 200 None 2026-03-29 06:41:18,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:41:18,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 06:41:18,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:41:18,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6268a2cb62322ad9698d8 HTTP/1.1" 200 None 2026-03-29 06:41:18,730 - lava_callback - ERROR - Error processing callback for node 69c6268a2cb62322ad9698d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38244 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:41:58,615 - lava_callback - INFO - Received callback for node 69c626b02cb62322ad9698e8 INFO: 172.20.0.1:38258 - "POST /node/69c626b02cb62322ad9698e8 HTTP/1.0" 202 Accepted 2026-03-29 06:41:58,621 - lava_callback - INFO - Starting async job submit for node 69c626b02cb62322ad9698e8 2026-03-29 06:41:58,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:41:58,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c626b02cb62322ad9698e8 HTTP/1.1" 200 None 2026-03-29 06:41:58,636 - lava_callback - DEBUG - Processing node 69c626b02cb62322ad9698e8: baseline-arm-clabbe 2026-03-29 06:41:58,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c626b02cb62322ad9698e8 2026-03-29 06:41:58,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:41:58,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c626b02cb62322ad9698e8 HTTP/1.1" 200 None 2026-03-29 06:41:58,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:41:58,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 06:41:58,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:41:58,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c626b02cb62322ad9698e8 HTTP/1.1" 200 None 2026-03-29 06:41:58,696 - lava_callback - ERROR - Error processing callback for node 69c626b02cb62322ad9698e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54000 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:42:38,663 - lava_callback - INFO - Received callback for node 69c626d52cb62322ad9698f7 INFO: 172.20.0.1:54014 - "POST /node/69c626d52cb62322ad9698f7 HTTP/1.0" 202 Accepted 2026-03-29 06:42:38,668 - lava_callback - INFO - Starting async job submit for node 69c626d52cb62322ad9698f7 2026-03-29 06:42:38,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:42:38,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c626d52cb62322ad9698f7 HTTP/1.1" 200 None 2026-03-29 06:42:38,684 - lava_callback - DEBUG - Processing node 69c626d52cb62322ad9698f7: baseline-arm-clabbe 2026-03-29 06:42:38,684 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c626d52cb62322ad9698f7 2026-03-29 06:42:38,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:42:38,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c626d52cb62322ad9698f7 HTTP/1.1" 200 None 2026-03-29 06:42:38,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:42:38,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-03-29 06:42:38,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:42:38,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c626d52cb62322ad9698f7 HTTP/1.1" 200 None 2026-03-29 06:42:38,751 - lava_callback - ERROR - Error processing callback for node 69c626d52cb62322ad9698f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48482 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:48484 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:43:03,879 - lava_callback - INFO - Received callback for node 69c626f52cb62322ad969904 INFO: 172.20.0.1:48486 - "POST /node/69c626f52cb62322ad969904 HTTP/1.0" 202 Accepted 2026-03-29 06:43:03,885 - lava_callback - INFO - Starting async job submit for node 69c626f52cb62322ad969904 2026-03-29 06:43:03,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:43:03,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c626f52cb62322ad969904 HTTP/1.1" 200 None 2026-03-29 06:43:03,901 - lava_callback - DEBUG - Processing node 69c626f52cb62322ad969904: baseline-arm-baylibre 2026-03-29 06:43:03,902 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c626f52cb62322ad969904 2026-03-29 06:43:03,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:43:03,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c626f52cb62322ad969904 HTTP/1.1" 200 None 2026-03-29 06:43:03,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:43:03,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:43:03,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:43:03,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c626f52cb62322ad969904 HTTP/1.1" 200 None 2026-03-29 06:43:03,961 - lava_callback - ERROR - Error processing callback for node 69c626f52cb62322ad969904: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:43:43,759 - lava_callback - INFO - Received callback for node 69c627192cb62322ad96990f INFO: 172.20.0.1:51462 - "POST /node/69c627192cb62322ad96990f HTTP/1.0" 202 Accepted 2026-03-29 06:43:43,764 - lava_callback - INFO - Starting async job submit for node 69c627192cb62322ad96990f 2026-03-29 06:43:43,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:43:43,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627192cb62322ad96990f HTTP/1.1" 200 None 2026-03-29 06:43:43,780 - lava_callback - DEBUG - Processing node 69c627192cb62322ad96990f: baseline-arm-baylibre 2026-03-29 06:43:43,780 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c627192cb62322ad96990f 2026-03-29 06:43:43,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:43:43,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627192cb62322ad96990f HTTP/1.1" 200 None 2026-03-29 06:43:43,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:43:43,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:43:43,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:43:43,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c627192cb62322ad96990f HTTP/1.1" 200 None 2026-03-29 06:43:43,859 - lava_callback - ERROR - Error processing callback for node 69c627192cb62322ad96990f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46014 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:45:03,873 - lava_callback - INFO - Received callback for node 69c627622cb62322ad969926 INFO: 172.20.0.1:46022 - "POST /node/69c627622cb62322ad969926 HTTP/1.0" 202 Accepted 2026-03-29 06:45:03,879 - lava_callback - INFO - Starting async job submit for node 69c627622cb62322ad969926 2026-03-29 06:45:03,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:45:03,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627622cb62322ad969926 HTTP/1.1" 200 None 2026-03-29 06:45:03,894 - lava_callback - DEBUG - Processing node 69c627622cb62322ad969926: baseline-arm-baylibre 2026-03-29 06:45:03,895 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c627622cb62322ad969926 2026-03-29 06:45:03,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:45:03,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627622cb62322ad969926 HTTP/1.1" 200 None 2026-03-29 06:45:03,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:45:03,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:45:03,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:45:03,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c627622cb62322ad969926 HTTP/1.1" 200 None 2026-03-29 06:45:03,967 - lava_callback - ERROR - Error processing callback for node 69c627622cb62322ad969926: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50512 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:45:38,772 - lava_callback - INFO - Received callback for node 69c627882cb62322ad969934 INFO: 172.20.0.1:50516 - "POST /node/69c627882cb62322ad969934 HTTP/1.0" 202 Accepted 2026-03-29 06:45:38,778 - lava_callback - INFO - Starting async job submit for node 69c627882cb62322ad969934 2026-03-29 06:45:38,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:45:38,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627882cb62322ad969934 HTTP/1.1" 200 None 2026-03-29 06:45:38,794 - lava_callback - DEBUG - Processing node 69c627882cb62322ad969934: baseline-arm-clabbe 2026-03-29 06:45:38,794 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c627882cb62322ad969934 2026-03-29 06:45:38,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:45:38,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627882cb62322ad969934 HTTP/1.1" 200 None 2026-03-29 06:45:38,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:45:38,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:45:38,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:45:38,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c627882cb62322ad969934 HTTP/1.1" 200 None 2026-03-29 06:45:38,864 - lava_callback - ERROR - Error processing callback for node 69c627882cb62322ad969934: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50768 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:46:18,759 - lava_callback - INFO - Received callback for node 69c627ae2cb62322ad969942 INFO: 172.20.0.1:50770 - "POST /node/69c627ae2cb62322ad969942 HTTP/1.0" 202 Accepted 2026-03-29 06:46:18,765 - lava_callback - INFO - Starting async job submit for node 69c627ae2cb62322ad969942 2026-03-29 06:46:18,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:46:18,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627ae2cb62322ad969942 HTTP/1.1" 200 None 2026-03-29 06:46:18,786 - lava_callback - DEBUG - Processing node 69c627ae2cb62322ad969942: baseline-arm-clabbe 2026-03-29 06:46:18,786 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c627ae2cb62322ad969942 2026-03-29 06:46:18,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:46:18,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627ae2cb62322ad969942 HTTP/1.1" 200 None 2026-03-29 06:46:18,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:46:18,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:46:18,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:46:18,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c627ae2cb62322ad969942 HTTP/1.1" 200 None 2026-03-29 06:46:18,837 - lava_callback - ERROR - Error processing callback for node 69c627ae2cb62322ad969942: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35602 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:46:58,867 - lava_callback - INFO - Received callback for node 69c627d22cb62322ad969952 INFO: 172.20.0.1:35604 - "POST /node/69c627d22cb62322ad969952 HTTP/1.0" 202 Accepted 2026-03-29 06:46:58,872 - lava_callback - INFO - Starting async job submit for node 69c627d22cb62322ad969952 2026-03-29 06:46:58,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:46:58,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627d22cb62322ad969952 HTTP/1.1" 200 None 2026-03-29 06:46:58,888 - lava_callback - DEBUG - Processing node 69c627d22cb62322ad969952: baseline-arm-clabbe 2026-03-29 06:46:58,888 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c627d22cb62322ad969952 2026-03-29 06:46:58,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:46:58,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627d22cb62322ad969952 HTTP/1.1" 200 None 2026-03-29 06:46:58,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:46:58,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:46:58,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:46:58,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c627d22cb62322ad969952 HTTP/1.1" 200 None 2026-03-29 06:46:58,945 - lava_callback - ERROR - Error processing callback for node 69c627d22cb62322ad969952: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43872 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:47:38,821 - lava_callback - INFO - Received callback for node 69c627f82cb62322ad96995e INFO: 172.20.0.1:43884 - "POST /node/69c627f82cb62322ad96995e HTTP/1.0" 202 Accepted 2026-03-29 06:47:38,826 - lava_callback - INFO - Starting async job submit for node 69c627f82cb62322ad96995e 2026-03-29 06:47:38,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:47:38,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627f82cb62322ad96995e HTTP/1.1" 200 None 2026-03-29 06:47:38,842 - lava_callback - DEBUG - Processing node 69c627f82cb62322ad96995e: baseline-arm-clabbe 2026-03-29 06:47:38,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c627f82cb62322ad96995e 2026-03-29 06:47:38,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:47:38,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c627f82cb62322ad96995e HTTP/1.1" 200 None 2026-03-29 06:47:38,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:47:38,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:47:38,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:47:38,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c627f82cb62322ad96995e HTTP/1.1" 200 None 2026-03-29 06:47:38,903 - lava_callback - ERROR - Error processing callback for node 69c627f82cb62322ad96995e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60420 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:48:18,873 - lava_callback - INFO - Received callback for node 69c6281d2cb62322ad96996a INFO: 172.20.0.1:60426 - "POST /node/69c6281d2cb62322ad96996a HTTP/1.0" 202 Accepted 2026-03-29 06:48:18,878 - lava_callback - INFO - Starting async job submit for node 69c6281d2cb62322ad96996a 2026-03-29 06:48:18,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:48:18,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6281d2cb62322ad96996a HTTP/1.1" 200 None 2026-03-29 06:48:18,906 - lava_callback - DEBUG - Processing node 69c6281d2cb62322ad96996a: baseline-arm-clabbe 2026-03-29 06:48:18,906 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6281d2cb62322ad96996a 2026-03-29 06:48:18,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:48:18,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6281d2cb62322ad96996a HTTP/1.1" 200 None 2026-03-29 06:48:18,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:48:18,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:48:18,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:48:18,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6281d2cb62322ad96996a HTTP/1.1" 200 None 2026-03-29 06:48:18,959 - lava_callback - ERROR - Error processing callback for node 69c6281d2cb62322ad96996a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36168 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:48:38,853 - lava_callback - INFO - Received callback for node 69c628422cb62322ad969977 INFO: 172.20.0.1:36176 - "POST /node/69c628422cb62322ad969977 HTTP/1.0" 202 Accepted 2026-03-29 06:48:38,859 - lava_callback - INFO - Starting async job submit for node 69c628422cb62322ad969977 2026-03-29 06:48:38,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:48:38,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c628422cb62322ad969977 HTTP/1.1" 200 None 2026-03-29 06:48:38,881 - lava_callback - DEBUG - Processing node 69c628422cb62322ad969977: baseline-arm-clabbe 2026-03-29 06:48:38,881 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c628422cb62322ad969977 2026-03-29 06:48:38,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:48:38,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c628422cb62322ad969977 HTTP/1.1" 200 None 2026-03-29 06:48:38,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:48:38,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:48:38,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:48:38,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c628422cb62322ad969977 HTTP/1.1" 200 None 2026-03-29 06:48:38,937 - lava_callback - ERROR - Error processing callback for node 69c628422cb62322ad969977: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52720 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:49:18,944 - lava_callback - INFO - Received callback for node 69c628672cb62322ad969987 INFO: 172.20.0.1:52734 - "POST /node/69c628672cb62322ad969987 HTTP/1.0" 202 Accepted 2026-03-29 06:49:18,950 - lava_callback - INFO - Starting async job submit for node 69c628672cb62322ad969987 2026-03-29 06:49:18,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:49:18,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c628672cb62322ad969987 HTTP/1.1" 200 None 2026-03-29 06:49:18,966 - lava_callback - DEBUG - Processing node 69c628672cb62322ad969987: baseline-arm-clabbe 2026-03-29 06:49:18,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c628672cb62322ad969987 2026-03-29 06:49:18,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:49:18,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c628672cb62322ad969987 HTTP/1.1" 200 None 2026-03-29 06:49:18,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:49:18,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:49:18,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:49:19,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c628672cb62322ad969987 HTTP/1.1" 200 None 2026-03-29 06:49:19,030 - lava_callback - ERROR - Error processing callback for node 69c628672cb62322ad969987: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59294 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:49:58,904 - lava_callback - INFO - Received callback for node 69c6288b2cb62322ad969990 INFO: 172.20.0.1:59308 - "POST /node/69c6288b2cb62322ad969990 HTTP/1.0" 202 Accepted 2026-03-29 06:49:58,909 - lava_callback - INFO - Starting async job submit for node 69c6288b2cb62322ad969990 2026-03-29 06:49:58,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:49:58,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6288b2cb62322ad969990 HTTP/1.1" 200 None 2026-03-29 06:49:58,925 - lava_callback - DEBUG - Processing node 69c6288b2cb62322ad969990: baseline-arm-clabbe 2026-03-29 06:49:58,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6288b2cb62322ad969990 2026-03-29 06:49:58,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:49:58,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6288b2cb62322ad969990 HTTP/1.1" 200 None 2026-03-29 06:49:58,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:49:58,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:49:58,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:49:58,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6288b2cb62322ad969990 HTTP/1.1" 200 None 2026-03-29 06:49:58,977 - lava_callback - ERROR - Error processing callback for node 69c6288b2cb62322ad969990: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41428 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:50:39,007 - lava_callback - INFO - Received callback for node 69c628b02cb62322ad969999 INFO: 172.20.0.1:41430 - "POST /node/69c628b02cb62322ad969999 HTTP/1.0" 202 Accepted 2026-03-29 06:50:39,013 - lava_callback - INFO - Starting async job submit for node 69c628b02cb62322ad969999 2026-03-29 06:50:39,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:50:39,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c628b02cb62322ad969999 HTTP/1.1" 200 None 2026-03-29 06:50:39,029 - lava_callback - DEBUG - Processing node 69c628b02cb62322ad969999: baseline-arm-clabbe 2026-03-29 06:50:39,030 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c628b02cb62322ad969999 2026-03-29 06:50:39,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:50:39,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c628b02cb62322ad969999 HTTP/1.1" 200 None 2026-03-29 06:50:39,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:50:39,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:50:39,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:50:39,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c628b02cb62322ad969999 HTTP/1.1" 200 None 2026-03-29 06:50:39,092 - lava_callback - ERROR - Error processing callback for node 69c628b02cb62322ad969999: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44918 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:51:19,005 - lava_callback - INFO - Received callback for node 69c628d52cb62322ad9699a4 INFO: 172.20.0.1:44934 - "POST /node/69c628d52cb62322ad9699a4 HTTP/1.0" 202 Accepted 2026-03-29 06:51:19,013 - lava_callback - INFO - Starting async job submit for node 69c628d52cb62322ad9699a4 2026-03-29 06:51:19,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:51:19,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c628d52cb62322ad9699a4 HTTP/1.1" 200 None 2026-03-29 06:51:19,029 - lava_callback - DEBUG - Processing node 69c628d52cb62322ad9699a4: baseline-arm-clabbe 2026-03-29 06:51:19,029 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c628d52cb62322ad9699a4 2026-03-29 06:51:19,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:51:19,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c628d52cb62322ad9699a4 HTTP/1.1" 200 None 2026-03-29 06:51:19,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:51:19,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-29 06:51:19,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:51:19,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c628d52cb62322ad9699a4 HTTP/1.1" 200 None 2026-03-29 06:51:19,082 - lava_callback - ERROR - Error processing callback for node 69c628d52cb62322ad9699a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41554 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:51:44,270 - lava_callback - INFO - Received callback for node 69c628f42cb62322ad9699ad INFO: 172.20.0.1:59234 - "POST /node/69c628f42cb62322ad9699ad HTTP/1.0" 202 Accepted 2026-03-29 06:51:44,276 - lava_callback - INFO - Starting async job submit for node 69c628f42cb62322ad9699ad 2026-03-29 06:51:44,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:51:44,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c628f42cb62322ad9699ad HTTP/1.1" 200 None 2026-03-29 06:51:44,291 - lava_callback - DEBUG - Processing node 69c628f42cb62322ad9699ad: baseline-arm-baylibre 2026-03-29 06:51:44,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c628f42cb62322ad9699ad 2026-03-29 06:51:44,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:51:44,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c628f42cb62322ad9699ad HTTP/1.1" 200 None 2026-03-29 06:51:44,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:51:44,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 06:51:44,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:51:44,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c628f42cb62322ad9699ad HTTP/1.1" 200 None 2026-03-29 06:51:44,352 - lava_callback - ERROR - Error processing callback for node 69c628f42cb62322ad9699ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44620 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:52:24,289 - lava_callback - INFO - Received callback for node 69c629192cb62322ad9699b8 INFO: 172.20.0.1:58826 - "POST /node/69c629192cb62322ad9699b8 HTTP/1.0" 202 Accepted 2026-03-29 06:52:24,295 - lava_callback - INFO - Starting async job submit for node 69c629192cb62322ad9699b8 2026-03-29 06:52:24,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:52:24,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c629192cb62322ad9699b8 HTTP/1.1" 200 None 2026-03-29 06:52:24,310 - lava_callback - DEBUG - Processing node 69c629192cb62322ad9699b8: baseline-arm-baylibre 2026-03-29 06:52:24,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c629192cb62322ad9699b8 2026-03-29 06:52:24,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:52:24,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c629192cb62322ad9699b8 HTTP/1.1" 200 None 2026-03-29 06:52:24,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:52:24,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 06:52:24,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:52:24,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c629192cb62322ad9699b8 HTTP/1.1" 200 None 2026-03-29 06:52:24,383 - lava_callback - ERROR - Error processing callback for node 69c629192cb62322ad9699b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53006 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:53:04,444 - lava_callback - INFO - Received callback for node 69c6293d2cb62322ad9699c1 INFO: 172.20.0.1:53016 - "POST /node/69c6293d2cb62322ad9699c1 HTTP/1.0" 202 Accepted 2026-03-29 06:53:04,450 - lava_callback - INFO - Starting async job submit for node 69c6293d2cb62322ad9699c1 2026-03-29 06:53:04,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:53:04,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6293d2cb62322ad9699c1 HTTP/1.1" 200 None 2026-03-29 06:53:04,466 - lava_callback - DEBUG - Processing node 69c6293d2cb62322ad9699c1: baseline-arm-baylibre 2026-03-29 06:53:04,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6293d2cb62322ad9699c1 2026-03-29 06:53:04,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:53:04,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6293d2cb62322ad9699c1 HTTP/1.1" 200 None 2026-03-29 06:53:04,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:53:04,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 06:53:04,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:53:04,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6293d2cb62322ad9699c1 HTTP/1.1" 200 None 2026-03-29 06:53:04,529 - lava_callback - ERROR - Error processing callback for node 69c6293d2cb62322ad9699c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32956 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:53:24,321 - lava_callback - INFO - Received callback for node 69c629612cb62322ad9699cf INFO: 172.20.0.1:37456 - "POST /node/69c629612cb62322ad9699cf HTTP/1.0" 202 Accepted 2026-03-29 06:53:24,327 - lava_callback - INFO - Starting async job submit for node 69c629612cb62322ad9699cf 2026-03-29 06:53:24,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:53:24,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c629612cb62322ad9699cf HTTP/1.1" 200 None 2026-03-29 06:53:24,343 - lava_callback - DEBUG - Processing node 69c629612cb62322ad9699cf: baseline-arm-baylibre 2026-03-29 06:53:24,343 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c629612cb62322ad9699cf 2026-03-29 06:53:24,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:53:24,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c629612cb62322ad9699cf HTTP/1.1" 200 None 2026-03-29 06:53:24,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:53:24,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 06:53:24,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:53:24,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c629612cb62322ad9699cf HTTP/1.1" 200 None 2026-03-29 06:53:24,393 - lava_callback - ERROR - Error processing callback for node 69c629612cb62322ad9699cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33486 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:54:04,370 - lava_callback - INFO - Received callback for node 69c629862cb62322ad9699d8 INFO: 172.20.0.1:33496 - "POST /node/69c629862cb62322ad9699d8 HTTP/1.0" 202 Accepted 2026-03-29 06:54:04,376 - lava_callback - INFO - Starting async job submit for node 69c629862cb62322ad9699d8 2026-03-29 06:54:04,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:54:04,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c629862cb62322ad9699d8 HTTP/1.1" 200 None 2026-03-29 06:54:04,392 - lava_callback - DEBUG - Processing node 69c629862cb62322ad9699d8: baseline-arm-baylibre 2026-03-29 06:54:04,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c629862cb62322ad9699d8 2026-03-29 06:54:04,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:54:04,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c629862cb62322ad9699d8 HTTP/1.1" 200 None 2026-03-29 06:54:04,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:54:04,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 06:54:04,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:54:04,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c629862cb62322ad9699d8 HTTP/1.1" 200 None 2026-03-29 06:54:04,447 - lava_callback - ERROR - Error processing callback for node 69c629862cb62322ad9699d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50598 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:54:44,409 - lava_callback - INFO - Received callback for node 69c629aa2cb62322ad9699e2 INFO: 172.20.0.1:51846 - "POST /node/69c629aa2cb62322ad9699e2 HTTP/1.0" 202 Accepted 2026-03-29 06:54:44,415 - lava_callback - INFO - Starting async job submit for node 69c629aa2cb62322ad9699e2 2026-03-29 06:54:44,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:54:44,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c629aa2cb62322ad9699e2 HTTP/1.1" 200 None 2026-03-29 06:54:44,431 - lava_callback - DEBUG - Processing node 69c629aa2cb62322ad9699e2: baseline-arm-baylibre 2026-03-29 06:54:44,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c629aa2cb62322ad9699e2 2026-03-29 06:54:44,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:54:44,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c629aa2cb62322ad9699e2 HTTP/1.1" 200 None 2026-03-29 06:54:44,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:54:44,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 06:54:44,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:54:44,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c629aa2cb62322ad9699e2 HTTP/1.1" 200 None 2026-03-29 06:54:44,489 - lava_callback - ERROR - Error processing callback for node 69c629aa2cb62322ad9699e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45614 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:56:39,225 - lava_callback - INFO - Received callback for node 69c62a212cb62322ad969a13 INFO: 172.20.0.1:45628 - "POST /node/69c62a212cb62322ad969a13 HTTP/1.0" 202 Accepted 2026-03-29 06:56:39,231 - lava_callback - INFO - Starting async job submit for node 69c62a212cb62322ad969a13 2026-03-29 06:56:39,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:56:39,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62a212cb62322ad969a13 HTTP/1.1" 200 None 2026-03-29 06:56:39,251 - lava_callback - DEBUG - Processing node 69c62a212cb62322ad969a13: baseline-arm-clabbe 2026-03-29 06:56:39,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62a212cb62322ad969a13 2026-03-29 06:56:39,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:56:39,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62a212cb62322ad969a13 HTTP/1.1" 200 None 2026-03-29 06:56:39,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:56:39,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 06:56:39,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:56:39,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62a212cb62322ad969a13 HTTP/1.1" 200 None 2026-03-29 06:56:39,310 - lava_callback - ERROR - Error processing callback for node 69c62a212cb62322ad969a13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37796 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:57:19,182 - lava_callback - INFO - Received callback for node 69c62a462cb62322ad969a44 INFO: 172.20.0.1:37806 - "POST /node/69c62a462cb62322ad969a44 HTTP/1.0" 202 Accepted 2026-03-29 06:57:19,188 - lava_callback - INFO - Starting async job submit for node 69c62a462cb62322ad969a44 2026-03-29 06:57:19,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:57:19,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62a462cb62322ad969a44 HTTP/1.1" 200 None 2026-03-29 06:57:19,207 - lava_callback - DEBUG - Processing node 69c62a462cb62322ad969a44: baseline-arm-clabbe 2026-03-29 06:57:19,207 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62a462cb62322ad969a44 2026-03-29 06:57:19,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:57:19,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62a462cb62322ad969a44 HTTP/1.1" 200 None 2026-03-29 06:57:19,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:57:19,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 06:57:19,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:57:19,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62a462cb62322ad969a44 HTTP/1.1" 200 None 2026-03-29 06:57:19,264 - lava_callback - ERROR - Error processing callback for node 69c62a462cb62322ad969a44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59160 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:57:59,289 - lava_callback - INFO - Received callback for node 69c62a6a2cb62322ad969a4f INFO: 172.20.0.1:59164 - "POST /node/69c62a6a2cb62322ad969a4f HTTP/1.0" 202 Accepted 2026-03-29 06:57:59,295 - lava_callback - INFO - Starting async job submit for node 69c62a6a2cb62322ad969a4f 2026-03-29 06:57:59,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:57:59,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62a6a2cb62322ad969a4f HTTP/1.1" 200 None 2026-03-29 06:57:59,312 - lava_callback - DEBUG - Processing node 69c62a6a2cb62322ad969a4f: baseline-arm-clabbe 2026-03-29 06:57:59,312 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62a6a2cb62322ad969a4f 2026-03-29 06:57:59,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:57:59,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62a6a2cb62322ad969a4f HTTP/1.1" 200 None 2026-03-29 06:57:59,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:57:59,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 06:57:59,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:57:59,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62a6a2cb62322ad969a4f HTTP/1.1" 200 None 2026-03-29 06:57:59,378 - lava_callback - ERROR - Error processing callback for node 69c62a6a2cb62322ad969a4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47514 - "GET / HTTP/1.0" 200 OK 2026-03-29 06:58:39,267 - lava_callback - INFO - Received callback for node 69c62a902cb62322ad969a5d INFO: 172.20.0.1:47516 - "POST /node/69c62a902cb62322ad969a5d HTTP/1.0" 202 Accepted 2026-03-29 06:58:39,275 - lava_callback - INFO - Starting async job submit for node 69c62a902cb62322ad969a5d 2026-03-29 06:58:39,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:58:39,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62a902cb62322ad969a5d HTTP/1.1" 200 None 2026-03-29 06:58:39,293 - lava_callback - DEBUG - Processing node 69c62a902cb62322ad969a5d: baseline-arm-clabbe 2026-03-29 06:58:39,293 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62a902cb62322ad969a5d 2026-03-29 06:58:39,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:58:39,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62a902cb62322ad969a5d HTTP/1.1" 200 None 2026-03-29 06:58:39,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:58:39,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 06:58:39,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 06:58:39,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62a902cb62322ad969a5d HTTP/1.1" 200 None 2026-03-29 06:58:39,350 - lava_callback - ERROR - Error processing callback for node 69c62a902cb62322ad969a5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41450 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:04:25,016 - lava_callback - INFO - Received callback for node 69c62bf72cb62322ad969aa9 INFO: 172.20.0.1:36670 - "POST /node/69c62bf72cb62322ad969aa9 HTTP/1.0" 202 Accepted 2026-03-29 07:04:25,024 - lava_callback - INFO - Starting async job submit for node 69c62bf72cb62322ad969aa9 2026-03-29 07:04:25,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:04:25,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62bf72cb62322ad969aa9 HTTP/1.1" 200 None 2026-03-29 07:04:25,039 - lava_callback - DEBUG - Processing node 69c62bf72cb62322ad969aa9: baseline-arm-baylibre 2026-03-29 07:04:25,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62bf72cb62322ad969aa9 2026-03-29 07:04:25,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:04:25,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62bf72cb62322ad969aa9 HTTP/1.1" 200 None 2026-03-29 07:04:25,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:04:25,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:04:25,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:04:25,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62bf72cb62322ad969aa9 HTTP/1.1" 200 None 2026-03-29 07:04:25,097 - lava_callback - ERROR - Error processing callback for node 69c62bf72cb62322ad969aa9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57436 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:06:45,064 - lava_callback - INFO - Received callback for node 69c62c772cb62322ad969aca INFO: 172.20.0.1:38870 - "POST /node/69c62c772cb62322ad969aca HTTP/1.0" 202 Accepted 2026-03-29 07:06:45,070 - lava_callback - INFO - Starting async job submit for node 69c62c772cb62322ad969aca 2026-03-29 07:06:45,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:06:45,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62c772cb62322ad969aca HTTP/1.1" 200 None 2026-03-29 07:06:45,086 - lava_callback - DEBUG - Processing node 69c62c772cb62322ad969aca: baseline-arm64-baylibre 2026-03-29 07:06:45,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62c772cb62322ad969aca 2026-03-29 07:06:45,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:06:45,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62c772cb62322ad969aca HTTP/1.1" 200 None 2026-03-29 07:06:45,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:06:45,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 07:06:45,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:06:45,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62c772cb62322ad969aca HTTP/1.1" 200 None 2026-03-29 07:06:45,141 - lava_callback - ERROR - Error processing callback for node 69c62c772cb62322ad969aca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60062 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:08:05,170 - lava_callback - INFO - Received callback for node 69c62cc82cb62322ad969ada INFO: 172.20.0.1:60066 - "POST /node/69c62cc82cb62322ad969ada HTTP/1.0" 202 Accepted 2026-03-29 07:08:05,175 - lava_callback - INFO - Starting async job submit for node 69c62cc82cb62322ad969ada 2026-03-29 07:08:05,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:08:05,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62cc82cb62322ad969ada HTTP/1.1" 200 None 2026-03-29 07:08:05,191 - lava_callback - DEBUG - Processing node 69c62cc82cb62322ad969ada: baseline-arm64-baylibre 2026-03-29 07:08:05,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62cc82cb62322ad969ada 2026-03-29 07:08:05,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:08:05,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62cc82cb62322ad969ada HTTP/1.1" 200 None 2026-03-29 07:08:05,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:08:05,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 07:08:05,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:08:05,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62cc82cb62322ad969ada HTTP/1.1" 200 None 2026-03-29 07:08:05,248 - lava_callback - ERROR - Error processing callback for node 69c62cc82cb62322ad969ada: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59330 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:09:19,771 - lava_callback - INFO - Received callback for node 69c62d132cb62322ad969ae3 INFO: 172.20.0.1:56662 - "POST /node/69c62d132cb62322ad969ae3 HTTP/1.0" 202 Accepted 2026-03-29 07:09:19,776 - lava_callback - INFO - Starting async job submit for node 69c62d132cb62322ad969ae3 2026-03-29 07:09:19,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:09:19,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62d132cb62322ad969ae3 HTTP/1.1" 200 None 2026-03-29 07:09:19,792 - lava_callback - DEBUG - Processing node 69c62d132cb62322ad969ae3: baseline-arm-clabbe 2026-03-29 07:09:19,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62d132cb62322ad969ae3 2026-03-29 07:09:19,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:09:19,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62d132cb62322ad969ae3 HTTP/1.1" 200 None 2026-03-29 07:09:19,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:09:19,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:09:19,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:09:19,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62d132cb62322ad969ae3 HTTP/1.1" 200 None 2026-03-29 07:09:19,851 - lava_callback - ERROR - Error processing callback for node 69c62d132cb62322ad969ae3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60098 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:11:25,460 - lava_callback - INFO - Received callback for node 69c62d902cb62322ad969afb INFO: 172.20.0.1:56844 - "POST /node/69c62d902cb62322ad969afb HTTP/1.0" 202 Accepted 2026-03-29 07:11:25,465 - lava_callback - INFO - Starting async job submit for node 69c62d902cb62322ad969afb 2026-03-29 07:11:25,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:11:25,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62d902cb62322ad969afb HTTP/1.1" 200 None 2026-03-29 07:11:25,501 - lava_callback - DEBUG - Processing node 69c62d902cb62322ad969afb: baseline-arm64-baylibre 2026-03-29 07:11:25,502 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62d902cb62322ad969afb 2026-03-29 07:11:25,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:11:25,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62d902cb62322ad969afb HTTP/1.1" 200 None 2026-03-29 07:11:25,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:11:25,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 07:11:25,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:11:25,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62d902cb62322ad969afb HTTP/1.1" 200 None 2026-03-29 07:11:25,555 - lava_callback - ERROR - Error processing callback for node 69c62d902cb62322ad969afb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42848 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:12:25,433 - lava_callback - INFO - Received callback for node 69c62dd42cb62322ad969b02 INFO: 172.20.0.1:35576 - "POST /node/69c62dd42cb62322ad969b02 HTTP/1.0" 202 Accepted 2026-03-29 07:12:25,439 - lava_callback - INFO - Starting async job submit for node 69c62dd42cb62322ad969b02 2026-03-29 07:12:25,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:12:25,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62dd42cb62322ad969b02 HTTP/1.1" 200 None 2026-03-29 07:12:25,457 - lava_callback - DEBUG - Processing node 69c62dd42cb62322ad969b02: baseline-arm-baylibre 2026-03-29 07:12:25,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62dd42cb62322ad969b02 2026-03-29 07:12:25,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:12:25,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62dd42cb62322ad969b02 HTTP/1.1" 200 None 2026-03-29 07:12:25,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:12:25,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:12:25,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:12:25,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62dd42cb62322ad969b02 HTTP/1.1" 200 None 2026-03-29 07:12:25,525 - lava_callback - ERROR - Error processing callback for node 69c62dd42cb62322ad969b02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38796 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:14:40,027 - lava_callback - INFO - Received callback for node 69c62e5c2cb62322ad969b1b INFO: 172.20.0.1:36902 - "POST /node/69c62e5c2cb62322ad969b1b HTTP/1.0" 202 Accepted 2026-03-29 07:14:40,033 - lava_callback - INFO - Starting async job submit for node 69c62e5c2cb62322ad969b1b 2026-03-29 07:14:40,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:14:40,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62e5c2cb62322ad969b1b HTTP/1.1" 200 None 2026-03-29 07:14:40,049 - lava_callback - DEBUG - Processing node 69c62e5c2cb62322ad969b1b: baseline-arm64-clabbe 2026-03-29 07:14:40,049 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62e5c2cb62322ad969b1b 2026-03-29 07:14:40,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:14:40,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62e5c2cb62322ad969b1b HTTP/1.1" 200 None 2026-03-29 07:14:40,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:14:40,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 07:14:40,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:14:40,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62e5c2cb62322ad969b1b HTTP/1.1" 200 None 2026-03-29 07:14:40,107 - lava_callback - ERROR - Error processing callback for node 69c62e5c2cb62322ad969b1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET /node/69981876e4d71f8580ba2537%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34778 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:16:00,029 - lava_callback - INFO - Received callback for node 69c62eac2cb62322ad969b28 INFO: 172.20.0.1:56332 - "POST /node/69c62eac2cb62322ad969b28 HTTP/1.0" 202 Accepted 2026-03-29 07:16:00,034 - lava_callback - INFO - Starting async job submit for node 69c62eac2cb62322ad969b28 2026-03-29 07:16:00,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:16:00,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62eac2cb62322ad969b28 HTTP/1.1" 200 None 2026-03-29 07:16:00,054 - lava_callback - DEBUG - Processing node 69c62eac2cb62322ad969b28: baseline-arm64-clabbe 2026-03-29 07:16:00,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62eac2cb62322ad969b28 2026-03-29 07:16:00,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:16:00,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62eac2cb62322ad969b28 HTTP/1.1" 200 None 2026-03-29 07:16:00,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:16:00,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 07:16:00,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:16:00,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62eac2cb62322ad969b28 HTTP/1.1" 200 None 2026-03-29 07:16:00,107 - lava_callback - ERROR - Error processing callback for node 69c62eac2cb62322ad969b28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37948 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:17:40,069 - lava_callback - INFO - Received callback for node 69c62efe2cb62322ad969b35 INFO: 172.20.0.1:55456 - "POST /node/69c62efe2cb62322ad969b35 HTTP/1.0" 202 Accepted 2026-03-29 07:17:40,075 - lava_callback - INFO - Starting async job submit for node 69c62efe2cb62322ad969b35 2026-03-29 07:17:40,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:17:40,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62efe2cb62322ad969b35 HTTP/1.1" 200 None 2026-03-29 07:17:40,097 - lava_callback - DEBUG - Processing node 69c62efe2cb62322ad969b35: baseline-arm64-clabbe 2026-03-29 07:17:40,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62efe2cb62322ad969b35 2026-03-29 07:17:40,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:17:40,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62efe2cb62322ad969b35 HTTP/1.1" 200 None 2026-03-29 07:17:40,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:17:40,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 07:17:40,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:17:40,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62efe2cb62322ad969b35 HTTP/1.1" 200 None 2026-03-29 07:17:40,149 - lava_callback - ERROR - Error processing callback for node 69c62efe2cb62322ad969b35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41736 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:19:00,175 - lava_callback - INFO - Received callback for node 69c62f502cb62322ad969b40 INFO: 172.20.0.1:48502 - "POST /node/69c62f502cb62322ad969b40 HTTP/1.0" 202 Accepted 2026-03-29 07:19:00,181 - lava_callback - INFO - Starting async job submit for node 69c62f502cb62322ad969b40 2026-03-29 07:19:00,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:19:00,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62f502cb62322ad969b40 HTTP/1.1" 200 None 2026-03-29 07:19:00,197 - lava_callback - DEBUG - Processing node 69c62f502cb62322ad969b40: baseline-arm64-clabbe 2026-03-29 07:19:00,197 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62f502cb62322ad969b40 2026-03-29 07:19:00,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:19:00,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62f502cb62322ad969b40 HTTP/1.1" 200 None 2026-03-29 07:19:00,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:19:00,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 07:19:00,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:19:00,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62f502cb62322ad969b40 HTTP/1.1" 200 None 2026-03-29 07:19:00,250 - lava_callback - ERROR - Error processing callback for node 69c62f502cb62322ad969b40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34622 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:20:20,194 - lava_callback - INFO - Received callback for node 69c62fa12cb62322ad969b46 INFO: 172.20.0.1:57200 - "POST /node/69c62fa12cb62322ad969b46 HTTP/1.0" 202 Accepted 2026-03-29 07:20:20,200 - lava_callback - INFO - Starting async job submit for node 69c62fa12cb62322ad969b46 2026-03-29 07:20:20,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:20:20,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62fa12cb62322ad969b46 HTTP/1.1" 200 None 2026-03-29 07:20:20,216 - lava_callback - DEBUG - Processing node 69c62fa12cb62322ad969b46: baseline-arm64-clabbe 2026-03-29 07:20:20,216 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62fa12cb62322ad969b46 2026-03-29 07:20:20,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:20:20,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62fa12cb62322ad969b46 HTTP/1.1" 200 None 2026-03-29 07:20:20,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:20:20,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 07:20:20,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:20:20,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62fa12cb62322ad969b46 HTTP/1.1" 200 None 2026-03-29 07:20:20,288 - lava_callback - ERROR - Error processing callback for node 69c62fa12cb62322ad969b46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46790 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:21:40,234 - lava_callback - INFO - Received callback for node 69c62ff52cb62322ad969b51 INFO: 172.20.0.1:33706 - "POST /node/69c62ff52cb62322ad969b51 HTTP/1.0" 202 Accepted 2026-03-29 07:21:40,239 - lava_callback - INFO - Starting async job submit for node 69c62ff52cb62322ad969b51 2026-03-29 07:21:40,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:21:40,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62ff52cb62322ad969b51 HTTP/1.1" 200 None 2026-03-29 07:21:40,256 - lava_callback - DEBUG - Processing node 69c62ff52cb62322ad969b51: baseline-arm64-clabbe 2026-03-29 07:21:40,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c62ff52cb62322ad969b51 2026-03-29 07:21:40,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:21:40,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62ff52cb62322ad969b51 HTTP/1.1" 200 None 2026-03-29 07:21:40,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:21:40,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-29 07:21:40,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:21:40,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62ff52cb62322ad969b51 HTTP/1.1" 200 None 2026-03-29 07:21:40,308 - lava_callback - ERROR - Error processing callback for node 69c62ff52cb62322ad969b51: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33728 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40596 - "GET /security.txt HTTP/1.0" 404 Not Found INFO: 172.20.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41412 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:22:40,289 - lava_callback - INFO - Received callback for node 69c630382cb62322ad969b58 INFO: 172.20.0.1:52020 - "POST /node/69c630382cb62322ad969b58 HTTP/1.0" 202 Accepted 2026-03-29 07:22:40,295 - lava_callback - INFO - Starting async job submit for node 69c630382cb62322ad969b58 2026-03-29 07:22:40,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:22:40,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c630382cb62322ad969b58 HTTP/1.1" 200 None 2026-03-29 07:22:40,323 - lava_callback - DEBUG - Processing node 69c630382cb62322ad969b58: baseline-arm-clabbe 2026-03-29 07:22:40,324 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c630382cb62322ad969b58 2026-03-29 07:22:40,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:22:40,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c630382cb62322ad969b58 HTTP/1.1" 200 None 2026-03-29 07:22:40,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:22:40,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:22:40,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:22:40,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c630382cb62322ad969b58 HTTP/1.1" 200 None 2026-03-29 07:22:40,387 - lava_callback - ERROR - Error processing callback for node 69c630382cb62322ad969b58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46904 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:23:26,018 - lava_callback - INFO - Received callback for node 69c630592cb62322ad969b65 INFO: 172.20.0.1:60678 - "POST /node/69c630592cb62322ad969b65 HTTP/1.0" 202 Accepted 2026-03-29 07:23:26,023 - lava_callback - INFO - Starting async job submit for node 69c630592cb62322ad969b65 2026-03-29 07:23:26,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:23:26,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c630592cb62322ad969b65 HTTP/1.1" 200 None 2026-03-29 07:23:26,040 - lava_callback - DEBUG - Processing node 69c630592cb62322ad969b65: baseline-arm-baylibre 2026-03-29 07:23:26,040 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c630592cb62322ad969b65 2026-03-29 07:23:26,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:23:26,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c630592cb62322ad969b65 HTTP/1.1" 200 None 2026-03-29 07:23:26,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:23:26,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:23:26,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:23:26,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c630592cb62322ad969b65 HTTP/1.1" 200 None 2026-03-29 07:23:26,101 - lava_callback - ERROR - Error processing callback for node 69c630592cb62322ad969b65: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52100 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:24:00,323 - lava_callback - INFO - Received callback for node 69c630842cb62322ad969b71 INFO: 172.20.0.1:58064 - "POST /node/69c630842cb62322ad969b71 HTTP/1.0" 202 Accepted 2026-03-29 07:24:00,329 - lava_callback - INFO - Starting async job submit for node 69c630842cb62322ad969b71 2026-03-29 07:24:00,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:24:00,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c630842cb62322ad969b71 HTTP/1.1" 200 None 2026-03-29 07:24:00,344 - lava_callback - DEBUG - Processing node 69c630842cb62322ad969b71: baseline-arm-clabbe 2026-03-29 07:24:00,345 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c630842cb62322ad969b71 2026-03-29 07:24:00,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:24:00,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c630842cb62322ad969b71 HTTP/1.1" 200 None 2026-03-29 07:24:00,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:24:00,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:24:00,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:24:00,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c630842cb62322ad969b71 HTTP/1.1" 200 None 2026-03-29 07:24:00,405 - lava_callback - ERROR - Error processing callback for node 69c630842cb62322ad969b71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51190 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:24:40,305 - lava_callback - INFO - Received callback for node 69c630aa2cb62322ad969b80 INFO: 172.20.0.1:45736 - "POST /node/69c630aa2cb62322ad969b80 HTTP/1.0" 202 Accepted 2026-03-29 07:24:40,311 - lava_callback - INFO - Starting async job submit for node 69c630aa2cb62322ad969b80 2026-03-29 07:24:40,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:24:40,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c630aa2cb62322ad969b80 HTTP/1.1" 200 None 2026-03-29 07:24:40,327 - lava_callback - DEBUG - Processing node 69c630aa2cb62322ad969b80: baseline-arm-clabbe 2026-03-29 07:24:40,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c630aa2cb62322ad969b80 2026-03-29 07:24:40,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:24:40,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c630aa2cb62322ad969b80 HTTP/1.1" 200 None 2026-03-29 07:24:40,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:24:40,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:24:40,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:24:40,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c630aa2cb62322ad969b80 HTTP/1.1" 200 None 2026-03-29 07:24:40,378 - lava_callback - ERROR - Error processing callback for node 69c630aa2cb62322ad969b80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39340 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:26:20,449 - lava_callback - INFO - Received callback for node 69c631192cb62322ad969b9d INFO: 172.20.0.1:37652 - "POST /node/69c631192cb62322ad969b9d HTTP/1.0" 202 Accepted 2026-03-29 07:26:20,454 - lava_callback - INFO - Starting async job submit for node 69c631192cb62322ad969b9d 2026-03-29 07:26:20,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:26:20,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c631192cb62322ad969b9d HTTP/1.1" 200 None 2026-03-29 07:26:20,471 - lava_callback - DEBUG - Processing node 69c631192cb62322ad969b9d: baseline-arm-clabbe 2026-03-29 07:26:20,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c631192cb62322ad969b9d 2026-03-29 07:26:20,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:26:20,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c631192cb62322ad969b9d HTTP/1.1" 200 None 2026-03-29 07:26:20,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:26:20,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:26:20,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:26:20,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c631192cb62322ad969b9d HTTP/1.1" 200 None 2026-03-29 07:26:20,545 - lava_callback - ERROR - Error processing callback for node 69c631192cb62322ad969b9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46316 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:29:00,546 - lava_callback - INFO - Received callback for node 69c631b02cb62322ad969bbf INFO: 172.20.0.1:53614 - "POST /node/69c631b02cb62322ad969bbf HTTP/1.0" 202 Accepted 2026-03-29 07:29:00,551 - lava_callback - INFO - Starting async job submit for node 69c631b02cb62322ad969bbf 2026-03-29 07:29:00,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:29:00,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c631b02cb62322ad969bbf HTTP/1.1" 200 None 2026-03-29 07:29:00,568 - lava_callback - DEBUG - Processing node 69c631b02cb62322ad969bbf: baseline-arm-clabbe 2026-03-29 07:29:00,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c631b02cb62322ad969bbf 2026-03-29 07:29:00,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:29:00,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c631b02cb62322ad969bbf HTTP/1.1" 200 None 2026-03-29 07:29:00,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:29:00,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:29:00,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:29:00,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c631b02cb62322ad969bbf HTTP/1.1" 200 None 2026-03-29 07:29:00,631 - lava_callback - ERROR - Error processing callback for node 69c631b02cb62322ad969bbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41460 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:29:40,526 - lava_callback - INFO - Received callback for node 69c631d52cb62322ad969bce INFO: 172.20.0.1:34878 - "POST /node/69c631d52cb62322ad969bce HTTP/1.0" 202 Accepted 2026-03-29 07:29:40,531 - lava_callback - INFO - Starting async job submit for node 69c631d52cb62322ad969bce 2026-03-29 07:29:40,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:29:40,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c631d52cb62322ad969bce HTTP/1.1" 200 None 2026-03-29 07:29:40,547 - lava_callback - DEBUG - Processing node 69c631d52cb62322ad969bce: baseline-arm-clabbe 2026-03-29 07:29:40,547 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c631d52cb62322ad969bce 2026-03-29 07:29:40,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:29:40,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c631d52cb62322ad969bce HTTP/1.1" 200 None 2026-03-29 07:29:40,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:29:40,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:29:40,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:29:40,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c631d52cb62322ad969bce HTTP/1.1" 200 None 2026-03-29 07:29:40,604 - lava_callback - ERROR - Error processing callback for node 69c631d52cb62322ad969bce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44256 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:30:20,576 - lava_callback - INFO - Received callback for node 69c631fa2cb62322ad969bdd INFO: 172.20.0.1:47470 - "POST /node/69c631fa2cb62322ad969bdd HTTP/1.0" 202 Accepted 2026-03-29 07:30:20,582 - lava_callback - INFO - Starting async job submit for node 69c631fa2cb62322ad969bdd 2026-03-29 07:30:20,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:30:20,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c631fa2cb62322ad969bdd HTTP/1.1" 200 None 2026-03-29 07:30:20,604 - lava_callback - DEBUG - Processing node 69c631fa2cb62322ad969bdd: baseline-arm-clabbe 2026-03-29 07:30:20,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c631fa2cb62322ad969bdd 2026-03-29 07:30:20,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:30:20,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c631fa2cb62322ad969bdd HTTP/1.1" 200 None 2026-03-29 07:30:20,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:30:20,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:30:20,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:30:20,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c631fa2cb62322ad969bdd HTTP/1.1" 200 None 2026-03-29 07:30:20,692 - lava_callback - ERROR - Error processing callback for node 69c631fa2cb62322ad969bdd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45470 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:31:00,561 - lava_callback - INFO - Received callback for node 69c6321f2cb62322ad969beb INFO: 172.20.0.1:37458 - "POST /node/69c6321f2cb62322ad969beb HTTP/1.0" 202 Accepted 2026-03-29 07:31:00,567 - lava_callback - INFO - Starting async job submit for node 69c6321f2cb62322ad969beb 2026-03-29 07:31:00,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:31:00,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6321f2cb62322ad969beb HTTP/1.1" 200 None 2026-03-29 07:31:00,587 - lava_callback - DEBUG - Processing node 69c6321f2cb62322ad969beb: baseline-arm-clabbe 2026-03-29 07:31:00,587 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6321f2cb62322ad969beb 2026-03-29 07:31:00,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:31:00,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6321f2cb62322ad969beb HTTP/1.1" 200 None 2026-03-29 07:31:00,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:31:00,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:31:00,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:31:00,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6321f2cb62322ad969beb HTTP/1.1" 200 None 2026-03-29 07:31:00,649 - lava_callback - ERROR - Error processing callback for node 69c6321f2cb62322ad969beb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41484 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:31:20,648 - lava_callback - INFO - Received callback for node 69c632442cb62322ad969bfc INFO: 172.20.0.1:47440 - "POST /node/69c632442cb62322ad969bfc HTTP/1.0" 202 Accepted 2026-03-29 07:31:20,654 - lava_callback - INFO - Starting async job submit for node 69c632442cb62322ad969bfc 2026-03-29 07:31:20,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:31:20,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c632442cb62322ad969bfc HTTP/1.1" 200 None 2026-03-29 07:31:20,670 - lava_callback - DEBUG - Processing node 69c632442cb62322ad969bfc: baseline-arm-clabbe 2026-03-29 07:31:20,671 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c632442cb62322ad969bfc 2026-03-29 07:31:20,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:31:20,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c632442cb62322ad969bfc HTTP/1.1" 200 None 2026-03-29 07:31:20,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:31:20,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:31:20,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:31:20,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c632442cb62322ad969bfc HTTP/1.1" 200 None 2026-03-29 07:31:20,726 - lava_callback - ERROR - Error processing callback for node 69c632442cb62322ad969bfc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44266 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:32:00,613 - lava_callback - INFO - Received callback for node 69c632692cb62322ad969c08 INFO: 172.20.0.1:40352 - "POST /node/69c632692cb62322ad969c08 HTTP/1.0" 202 Accepted 2026-03-29 07:32:00,619 - lava_callback - INFO - Starting async job submit for node 69c632692cb62322ad969c08 2026-03-29 07:32:00,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:32:00,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c632692cb62322ad969c08 HTTP/1.1" 200 None 2026-03-29 07:32:00,635 - lava_callback - DEBUG - Processing node 69c632692cb62322ad969c08: baseline-arm-clabbe 2026-03-29 07:32:00,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c632692cb62322ad969c08 2026-03-29 07:32:00,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:32:00,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c632692cb62322ad969c08 HTTP/1.1" 200 None 2026-03-29 07:32:00,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:32:00,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-29 07:32:00,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:32:00,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c632692cb62322ad969c08 HTTP/1.1" 200 None 2026-03-29 07:32:00,729 - lava_callback - ERROR - Error processing callback for node 69c632692cb62322ad969c08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50228 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:34:06,595 - lava_callback - INFO - Received callback for node 69c632e42cb62322ad969c23 INFO: 172.20.0.1:50230 - "POST /node/69c632e42cb62322ad969c23 HTTP/1.0" 202 Accepted 2026-03-29 07:34:06,601 - lava_callback - INFO - Starting async job submit for node 69c632e42cb62322ad969c23 2026-03-29 07:34:06,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:34:06,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c632e42cb62322ad969c23 HTTP/1.1" 200 None 2026-03-29 07:34:06,617 - lava_callback - DEBUG - Processing node 69c632e42cb62322ad969c23: baseline-arm64-baylibre 2026-03-29 07:34:06,618 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c632e42cb62322ad969c23 2026-03-29 07:34:06,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:34:06,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c632e42cb62322ad969c23 HTTP/1.1" 200 None 2026-03-29 07:34:06,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:34:06,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:34:06,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:34:06,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c632e42cb62322ad969c23 HTTP/1.1" 200 None 2026-03-29 07:34:06,697 - lava_callback - ERROR - Error processing callback for node 69c632e42cb62322ad969c23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50022 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:37:06,736 - lava_callback - INFO - Received callback for node 69c6338f2cb62322ad969c3f INFO: 172.20.0.1:50028 - "POST /node/69c6338f2cb62322ad969c3f HTTP/1.0" 202 Accepted 2026-03-29 07:37:06,742 - lava_callback - INFO - Starting async job submit for node 69c6338f2cb62322ad969c3f 2026-03-29 07:37:06,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:37:06,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6338f2cb62322ad969c3f HTTP/1.1" 200 None 2026-03-29 07:37:06,759 - lava_callback - DEBUG - Processing node 69c6338f2cb62322ad969c3f: baseline-arm64-baylibre 2026-03-29 07:37:06,759 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6338f2cb62322ad969c3f 2026-03-29 07:37:06,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:37:06,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6338f2cb62322ad969c3f HTTP/1.1" 200 None 2026-03-29 07:37:06,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:37:06,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:37:06,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:37:06,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6338f2cb62322ad969c3f HTTP/1.1" 200 None 2026-03-29 07:37:06,831 - lava_callback - ERROR - Error processing callback for node 69c6338f2cb62322ad969c3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:39:46,922 - lava_callback - INFO - Received callback for node 69c634312cb62322ad969c53 INFO: 172.20.0.1:55330 - "POST /node/69c634312cb62322ad969c53 HTTP/1.0" 202 Accepted 2026-03-29 07:39:46,928 - lava_callback - INFO - Starting async job submit for node 69c634312cb62322ad969c53 2026-03-29 07:39:46,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:39:46,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c634312cb62322ad969c53 HTTP/1.1" 200 None 2026-03-29 07:39:46,944 - lava_callback - DEBUG - Processing node 69c634312cb62322ad969c53: baseline-arm64-baylibre 2026-03-29 07:39:46,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c634312cb62322ad969c53 2026-03-29 07:39:46,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:39:46,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c634312cb62322ad969c53 HTTP/1.1" 200 None 2026-03-29 07:39:46,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:39:46,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:39:46,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:39:46,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c634312cb62322ad969c53 HTTP/1.1" 200 None 2026-03-29 07:39:47,002 - lava_callback - ERROR - Error processing callback for node 69c634312cb62322ad969c53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54768 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:41:06,960 - lava_callback - INFO - Received callback for node 69c634802cb62322ad969c63 INFO: 172.20.0.1:54776 - "POST /node/69c634802cb62322ad969c63 HTTP/1.0" 202 Accepted 2026-03-29 07:41:06,965 - lava_callback - INFO - Starting async job submit for node 69c634802cb62322ad969c63 2026-03-29 07:41:06,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:41:06,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c634802cb62322ad969c63 HTTP/1.1" 200 None 2026-03-29 07:41:06,985 - lava_callback - DEBUG - Processing node 69c634802cb62322ad969c63: baseline-arm64-baylibre 2026-03-29 07:41:06,985 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c634802cb62322ad969c63 2026-03-29 07:41:06,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:41:06,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c634802cb62322ad969c63 HTTP/1.1" 200 None 2026-03-29 07:41:07,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:41:07,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:41:07,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:41:07,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c634802cb62322ad969c63 HTTP/1.1" 200 None 2026-03-29 07:41:07,037 - lava_callback - ERROR - Error processing callback for node 69c634802cb62322ad969c63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37468 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:42:27,082 - lava_callback - INFO - Received callback for node 69c634d42cb62322ad969c75 INFO: 172.20.0.1:37480 - "POST /node/69c634d42cb62322ad969c75 HTTP/1.0" 202 Accepted 2026-03-29 07:42:27,087 - lava_callback - INFO - Starting async job submit for node 69c634d42cb62322ad969c75 2026-03-29 07:42:27,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:42:27,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c634d42cb62322ad969c75 HTTP/1.1" 200 None 2026-03-29 07:42:27,103 - lava_callback - DEBUG - Processing node 69c634d42cb62322ad969c75: baseline-arm64-baylibre 2026-03-29 07:42:27,104 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c634d42cb62322ad969c75 2026-03-29 07:42:27,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:42:27,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c634d42cb62322ad969c75 HTTP/1.1" 200 None 2026-03-29 07:42:27,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:42:27,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:42:27,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:42:27,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c634d42cb62322ad969c75 HTTP/1.1" 200 None 2026-03-29 07:42:27,163 - lava_callback - ERROR - Error processing callback for node 69c634d42cb62322ad969c75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57462 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:45:07,367 - lava_callback - INFO - Received callback for node 69c635772cb62322ad969c8b INFO: 172.20.0.1:57466 - "POST /node/69c635772cb62322ad969c8b HTTP/1.0" 202 Accepted 2026-03-29 07:45:07,373 - lava_callback - INFO - Starting async job submit for node 69c635772cb62322ad969c8b 2026-03-29 07:45:07,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:45:07,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c635772cb62322ad969c8b HTTP/1.1" 200 None 2026-03-29 07:45:07,388 - lava_callback - DEBUG - Processing node 69c635772cb62322ad969c8b: baseline-arm64-baylibre 2026-03-29 07:45:07,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c635772cb62322ad969c8b 2026-03-29 07:45:07,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:45:07,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c635772cb62322ad969c8b HTTP/1.1" 200 None 2026-03-29 07:45:07,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:45:07,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:45:07,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:45:07,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c635772cb62322ad969c8b HTTP/1.1" 200 None 2026-03-29 07:45:07,449 - lava_callback - ERROR - Error processing callback for node 69c635772cb62322ad969c8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54632 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:46:27,281 - lava_callback - INFO - Received callback for node 69c635c72cb62322ad969c98 INFO: 172.20.0.1:54636 - "POST /node/69c635c72cb62322ad969c98 HTTP/1.0" 202 Accepted 2026-03-29 07:46:27,287 - lava_callback - INFO - Starting async job submit for node 69c635c72cb62322ad969c98 2026-03-29 07:46:27,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:46:27,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c635c72cb62322ad969c98 HTTP/1.1" 200 None 2026-03-29 07:46:27,304 - lava_callback - DEBUG - Processing node 69c635c72cb62322ad969c98: baseline-arm64-baylibre 2026-03-29 07:46:27,305 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c635c72cb62322ad969c98 2026-03-29 07:46:27,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:46:27,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c635c72cb62322ad969c98 HTTP/1.1" 200 None 2026-03-29 07:46:27,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:46:27,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:46:27,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:46:27,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c635c72cb62322ad969c98 HTTP/1.1" 200 None 2026-03-29 07:46:27,358 - lava_callback - ERROR - Error processing callback for node 69c635c72cb62322ad969c98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40906 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:47:47,428 - lava_callback - INFO - Received callback for node 69c636172cb62322ad969ca0 INFO: 172.20.0.1:40914 - "POST /node/69c636172cb62322ad969ca0 HTTP/1.0" 202 Accepted 2026-03-29 07:47:47,434 - lava_callback - INFO - Starting async job submit for node 69c636172cb62322ad969ca0 2026-03-29 07:47:47,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:47:47,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c636172cb62322ad969ca0 HTTP/1.1" 200 None 2026-03-29 07:47:47,450 - lava_callback - DEBUG - Processing node 69c636172cb62322ad969ca0: baseline-arm64-baylibre 2026-03-29 07:47:47,450 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c636172cb62322ad969ca0 2026-03-29 07:47:47,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:47:47,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c636172cb62322ad969ca0 HTTP/1.1" 200 None 2026-03-29 07:47:47,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:47:47,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:47:47,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:47:47,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c636172cb62322ad969ca0 HTTP/1.1" 200 None 2026-03-29 07:47:47,508 - lava_callback - ERROR - Error processing callback for node 69c636172cb62322ad969ca0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42660 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:49:07,427 - lava_callback - INFO - Received callback for node 69c636662cb62322ad969cac INFO: 172.20.0.1:42666 - "POST /node/69c636662cb62322ad969cac HTTP/1.0" 202 Accepted 2026-03-29 07:49:07,433 - lava_callback - INFO - Starting async job submit for node 69c636662cb62322ad969cac 2026-03-29 07:49:07,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:49:07,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c636662cb62322ad969cac HTTP/1.1" 200 None 2026-03-29 07:49:07,449 - lava_callback - DEBUG - Processing node 69c636662cb62322ad969cac: baseline-arm64-baylibre 2026-03-29 07:49:07,449 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c636662cb62322ad969cac 2026-03-29 07:49:07,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:49:07,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c636662cb62322ad969cac HTTP/1.1" 200 None 2026-03-29 07:49:07,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:49:07,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:49:07,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:49:07,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c636662cb62322ad969cac HTTP/1.1" 200 None 2026-03-29 07:49:07,507 - lava_callback - ERROR - Error processing callback for node 69c636662cb62322ad969cac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45458 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:50:41,448 - lava_callback - INFO - Received callback for node 69c636be2cb62322ad969cb9 INFO: 172.20.0.1:50102 - "POST /node/69c636be2cb62322ad969cb9 HTTP/1.0" 202 Accepted 2026-03-29 07:50:41,454 - lava_callback - INFO - Starting async job submit for node 69c636be2cb62322ad969cb9 2026-03-29 07:50:41,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:50:41,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c636be2cb62322ad969cb9 HTTP/1.1" 200 None 2026-03-29 07:50:41,489 - lava_callback - DEBUG - Processing node 69c636be2cb62322ad969cb9: baseline-arm64-clabbe 2026-03-29 07:50:41,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c636be2cb62322ad969cb9 2026-03-29 07:50:41,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:50:41,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c636be2cb62322ad969cb9 HTTP/1.1" 200 None 2026-03-29 07:50:41,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:50:41,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:50:41,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:50:41,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c636be2cb62322ad969cb9 HTTP/1.1" 200 None 2026-03-29 07:50:41,544 - lava_callback - ERROR - Error processing callback for node 69c636be2cb62322ad969cb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58452 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:51:47,618 - lava_callback - INFO - Received callback for node 69c637092cb62322ad969cc1 INFO: 172.20.0.1:58454 - "POST /node/69c637092cb62322ad969cc1 HTTP/1.0" 202 Accepted 2026-03-29 07:51:47,624 - lava_callback - INFO - Starting async job submit for node 69c637092cb62322ad969cc1 2026-03-29 07:51:47,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:51:47,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c637092cb62322ad969cc1 HTTP/1.1" 200 None 2026-03-29 07:51:47,652 - lava_callback - DEBUG - Processing node 69c637092cb62322ad969cc1: baseline-arm64-baylibre 2026-03-29 07:51:47,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c637092cb62322ad969cc1 2026-03-29 07:51:47,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:51:47,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c637092cb62322ad969cc1 HTTP/1.1" 200 None 2026-03-29 07:51:47,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:51:47,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:51:47,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:51:47,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c637092cb62322ad969cc1 HTTP/1.1" 200 None 2026-03-29 07:51:47,716 - lava_callback - ERROR - Error processing callback for node 69c637092cb62322ad969cc1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54976 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:53:21,560 - lava_callback - INFO - Received callback for node 69c6375d2cb62322ad969ccf INFO: 172.20.0.1:46926 - "POST /node/69c6375d2cb62322ad969ccf HTTP/1.0" 202 Accepted 2026-03-29 07:53:21,566 - lava_callback - INFO - Starting async job submit for node 69c6375d2cb62322ad969ccf 2026-03-29 07:53:21,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:53:21,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6375d2cb62322ad969ccf HTTP/1.1" 200 None 2026-03-29 07:53:21,584 - lava_callback - DEBUG - Processing node 69c6375d2cb62322ad969ccf: baseline-arm64-clabbe 2026-03-29 07:53:21,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6375d2cb62322ad969ccf 2026-03-29 07:53:21,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:53:21,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6375d2cb62322ad969ccf HTTP/1.1" 200 None 2026-03-29 07:53:21,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:53:21,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:53:21,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:53:21,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6375d2cb62322ad969ccf HTTP/1.1" 200 None 2026-03-29 07:53:21,646 - lava_callback - ERROR - Error processing callback for node 69c6375d2cb62322ad969ccf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53292 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:54:41,559 - lava_callback - INFO - Received callback for node 69c637ab2cb62322ad969cdc INFO: 172.20.0.1:49424 - "POST /node/69c637ab2cb62322ad969cdc HTTP/1.0" 202 Accepted 2026-03-29 07:54:41,565 - lava_callback - INFO - Starting async job submit for node 69c637ab2cb62322ad969cdc 2026-03-29 07:54:41,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:54:41,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c637ab2cb62322ad969cdc HTTP/1.1" 200 None 2026-03-29 07:54:41,588 - lava_callback - DEBUG - Processing node 69c637ab2cb62322ad969cdc: baseline-arm64-clabbe 2026-03-29 07:54:41,588 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c637ab2cb62322ad969cdc 2026-03-29 07:54:41,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:54:41,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c637ab2cb62322ad969cdc HTTP/1.1" 200 None 2026-03-29 07:54:41,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:54:41,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:54:41,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:54:41,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c637ab2cb62322ad969cdc HTTP/1.1" 200 None 2026-03-29 07:54:41,643 - lava_callback - ERROR - Error processing callback for node 69c637ab2cb62322ad969cdc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52206 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:56:01,647 - lava_callback - INFO - Received callback for node 69c637fb2cb62322ad969ce6 INFO: 172.20.0.1:45972 - "POST /node/69c637fb2cb62322ad969ce6 HTTP/1.0" 202 Accepted 2026-03-29 07:56:01,652 - lava_callback - INFO - Starting async job submit for node 69c637fb2cb62322ad969ce6 2026-03-29 07:56:01,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:56:01,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c637fb2cb62322ad969ce6 HTTP/1.1" 200 None 2026-03-29 07:56:01,689 - lava_callback - DEBUG - Processing node 69c637fb2cb62322ad969ce6: baseline-arm64-clabbe 2026-03-29 07:56:01,690 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c637fb2cb62322ad969ce6 2026-03-29 07:56:01,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:56:01,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c637fb2cb62322ad969ce6 HTTP/1.1" 200 None 2026-03-29 07:56:01,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:56:01,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:56:01,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:56:01,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c637fb2cb62322ad969ce6 HTTP/1.1" 200 None 2026-03-29 07:56:01,745 - lava_callback - ERROR - Error processing callback for node 69c637fb2cb62322ad969ce6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49652 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:57:21,672 - lava_callback - INFO - Received callback for node 69c6384b2cb62322ad969cf1 INFO: 172.20.0.1:47818 - "POST /node/69c6384b2cb62322ad969cf1 HTTP/1.0" 202 Accepted 2026-03-29 07:57:21,677 - lava_callback - INFO - Starting async job submit for node 69c6384b2cb62322ad969cf1 2026-03-29 07:57:21,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:57:21,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6384b2cb62322ad969cf1 HTTP/1.1" 200 None 2026-03-29 07:57:21,694 - lava_callback - DEBUG - Processing node 69c6384b2cb62322ad969cf1: baseline-arm64-clabbe 2026-03-29 07:57:21,694 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6384b2cb62322ad969cf1 2026-03-29 07:57:21,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:57:21,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6384b2cb62322ad969cf1 HTTP/1.1" 200 None 2026-03-29 07:57:21,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:57:21,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:57:21,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:57:21,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6384b2cb62322ad969cf1 HTTP/1.1" 200 None 2026-03-29 07:57:21,770 - lava_callback - ERROR - Error processing callback for node 69c6384b2cb62322ad969cf1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45092 - "GET / HTTP/1.0" 200 OK 2026-03-29 07:58:41,773 - lava_callback - INFO - Received callback for node 69c6389b2cb62322ad969cff INFO: 172.20.0.1:47970 - "POST /node/69c6389b2cb62322ad969cff HTTP/1.0" 202 Accepted 2026-03-29 07:58:41,778 - lava_callback - INFO - Starting async job submit for node 69c6389b2cb62322ad969cff 2026-03-29 07:58:41,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:58:41,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6389b2cb62322ad969cff HTTP/1.1" 200 None 2026-03-29 07:58:41,800 - lava_callback - DEBUG - Processing node 69c6389b2cb62322ad969cff: baseline-arm64-clabbe 2026-03-29 07:58:41,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6389b2cb62322ad969cff 2026-03-29 07:58:41,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:58:41,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6389b2cb62322ad969cff HTTP/1.1" 200 None 2026-03-29 07:58:41,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:58:41,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 07:58:41,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 07:58:41,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6389b2cb62322ad969cff HTTP/1.1" 200 None 2026-03-29 07:58:41,876 - lava_callback - ERROR - Error processing callback for node 69c6389b2cb62322ad969cff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49120 - "GET / HTTP/1.0" 200 OK 2026-03-29 08:00:02,167 - lava_callback - INFO - Received callback for node 69c638ed2cb62322ad969d0a INFO: 172.20.0.1:37396 - "POST /node/69c638ed2cb62322ad969d0a HTTP/1.0" 202 Accepted 2026-03-29 08:00:02,172 - lava_callback - INFO - Starting async job submit for node 69c638ed2cb62322ad969d0a 2026-03-29 08:00:02,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:00:02,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c638ed2cb62322ad969d0a HTTP/1.1" 200 None 2026-03-29 08:00:02,192 - lava_callback - DEBUG - Processing node 69c638ed2cb62322ad969d0a: baseline-arm64-clabbe 2026-03-29 08:00:02,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c638ed2cb62322ad969d0a 2026-03-29 08:00:02,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:00:02,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c638ed2cb62322ad969d0a HTTP/1.1" 200 None 2026-03-29 08:00:02,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:00:02,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 08:00:02,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:00:02,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c638ed2cb62322ad969d0a HTTP/1.1" 200 None 2026-03-29 08:00:02,252 - lava_callback - ERROR - Error processing callback for node 69c638ed2cb62322ad969d0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40440 - "GET / HTTP/1.0" 200 OK 2026-03-29 08:01:21,984 - lava_callback - INFO - Received callback for node 69c6393f2cb62322ad969d11 INFO: 172.20.0.1:43174 - "POST /node/69c6393f2cb62322ad969d11 HTTP/1.0" 202 Accepted 2026-03-29 08:01:21,990 - lava_callback - INFO - Starting async job submit for node 69c6393f2cb62322ad969d11 2026-03-29 08:01:21,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:01:22,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6393f2cb62322ad969d11 HTTP/1.1" 200 None 2026-03-29 08:01:22,017 - lava_callback - DEBUG - Processing node 69c6393f2cb62322ad969d11: baseline-arm64-clabbe 2026-03-29 08:01:22,017 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c6393f2cb62322ad969d11 2026-03-29 08:01:22,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:01:22,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6393f2cb62322ad969d11 HTTP/1.1" 200 None 2026-03-29 08:01:22,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:01:22,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 08:01:22,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:01:22,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6393f2cb62322ad969d11 HTTP/1.1" 200 None 2026-03-29 08:01:22,076 - lava_callback - ERROR - Error processing callback for node 69c6393f2cb62322ad969d11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52616 - "GET / HTTP/1.0" 200 OK 2026-03-29 08:02:41,915 - lava_callback - INFO - Received callback for node 69c639912cb62322ad969d1d INFO: 172.20.0.1:51600 - "POST /node/69c639912cb62322ad969d1d HTTP/1.0" 202 Accepted 2026-03-29 08:02:41,920 - lava_callback - INFO - Starting async job submit for node 69c639912cb62322ad969d1d 2026-03-29 08:02:41,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:02:41,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c639912cb62322ad969d1d HTTP/1.1" 200 None 2026-03-29 08:02:41,936 - lava_callback - DEBUG - Processing node 69c639912cb62322ad969d1d: baseline-arm64-clabbe 2026-03-29 08:02:41,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c639912cb62322ad969d1d 2026-03-29 08:02:41,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:02:41,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c639912cb62322ad969d1d HTTP/1.1" 200 None 2026-03-29 08:02:41,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:02:41,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 08:02:41,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:02:41,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c639912cb62322ad969d1d HTTP/1.1" 200 None 2026-03-29 08:02:41,999 - lava_callback - ERROR - Error processing callback for node 69c639912cb62322ad969d1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK 2026-03-29 08:04:01,970 - lava_callback - INFO - Received callback for node 69c639e22cb62322ad969d28 INFO: 172.20.0.1:58322 - "POST /node/69c639e22cb62322ad969d28 HTTP/1.0" 202 Accepted 2026-03-29 08:04:01,976 - lava_callback - INFO - Starting async job submit for node 69c639e22cb62322ad969d28 2026-03-29 08:04:01,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:04:01,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c639e22cb62322ad969d28 HTTP/1.1" 200 None 2026-03-29 08:04:01,993 - lava_callback - DEBUG - Processing node 69c639e22cb62322ad969d28: baseline-arm64-clabbe 2026-03-29 08:04:01,993 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c639e22cb62322ad969d28 2026-03-29 08:04:01,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:04:02,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c639e22cb62322ad969d28 HTTP/1.1" 200 None 2026-03-29 08:04:02,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:04:02,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 08:04:02,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:04:02,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c639e22cb62322ad969d28 HTTP/1.1" 200 None 2026-03-29 08:04:02,046 - lava_callback - ERROR - Error processing callback for node 69c639e22cb62322ad969d28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57128 - "GET / HTTP/1.0" 200 OK 2026-03-29 08:05:22,019 - lava_callback - INFO - Received callback for node 69c63a322cb62322ad969d2f INFO: 172.20.0.1:39276 - "POST /node/69c63a322cb62322ad969d2f HTTP/1.0" 202 Accepted 2026-03-29 08:05:22,025 - lava_callback - INFO - Starting async job submit for node 69c63a322cb62322ad969d2f 2026-03-29 08:05:22,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:05:22,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63a322cb62322ad969d2f HTTP/1.1" 200 None 2026-03-29 08:05:22,041 - lava_callback - DEBUG - Processing node 69c63a322cb62322ad969d2f: baseline-arm64-clabbe 2026-03-29 08:05:22,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c63a322cb62322ad969d2f 2026-03-29 08:05:22,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:05:22,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63a322cb62322ad969d2f HTTP/1.1" 200 None 2026-03-29 08:05:22,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:05:22,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 08:05:22,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:05:22,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c63a322cb62322ad969d2f HTTP/1.1" 200 None 2026-03-29 08:05:22,115 - lava_callback - ERROR - Error processing callback for node 69c63a322cb62322ad969d2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43862 - "GET / HTTP/1.0" 200 OK 2026-03-29 08:06:42,102 - lava_callback - INFO - Received callback for node 69c63a832cb62322ad969d3a INFO: 172.20.0.1:48636 - "POST /node/69c63a832cb62322ad969d3a HTTP/1.0" 202 Accepted 2026-03-29 08:06:42,108 - lava_callback - INFO - Starting async job submit for node 69c63a832cb62322ad969d3a 2026-03-29 08:06:42,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:06:42,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63a832cb62322ad969d3a HTTP/1.1" 200 None 2026-03-29 08:06:42,124 - lava_callback - DEBUG - Processing node 69c63a832cb62322ad969d3a: baseline-arm64-clabbe 2026-03-29 08:06:42,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c63a832cb62322ad969d3a 2026-03-29 08:06:42,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:06:42,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63a832cb62322ad969d3a HTTP/1.1" 200 None 2026-03-29 08:06:42,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:06:42,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-29 08:06:42,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-29 08:06:42,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c63a832cb62322ad969d3a HTTP/1.1" 200 None 2026-03-29 08:06:42,213 - lava_callback - ERROR - Error processing callback for node 69c63a832cb62322ad969d3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38276 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]