INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:38056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49940 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:12:49,892 - lava_callback - INFO - Received callback for node 69c08576c3d73288b0d08f21 2026-03-25 00:12:49,899 - lava_callback - INFO - Starting async job submit for node 69c08576c3d73288b0d08f21 INFO: 172.21.0.1:45544 - "POST /node/69c08576c3d73288b0d08f21 HTTP/1.0" 202 Accepted 2026-03-25 00:12:49,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:12:49,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08576c3d73288b0d08f21 HTTP/1.1" 200 None 2026-03-25 00:12:49,931 - lava_callback - DEBUG - Processing node 69c08576c3d73288b0d08f21: baseline-arm64-baylibre 2026-03-25 00:12:49,932 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08576c3d73288b0d08f21 2026-03-25 00:12:49,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:12:49,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08576c3d73288b0d08f21 HTTP/1.1" 200 None 2026-03-25 00:12:49,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:12:49,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-25 00:12:49,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:12:49,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08576c3d73288b0d08f21 HTTP/1.1" 200 None 2026-03-25 00:12:49,996 - lava_callback - ERROR - Error processing callback for node 69c08576c3d73288b0d08f21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43382 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:33:23,793 - lava_callback - INFO - Received callback for node 69c32d4618a3717aa65d0210 INFO: 172.21.0.1:53052 - "POST /node/69c32d4618a3717aa65d0210 HTTP/1.0" 202 Accepted 2026-03-25 00:33:23,799 - lava_callback - INFO - Starting async job submit for node 69c32d4618a3717aa65d0210 2026-03-25 00:33:23,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:33:23,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4618a3717aa65d0210 HTTP/1.1" 200 None 2026-03-25 00:33:23,818 - lava_callback - DEBUG - Processing node 69c32d4618a3717aa65d0210: baseline-arm 2026-03-25 00:33:23,824 - lava_callback - WARNING - Failed to upload log 2026-03-25 00:33:23,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:33:23,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:33:23,987 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4618a3717aa65d0210/lava-logs.yaml 2026-03-25 00:33:23,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:33:24,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:33:24,124 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4618a3717aa65d0210/lava_callback.json.gz 2026-03-25 00:33:24,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:33:24,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4618a3717aa65d0210 HTTP/1.1" 200 None 2026-03-25 00:33:24,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:33:24,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:33:24,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:33:24,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d4618a3717aa65d0210 HTTP/1.1" 200 None 2026-03-25 00:33:24,208 - lava_callback - INFO - Completed processing callback for node 69c32d4618a3717aa65d0210 INFO: 172.21.0.1:53062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46606 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:33:50,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:33:50,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38810 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:34:00,220 - lava_callback - INFO - Received callback for node 69c32d4a18a3717aa65d021c INFO: 172.21.0.1:42918 - "POST /node/69c32d4a18a3717aa65d021c HTTP/1.0" 202 Accepted 2026-03-25 00:34:00,270 - lava_callback - INFO - Starting async job submit for node 69c32d4a18a3717aa65d021c 2026-03-25 00:34:00,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:00,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4a18a3717aa65d021c HTTP/1.1" 200 None 2026-03-25 00:34:00,303 - lava_callback - DEBUG - Processing node 69c32d4a18a3717aa65d021c: baseline-arm 2026-03-25 00:34:00,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:00,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:00,565 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4a18a3717aa65d021c/log.txt.gz 2026-03-25 00:34:00,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:00,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:00,810 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4a18a3717aa65d021c/lava-logs.yaml 2026-03-25 00:34:00,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:00,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:00,932 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4a18a3717aa65d021c/lava_callback.json.gz 2026-03-25 00:34:00,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:00,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4a18a3717aa65d021c HTTP/1.1" 200 None 2026-03-25 00:34:00,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:00,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:34:00,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:01,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d4a18a3717aa65d021c HTTP/1.1" 200 None 2026-03-25 00:34:01,012 - lava_callback - INFO - Completed processing callback for node 69c32d4a18a3717aa65d021c 2026-03-25 00:34:04,228 - lava_callback - INFO - Received callback for node 69c32d4718a3717aa65d0213 INFO: 172.21.0.1:42920 - "POST /node/69c32d4718a3717aa65d0213 HTTP/1.0" 202 Accepted 2026-03-25 00:34:04,278 - lava_callback - INFO - Starting async job submit for node 69c32d4718a3717aa65d0213 2026-03-25 00:34:04,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:04,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4718a3717aa65d0213 HTTP/1.1" 200 None 2026-03-25 00:34:04,312 - lava_callback - DEBUG - Processing node 69c32d4718a3717aa65d0213: baseline-arm 2026-03-25 00:34:04,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:04,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:04,549 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4718a3717aa65d0213/log.txt.gz 2026-03-25 00:34:04,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:04,775 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:04,777 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4718a3717aa65d0213/lava-logs.yaml 2026-03-25 00:34:04,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:04,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:04,959 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4718a3717aa65d0213/lava_callback.json.gz 2026-03-25 00:34:04,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:04,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4718a3717aa65d0213 HTTP/1.1" 200 None 2026-03-25 00:34:04,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:05,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:34:05,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:05,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d4718a3717aa65d0213 HTTP/1.1" 200 None 2026-03-25 00:34:05,045 - lava_callback - INFO - Completed processing callback for node 69c32d4718a3717aa65d0213 INFO: 172.21.0.1:42936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42932 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:34:20,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:20,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-25 00:34:26,446 - lava_callback - INFO - Received callback for node 69c32d4918a3717aa65d0219 INFO: 172.21.0.1:60760 - "POST /node/69c32d4918a3717aa65d0219 HTTP/1.0" 202 Accepted 2026-03-25 00:34:26,452 - lava_callback - INFO - Starting async job submit for node 69c32d4918a3717aa65d0219 2026-03-25 00:34:26,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:26,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4918a3717aa65d0219 HTTP/1.1" 200 None 2026-03-25 00:34:26,490 - lava_callback - DEBUG - Processing node 69c32d4918a3717aa65d0219: baseline-arm INFO: 172.21.0.1:60762 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:34:26,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:26,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:26,825 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4918a3717aa65d0219/log.txt.gz 2026-03-25 00:34:26,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:27,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:27,134 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4918a3717aa65d0219/lava-logs.yaml 2026-03-25 00:34:27,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:27,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:27,330 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4918a3717aa65d0219/lava_callback.json.gz 2026-03-25 00:34:27,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:27,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4918a3717aa65d0219 HTTP/1.1" 200 None 2026-03-25 00:34:27,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:27,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:34:27,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:27,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d4918a3717aa65d0219 HTTP/1.1" 200 None 2026-03-25 00:34:27,427 - lava_callback - INFO - Completed processing callback for node 69c32d4918a3717aa65d0219 2026-03-25 00:34:29,279 - lava_callback - INFO - Received callback for node 69c32d4618a3717aa65d020d INFO: 172.21.0.1:60768 - "POST /node/69c32d4618a3717aa65d020d HTTP/1.0" 202 Accepted 2026-03-25 00:34:29,294 - lava_callback - INFO - Starting async job submit for node 69c32d4618a3717aa65d020d 2026-03-25 00:34:29,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:29,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4618a3717aa65d020d HTTP/1.1" 200 None 2026-03-25 00:34:29,350 - lava_callback - DEBUG - Processing node 69c32d4618a3717aa65d020d: baseline-arm 2026-03-25 00:34:29,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:29,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:29,680 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4618a3717aa65d020d/log.txt.gz 2026-03-25 00:34:29,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:29,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:29,972 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4618a3717aa65d020d/lava-logs.yaml 2026-03-25 00:34:29,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:30,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:30,120 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4618a3717aa65d020d/lava_callback.json.gz 2026-03-25 00:34:30,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50004 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:34:30,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4618a3717aa65d020d HTTP/1.1" 200 None 2026-03-25 00:34:30,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:30,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:34:30,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:30,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d4618a3717aa65d020d HTTP/1.1" 200 None 2026-03-25 00:34:30,265 - lava_callback - INFO - Completed processing callback for node 69c32d4618a3717aa65d020d INFO: 172.21.0.1:50012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43770 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:34:50,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:50,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48772 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:34:57,454 - lava_callback - INFO - Received callback for node 69c32d6c18a3717aa65d02b1 INFO: 172.21.0.1:48780 - "POST /node/69c32d6c18a3717aa65d02b1 HTTP/1.0" 202 Accepted 2026-03-25 00:34:57,501 - lava_callback - INFO - Starting async job submit for node 69c32d6c18a3717aa65d02b1 2026-03-25 00:34:57,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:57,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6c18a3717aa65d02b1 HTTP/1.1" 200 None 2026-03-25 00:34:57,539 - lava_callback - DEBUG - Processing node 69c32d6c18a3717aa65d02b1: baseline-arm-pengutronix 2026-03-25 00:34:57,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:57,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:57,766 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6c18a3717aa65d02b1/log.txt.gz 2026-03-25 00:34:57,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:57,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:57,985 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6c18a3717aa65d02b1/lava-logs.yaml 2026-03-25 00:34:57,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:34:58,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:34:58,109 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6c18a3717aa65d02b1/lava_callback.json.gz 2026-03-25 00:34:58,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:58,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6c18a3717aa65d02b1 HTTP/1.1" 200 None 2026-03-25 00:34:58,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:58,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:34:58,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:34:58,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d6c18a3717aa65d02b1 HTTP/1.1" 200 None 2026-03-25 00:34:58,188 - lava_callback - INFO - Completed processing callback for node 69c32d6c18a3717aa65d02b1 2026-03-25 00:35:00,293 - lava_callback - INFO - Received callback for node 69c32d9f18a3717aa65d031b INFO: 172.21.0.1:60132 - "POST /node/69c32d9f18a3717aa65d031b HTTP/1.0" 202 Accepted 2026-03-25 00:35:00,299 - lava_callback - INFO - Starting async job submit for node 69c32d9f18a3717aa65d031b 2026-03-25 00:35:00,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:00,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9f18a3717aa65d031b HTTP/1.1" 200 None 2026-03-25 00:35:00,317 - lava_callback - DEBUG - Processing node 69c32d9f18a3717aa65d031b: baseline-arm 2026-03-25 00:35:00,322 - lava_callback - WARNING - Failed to upload log 2026-03-25 00:35:00,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:00,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:00,436 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d9f18a3717aa65d031b/lava-logs.yaml 2026-03-25 00:35:00,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:00,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:00,551 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d9f18a3717aa65d031b/lava_callback.json.gz 2026-03-25 00:35:00,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:00,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9f18a3717aa65d031b HTTP/1.1" 200 None 2026-03-25 00:35:00,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:00,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:35:00,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:00,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d9f18a3717aa65d031b HTTP/1.1" 200 None 2026-03-25 00:35:00,615 - lava_callback - INFO - Completed processing callback for node 69c32d9f18a3717aa65d031b 2026-03-25 00:35:04,505 - lava_callback - INFO - Received callback for node 69c32d5718a3717aa65d024d INFO: 172.21.0.1:60146 - "POST /node/69c32d5718a3717aa65d024d HTTP/1.0" 202 Accepted 2026-03-25 00:35:04,512 - lava_callback - INFO - Starting async job submit for node 69c32d5718a3717aa65d024d 2026-03-25 00:35:04,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:04,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5718a3717aa65d024d HTTP/1.1" 200 None 2026-03-25 00:35:04,552 - lava_callback - DEBUG - Processing node 69c32d5718a3717aa65d024d: baseline-arm-broonie 2026-03-25 00:35:04,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:04,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:04,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5718a3717aa65d024d/log.txt.gz 2026-03-25 00:35:05,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:05,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:05,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5718a3717aa65d024d/lava-logs.yaml 2026-03-25 00:35:05,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:05,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:05,382 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5718a3717aa65d024d/lava_callback.json.gz 2026-03-25 00:35:05,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:05,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5718a3717aa65d024d HTTP/1.1" 200 None 2026-03-25 00:35:05,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:05,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:35:05,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:05,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d5718a3717aa65d024d HTTP/1.1" 200 None 2026-03-25 00:35:05,467 - lava_callback - INFO - Completed processing callback for node 69c32d5718a3717aa65d024d 2026-03-25 00:35:06,017 - lava_callback - INFO - Received callback for node 69c32d5918a3717aa65d0250 INFO: 172.21.0.1:60160 - "POST /node/69c32d5918a3717aa65d0250 HTTP/1.0" 202 Accepted 2026-03-25 00:35:06,024 - lava_callback - INFO - Starting async job submit for node 69c32d5918a3717aa65d0250 2026-03-25 00:35:06,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:06,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5918a3717aa65d0250 HTTP/1.1" 200 None 2026-03-25 00:35:06,068 - lava_callback - DEBUG - Processing node 69c32d5918a3717aa65d0250: baseline-arm-broonie 2026-03-25 00:35:06,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:06,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:06,407 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5918a3717aa65d0250/log.txt.gz INFO: 172.21.0.1:60172 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:35:06,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:06,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:06,773 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5918a3717aa65d0250/lava-logs.yaml 2026-03-25 00:35:06,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:06,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:06,940 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5918a3717aa65d0250/lava_callback.json.gz 2026-03-25 00:35:06,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:06,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5918a3717aa65d0250 HTTP/1.1" 200 None 2026-03-25 00:35:06,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:07,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:35:07,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:07,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d5918a3717aa65d0250 HTTP/1.1" 200 None 2026-03-25 00:35:07,041 - lava_callback - INFO - Completed processing callback for node 69c32d5918a3717aa65d0250 2026-03-25 00:35:08,878 - lava_callback - INFO - Received callback for node 69c32d6818a3717aa65d0284 INFO: 172.21.0.1:60176 - "POST /node/69c32d6818a3717aa65d0284 HTTP/1.0" 202 Accepted 2026-03-25 00:35:08,888 - lava_callback - INFO - Starting async job submit for node 69c32d6818a3717aa65d0284 2026-03-25 00:35:08,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:08,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6818a3717aa65d0284 HTTP/1.1" 200 None 2026-03-25 00:35:08,924 - lava_callback - DEBUG - Processing node 69c32d6818a3717aa65d0284: baseline-arm-pengutronix 2026-03-25 00:35:09,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:09,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:09,201 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6818a3717aa65d0284/log.txt.gz 2026-03-25 00:35:09,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:09,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:09,485 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6818a3717aa65d0284/lava-logs.yaml 2026-03-25 00:35:09,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:09,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:09,626 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6818a3717aa65d0284/lava_callback.json.gz 2026-03-25 00:35:09,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:09,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6818a3717aa65d0284 HTTP/1.1" 200 None 2026-03-25 00:35:09,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:09,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:35:09,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:09,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d6818a3717aa65d0284 HTTP/1.1" 200 None 2026-03-25 00:35:09,706 - lava_callback - INFO - Completed processing callback for node 69c32d6818a3717aa65d0284 INFO: 172.21.0.1:39594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39604 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:35:20,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:20,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-25 00:35:25,301 - lava_callback - INFO - Received callback for node 69c32d5e18a3717aa65d0262 INFO: 172.21.0.1:40008 - "POST /node/69c32d5e18a3717aa65d0262 HTTP/1.0" 202 Accepted 2026-03-25 00:35:25,313 - lava_callback - INFO - Starting async job submit for node 69c32d5e18a3717aa65d0262 2026-03-25 00:35:25,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:25,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5e18a3717aa65d0262 HTTP/1.1" 200 None 2026-03-25 00:35:25,386 - lava_callback - DEBUG - Processing node 69c32d5e18a3717aa65d0262: baseline-arm-clabbe 2026-03-25 00:35:25,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:26,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:26,203 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d5e18a3717aa65d0262/log.txt.gz 2026-03-25 00:35:26,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40014 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:35:26,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:26,922 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d5e18a3717aa65d0262/lava-logs.yaml 2026-03-25 00:35:26,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:27,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:27,125 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d5e18a3717aa65d0262/lava_callback.json.gz 2026-03-25 00:35:27,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:27,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5e18a3717aa65d0262 HTTP/1.1" 200 None 2026-03-25 00:35:27,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:27,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:35:27,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:27,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d5e18a3717aa65d0262 HTTP/1.1" 200 None 2026-03-25 00:35:27,277 - lava_callback - INFO - Completed processing callback for node 69c32d5e18a3717aa65d0262 INFO: 172.21.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45614 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:35:50,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:50,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47548 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:35:57,672 - lava_callback - INFO - Received callback for node 69c32d8418a3717aa65d02d1 INFO: 172.21.0.1:47554 - "POST /node/69c32d8418a3717aa65d02d1 HTTP/1.0" 202 Accepted 2026-03-25 00:35:57,683 - lava_callback - INFO - Starting async job submit for node 69c32d8418a3717aa65d02d1 2026-03-25 00:35:57,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:35:57,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8418a3717aa65d02d1 HTTP/1.1" 200 None 2026-03-25 00:35:57,782 - lava_callback - DEBUG - Processing node 69c32d8418a3717aa65d02d1: ltp-dio 2026-03-25 00:35:58,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:58,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:58,849 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c32d8418a3717aa65d02d1/log.txt.gz 2026-03-25 00:35:59,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:35:59,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:35:59,721 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c32d8418a3717aa65d02d1/lava-logs.yaml 2026-03-25 00:35:59,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:36:00,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:36:00,054 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c32d8418a3717aa65d02d1/lava_callback.json.gz 2026-03-25 00:36:00,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:00,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8418a3717aa65d02d1 HTTP/1.1" 200 None 2026-03-25 00:36:00,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:00,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:36:00,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:00,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d8418a3717aa65d02d1 HTTP/1.1" 200 None 2026-03-25 00:36:00,194 - lava_callback - INFO - Completed processing callback for node 69c32d8418a3717aa65d02d1 INFO: 172.21.0.1:41652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33684 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 00:36:15,363 - lava_callback - INFO - Received callback for node 69c32d6918a3717aa65d02ab INFO: 172.21.0.1:33696 - "POST /node/69c32d6918a3717aa65d02ab HTTP/1.0" 202 Accepted 2026-03-25 00:36:15,369 - lava_callback - INFO - Starting async job submit for node 69c32d6918a3717aa65d02ab 2026-03-25 00:36:15,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:15,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6918a3717aa65d02ab HTTP/1.1" 200 None 2026-03-25 00:36:15,426 - lava_callback - DEBUG - Processing node 69c32d6918a3717aa65d02ab: baseline-arm-pengutronix 2026-03-25 00:36:15,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:36:15,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:36:15,992 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6918a3717aa65d02ab/log.txt.gz 2026-03-25 00:36:16,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:36:16,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:36:16,283 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6918a3717aa65d02ab/lava-logs.yaml 2026-03-25 00:36:16,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:33704 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:36:16,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:36:16,630 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6918a3717aa65d02ab/lava_callback.json.gz 2026-03-25 00:36:16,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:16,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6918a3717aa65d02ab HTTP/1.1" 200 None 2026-03-25 00:36:16,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:16,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:36:16,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:16,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d6918a3717aa65d02ab HTTP/1.1" 200 None 2026-03-25 00:36:16,803 - lava_callback - INFO - Completed processing callback for node 69c32d6918a3717aa65d02ab 2026-03-25 00:36:20,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:20,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53922 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:36:43,613 - lava_callback - INFO - Received callback for node 69c32d9618a3717aa65d0315 INFO: 172.21.0.1:35170 - "POST /node/69c32d9618a3717aa65d0315 HTTP/1.0" 202 Accepted 2026-03-25 00:36:43,628 - lava_callback - INFO - Starting async job submit for node 69c32d9618a3717aa65d0315 2026-03-25 00:36:43,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:43,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9618a3717aa65d0315 HTTP/1.1" 200 None 2026-03-25 00:36:43,698 - lava_callback - DEBUG - Processing node 69c32d9618a3717aa65d0315: baseline-arm 2026-03-25 00:36:44,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:36:44,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:36:44,397 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d9618a3717aa65d0315/log.txt.gz 2026-03-25 00:36:44,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:36:45,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:36:45,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d9618a3717aa65d0315/lava-logs.yaml 2026-03-25 00:36:45,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:36:45,564 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:36:45,566 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d9618a3717aa65d0315/lava_callback.json.gz 2026-03-25 00:36:45,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:45,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9618a3717aa65d0315 HTTP/1.1" 200 None 2026-03-25 00:36:45,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:45,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3294718a3717aa65d01b3 HTTP/1.1" 200 None 2026-03-25 00:36:45,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:45,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d9618a3717aa65d0315 HTTP/1.1" 200 None 2026-03-25 00:36:45,660 - lava_callback - INFO - Completed processing callback for node 69c32d9618a3717aa65d0315 INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:36:50,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:50,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-25 00:36:54,956 - lava_callback - INFO - Received callback for node 69c32d5a18a3717aa65d0253 INFO: 172.21.0.1:47572 - "POST /node/69c32d5a18a3717aa65d0253 HTTP/1.0" 202 Accepted 2026-03-25 00:36:54,966 - lava_callback - INFO - Starting async job submit for node 69c32d5a18a3717aa65d0253 2026-03-25 00:36:55,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:55,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5a18a3717aa65d0253 HTTP/1.1" 200 None 2026-03-25 00:36:55,208 - lava_callback - DEBUG - Processing node 69c32d5a18a3717aa65d0253: baseline-arm-broonie 2026-03-25 00:36:55,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:36:56,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:36:56,501 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5a18a3717aa65d0253/log.txt.gz INFO: 172.21.0.1:47580 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:36:56,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:36:57,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:36:57,723 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5a18a3717aa65d0253/lava-logs.yaml 2026-03-25 00:36:57,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:36:58,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:36:58,619 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5a18a3717aa65d0253/lava_callback.json.gz 2026-03-25 00:36:58,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:58,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5a18a3717aa65d0253 HTTP/1.1" 200 None 2026-03-25 00:36:58,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:58,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:36:58,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:36:58,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d5a18a3717aa65d0253 HTTP/1.1" 200 None 2026-03-25 00:36:58,770 - lava_callback - INFO - Completed processing callback for node 69c32d5a18a3717aa65d0253 INFO: 172.21.0.1:38966 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:37:15,135 - lava_callback - INFO - Received callback for node 69c32d5618a3717aa65d0246 INFO: 172.21.0.1:60158 - "POST /node/69c32d5618a3717aa65d0246 HTTP/1.0" 202 Accepted 2026-03-25 00:37:15,145 - lava_callback - INFO - Starting async job submit for node 69c32d5618a3717aa65d0246 INFO: 172.21.0.1:60170 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 00:37:15,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:37:15,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5618a3717aa65d0246 HTTP/1.1" 200 None 2026-03-25 00:37:15,220 - lava_callback - DEBUG - Processing node 69c32d5618a3717aa65d0246: baseline-arm-broonie 2026-03-25 00:37:15,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:37:16,254 - lava_callback - INFO - Received callback for node 69c32d5f18a3717aa65d0266 2026-03-25 00:37:16,265 - lava_callback - INFO - Starting async job submit for node 69c32d5f18a3717aa65d0266 INFO: 172.21.0.1:60184 - "POST /node/69c32d5f18a3717aa65d0266 HTTP/1.0" 202 Accepted 2026-03-25 00:37:16,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:37:16,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5f18a3717aa65d0266 HTTP/1.1" 200 None 2026-03-25 00:37:16,332 - lava_callback - DEBUG - Processing node 69c32d5f18a3717aa65d0266: baseline-arm-clabbe INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:37:16,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:37:16,622 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5618a3717aa65d0246/log.txt.gz 2026-03-25 00:37:16,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:37:16,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:37:17,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:37:17,706 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d5f18a3717aa65d0266/log.txt.gz 2026-03-25 00:37:17,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:37:17,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:37:17,909 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5618a3717aa65d0246/lava-logs.yaml 2026-03-25 00:37:17,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:37:18,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:37:18,881 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d5f18a3717aa65d0266/lava-logs.yaml 2026-03-25 00:37:18,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:37:18,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:37:18,891 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c32d5618a3717aa65d0246/lava_callback.json.gz 2026-03-25 00:37:18,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:37:18,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5618a3717aa65d0246 HTTP/1.1" 200 None 2026-03-25 00:37:18,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:37:18,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:37:18,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:37:19,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d5618a3717aa65d0246 HTTP/1.1" 200 None 2026-03-25 00:37:19,049 - lava_callback - INFO - Completed processing callback for node 69c32d5618a3717aa65d0246 2026-03-25 00:37:19,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:37:19,952 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d5f18a3717aa65d0266/lava_callback.json.gz 2026-03-25 00:37:19,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:37:20,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d5f18a3717aa65d0266 HTTP/1.1" 200 None 2026-03-25 00:37:20,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:37:20,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:37:20,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:37:20,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d5f18a3717aa65d0266 HTTP/1.1" 200 None 2026-03-25 00:37:20,095 - lava_callback - INFO - Completed processing callback for node 69c32d5f18a3717aa65d0266 2026-03-25 00:37:20,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:37:20,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33714 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:39:19,427 - lava_callback - INFO - Received callback for node 69c32d6318a3717aa65d0272 INFO: 172.21.0.1:33730 - "POST /node/69c32d6318a3717aa65d0272 HTTP/1.0" 202 Accepted 2026-03-25 00:39:19,433 - lava_callback - INFO - Starting async job submit for node 69c32d6318a3717aa65d0272 2026-03-25 00:39:19,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:19,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6318a3717aa65d0272 HTTP/1.1" 200 None 2026-03-25 00:39:19,481 - lava_callback - DEBUG - Processing node 69c32d6318a3717aa65d0272: baseline-arm-clabbe 2026-03-25 00:39:19,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:19,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:19,907 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d6318a3717aa65d0272/log.txt.gz 2026-03-25 00:39:20,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:20,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:20,295 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d6318a3717aa65d0272/lava-logs.yaml 2026-03-25 00:39:20,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:20,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:20,451 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d6318a3717aa65d0272/lava_callback.json.gz 2026-03-25 00:39:20,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:20,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6318a3717aa65d0272 HTTP/1.1" 200 None 2026-03-25 00:39:20,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:20,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:39:20,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:20,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d6318a3717aa65d0272 HTTP/1.1" 200 None 2026-03-25 00:39:20,597 - lava_callback - INFO - Completed processing callback for node 69c32d6318a3717aa65d0272 2026-03-25 00:39:23,885 - lava_callback - INFO - Received callback for node 69c32d8218a3717aa65d02ce INFO: 172.21.0.1:45020 - "POST /node/69c32d8218a3717aa65d02ce HTTP/1.0" 202 Accepted 2026-03-25 00:39:23,893 - lava_callback - INFO - Starting async job submit for node 69c32d8218a3717aa65d02ce 2026-03-25 00:39:23,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:23,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8218a3717aa65d02ce HTTP/1.1" 200 None 2026-03-25 00:39:23,960 - lava_callback - DEBUG - Processing node 69c32d8218a3717aa65d02ce: ltp-cve 2026-03-25 00:39:24,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:24,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:24,593 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c32d8218a3717aa65d02ce/log.txt.gz 2026-03-25 00:39:24,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:25,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:25,146 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c32d8218a3717aa65d02ce/lava-logs.yaml 2026-03-25 00:39:25,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:25,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:25,324 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c32d8218a3717aa65d02ce/lava_callback.json.gz 2026-03-25 00:39:25,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:25,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8218a3717aa65d02ce HTTP/1.1" 200 None 2026-03-25 00:39:25,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:25,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:39:25,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:25,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d8218a3717aa65d02ce HTTP/1.1" 200 None 2026-03-25 00:39:25,434 - lava_callback - INFO - Completed processing callback for node 69c32d8218a3717aa65d02ce INFO: 172.21.0.1:45036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50052 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:39:35,170 - lava_callback - INFO - Received callback for node 69c32eae18a3717aa65d296a INFO: 172.21.0.1:50068 - "POST /node/69c32eae18a3717aa65d296a HTTP/1.0" 202 Accepted 2026-03-25 00:39:35,176 - lava_callback - INFO - Starting async job submit for node 69c32eae18a3717aa65d296a 2026-03-25 00:39:35,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:35,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32eae18a3717aa65d296a HTTP/1.1" 200 None 2026-03-25 00:39:35,200 - lava_callback - DEBUG - Processing node 69c32eae18a3717aa65d296a: baseline-arm 2026-03-25 00:39:35,207 - lava_callback - WARNING - Failed to upload log 2026-03-25 00:39:35,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:35,289 - lava_callback - INFO - Received callback for node 69c32d7d18a3717aa65d02c1 INFO: 172.21.0.1:50082 - "POST /node/69c32d7d18a3717aa65d02c1 HTTP/1.0" 202 Accepted 2026-03-25 00:39:35,299 - lava_callback - INFO - Starting async job submit for node 69c32d7d18a3717aa65d02c1 2026-03-25 00:39:35,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:35,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:35,338 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c32eae18a3717aa65d296a/lava-logs.yaml 2026-03-25 00:39:35,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:35,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d7d18a3717aa65d02c1 HTTP/1.1" 200 None 2026-03-25 00:39:35,353 - lava_callback - DEBUG - Processing node 69c32d7d18a3717aa65d02c1: ltp-cap_bounds 2026-03-25 00:39:35,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:35,588 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c32eae18a3717aa65d296a/lava_callback.json.gz 2026-03-25 00:39:35,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:35,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32eae18a3717aa65d296a HTTP/1.1" 200 None 2026-03-25 00:39:35,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:35,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:39:35,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:35,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:35,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32eae18a3717aa65d296a HTTP/1.1" 200 None 2026-03-25 00:39:35,886 - lava_callback - INFO - Completed processing callback for node 69c32eae18a3717aa65d296a 2026-03-25 00:39:36,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:36,059 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c32d7d18a3717aa65d02c1/log.txt.gz 2026-03-25 00:39:36,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:50088 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:39:36,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:36,542 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c32d7d18a3717aa65d02c1/lava-logs.yaml 2026-03-25 00:39:36,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:36,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:36,709 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c32d7d18a3717aa65d02c1/lava_callback.json.gz 2026-03-25 00:39:36,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:36,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d7d18a3717aa65d02c1 HTTP/1.1" 200 None 2026-03-25 00:39:36,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:36,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:39:36,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:36,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d7d18a3717aa65d02c1 HTTP/1.1" 200 None 2026-03-25 00:39:36,814 - lava_callback - INFO - Completed processing callback for node 69c32d7d18a3717aa65d02c1 INFO: 172.21.0.1:33760 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:39:50,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:50,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-25 00:39:55,584 - lava_callback - INFO - Received callback for node 69c32d8018a3717aa65d02cb INFO: 172.21.0.1:37714 - "POST /node/69c32d8018a3717aa65d02cb HTTP/1.0" 202 Accepted 2026-03-25 00:39:55,595 - lava_callback - INFO - Starting async job submit for node 69c32d8018a3717aa65d02cb 2026-03-25 00:39:55,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:55,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8018a3717aa65d02cb HTTP/1.1" 200 None 2026-03-25 00:39:55,662 - lava_callback - DEBUG - Processing node 69c32d8018a3717aa65d02cb: ltp-controllers 2026-03-25 00:39:55,861 - lava_callback - INFO - Received callback for node 69c08bcece0c9bc43bccdc84 INFO: 172.21.0.1:37720 - "POST /node/69c08bcece0c9bc43bccdc84 HTTP/1.0" 202 Accepted 2026-03-25 00:39:55,878 - lava_callback - INFO - Starting async job submit for node 69c08bcece0c9bc43bccdc84 2026-03-25 00:39:55,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:55,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08bcece0c9bc43bccdc84 HTTP/1.1" 200 None 2026-03-25 00:39:56,000 - lava_callback - DEBUG - Processing node 69c08bcece0c9bc43bccdc84: baseline-arm-clabbe 2026-03-25 00:39:56,000 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08bcece0c9bc43bccdc84 2026-03-25 00:39:56,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:56,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08bcece0c9bc43bccdc84 HTTP/1.1" 200 None 2026-03-25 00:39:56,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:56,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 00:39:56,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:56,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08bcece0c9bc43bccdc84 HTTP/1.1" 200 None 2026-03-25 00:39:56,344 - lava_callback - ERROR - Error processing callback for node 69c08bcece0c9bc43bccdc84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 00:39:56,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:37734 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:39:56,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:56,585 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c32d8018a3717aa65d02cb/log.txt.gz 2026-03-25 00:39:56,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:57,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:57,154 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c32d8018a3717aa65d02cb/lava-logs.yaml 2026-03-25 00:39:57,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:39:57,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:39:57,337 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c32d8018a3717aa65d02cb/lava_callback.json.gz 2026-03-25 00:39:57,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:57,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8018a3717aa65d02cb HTTP/1.1" 200 None 2026-03-25 00:39:57,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:57,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:39:57,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:39:57,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d8018a3717aa65d02cb HTTP/1.1" 200 None 2026-03-25 00:39:57,441 - lava_callback - INFO - Completed processing callback for node 69c32d8018a3717aa65d02cb INFO: 172.21.0.1:58320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39362 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 00:40:15,502 - lava_callback - INFO - Received callback for node 69c32d9018a3717aa65d0301 INFO: 172.21.0.1:39374 - "POST /node/69c32d9018a3717aa65d0301 HTTP/1.0" 202 Accepted 2026-03-25 00:40:15,512 - lava_callback - INFO - Starting async job submit for node 69c32d9018a3717aa65d0301 2026-03-25 00:40:15,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:40:15,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9018a3717aa65d0301 HTTP/1.1" 200 None 2026-03-25 00:40:15,563 - lava_callback - DEBUG - Processing node 69c32d9018a3717aa65d0301: ltp-syscalls INFO: 172.21.0.1:39384 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:40:16,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:40:16,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:40:16,700 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c32d9018a3717aa65d0301/log.txt.gz 2026-03-25 00:40:17,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:40:17,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:40:17,496 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c32d9018a3717aa65d0301/lava-logs.yaml 2026-03-25 00:40:17,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:40:17,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:40:17,673 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c32d9018a3717aa65d0301/lava_callback.json.gz 2026-03-25 00:40:17,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:40:17,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9018a3717aa65d0301 HTTP/1.1" 200 None 2026-03-25 00:40:17,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:40:17,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:40:17,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:40:17,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d9018a3717aa65d0301 HTTP/1.1" 200 None 2026-03-25 00:40:17,764 - lava_callback - INFO - Completed processing callback for node 69c32d9018a3717aa65d0301 2026-03-25 00:40:20,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:40:20,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46670 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:40:35,932 - lava_callback - INFO - Received callback for node 69c08bf4ce0c9bc43bccdc8c INFO: 172.21.0.1:46674 - "POST /node/69c08bf4ce0c9bc43bccdc8c HTTP/1.0" 202 Accepted 2026-03-25 00:40:35,940 - lava_callback - INFO - Starting async job submit for node 69c08bf4ce0c9bc43bccdc8c 2026-03-25 00:40:35,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:40:35,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08bf4ce0c9bc43bccdc8c HTTP/1.1" 200 None 2026-03-25 00:40:35,959 - lava_callback - DEBUG - Processing node 69c08bf4ce0c9bc43bccdc8c: baseline-arm-clabbe 2026-03-25 00:40:35,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08bf4ce0c9bc43bccdc8c 2026-03-25 00:40:35,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:40:35,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08bf4ce0c9bc43bccdc8c HTTP/1.1" 200 None 2026-03-25 00:40:35,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:40:35,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 00:40:35,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:40:36,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08bf4ce0c9bc43bccdc8c HTTP/1.1" 200 None 2026-03-25 00:40:36,023 - lava_callback - ERROR - Error processing callback for node 69c08bf4ce0c9bc43bccdc8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54062 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:41:11,188 - lava_callback - INFO - Received callback for node 69c08c22ce0c9bc43bccdcb3 INFO: 172.21.0.1:34774 - "POST /node/69c08c22ce0c9bc43bccdcb3 HTTP/1.0" 202 Accepted 2026-03-25 00:41:11,194 - lava_callback - INFO - Starting async job submit for node 69c08c22ce0c9bc43bccdcb3 2026-03-25 00:41:11,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:11,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c22ce0c9bc43bccdcb3 HTTP/1.1" 200 None 2026-03-25 00:41:11,209 - lava_callback - DEBUG - Processing node 69c08c22ce0c9bc43bccdcb3: baseline-arm-baylibre 2026-03-25 00:41:11,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c22ce0c9bc43bccdcb3 2026-03-25 00:41:11,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:11,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c22ce0c9bc43bccdcb3 HTTP/1.1" 200 None 2026-03-25 00:41:11,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:11,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:11,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:11,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c22ce0c9bc43bccdcb3 HTTP/1.1" 200 None 2026-03-25 00:41:11,268 - lava_callback - ERROR - Error processing callback for node 69c08c22ce0c9bc43bccdcb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 00:41:11,873 - lava_callback - INFO - Received callback for node 69c08c24ce0c9bc43bccdcb6 INFO: 172.21.0.1:34780 - "POST /node/69c08c24ce0c9bc43bccdcb6 HTTP/1.0" 202 Accepted 2026-03-25 00:41:11,879 - lava_callback - INFO - Starting async job submit for node 69c08c24ce0c9bc43bccdcb6 2026-03-25 00:41:11,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:11,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c24ce0c9bc43bccdcb6 HTTP/1.1" 200 None 2026-03-25 00:41:11,900 - lava_callback - DEBUG - Processing node 69c08c24ce0c9bc43bccdcb6: baseline-arm-baylibre 2026-03-25 00:41:11,901 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c24ce0c9bc43bccdcb6 2026-03-25 00:41:11,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:11,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c24ce0c9bc43bccdcb6 HTTP/1.1" 200 None 2026-03-25 00:41:11,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:11,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:11,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:11,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c24ce0c9bc43bccdcb6 HTTP/1.1" 200 None 2026-03-25 00:41:11,962 - lava_callback - ERROR - Error processing callback for node 69c08c24ce0c9bc43bccdcb6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 00:41:12,540 - lava_callback - INFO - Received callback for node 69c08c25ce0c9bc43bccdcb9 INFO: 172.21.0.1:34788 - "POST /node/69c08c25ce0c9bc43bccdcb9 HTTP/1.0" 202 Accepted 2026-03-25 00:41:12,545 - lava_callback - INFO - Starting async job submit for node 69c08c25ce0c9bc43bccdcb9 2026-03-25 00:41:12,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:12,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c25ce0c9bc43bccdcb9 HTTP/1.1" 200 None 2026-03-25 00:41:12,562 - lava_callback - DEBUG - Processing node 69c08c25ce0c9bc43bccdcb9: baseline-arm-baylibre 2026-03-25 00:41:12,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c25ce0c9bc43bccdcb9 2026-03-25 00:41:12,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:12,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c25ce0c9bc43bccdcb9 HTTP/1.1" 200 None 2026-03-25 00:41:12,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:12,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:12,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:12,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c25ce0c9bc43bccdcb9 HTTP/1.1" 200 None 2026-03-25 00:41:12,626 - lava_callback - ERROR - Error processing callback for node 69c08c25ce0c9bc43bccdcb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34796 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 00:41:15,305 - lava_callback - INFO - Received callback for node 69c32d8518a3717aa65d02d4 INFO: 172.21.0.1:34800 - "POST /node/69c32d8518a3717aa65d02d4 HTTP/1.0" 202 Accepted 2026-03-25 00:41:15,311 - lava_callback - INFO - Starting async job submit for node 69c32d8518a3717aa65d02d4 2026-03-25 00:41:15,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:15,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8518a3717aa65d02d4 HTTP/1.1" 200 None 2026-03-25 00:41:15,353 - lava_callback - DEBUG - Processing node 69c32d8518a3717aa65d02d4: ltp-fs_bind 2026-03-25 00:41:15,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:41:15,982 - lava_callback - INFO - Received callback for node 69c08c19ce0c9bc43bccdc96 INFO: 172.21.0.1:34806 - "POST /node/69c08c19ce0c9bc43bccdc96 HTTP/1.0" 202 Accepted 2026-03-25 00:41:15,988 - lava_callback - INFO - Starting async job submit for node 69c08c19ce0c9bc43bccdc96 2026-03-25 00:41:15,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:15,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:41:16,002 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c32d8518a3717aa65d02d4/log.txt.gz 2026-03-25 00:41:16,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c19ce0c9bc43bccdc96 HTTP/1.1" 200 None 2026-03-25 00:41:16,030 - lava_callback - DEBUG - Processing node 69c08c19ce0c9bc43bccdc96: baseline-arm-clabbe 2026-03-25 00:41:16,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c19ce0c9bc43bccdc96 2026-03-25 00:41:16,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:16,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c19ce0c9bc43bccdc96 HTTP/1.1" 200 None 2026-03-25 00:41:16,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:16,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 00:41:16,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:16,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:41:16,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c19ce0c9bc43bccdc96 HTTP/1.1" 200 None 2026-03-25 00:41:16,260 - lava_callback - ERROR - Error processing callback for node 69c08c19ce0c9bc43bccdc96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34808 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:41:16,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:41:16,522 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c32d8518a3717aa65d02d4/lava-logs.yaml 2026-03-25 00:41:16,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:41:16,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:41:16,684 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c32d8518a3717aa65d02d4/lava_callback.json.gz 2026-03-25 00:41:16,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:16,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8518a3717aa65d02d4 HTTP/1.1" 200 None 2026-03-25 00:41:16,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:16,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:41:16,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:16,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d8518a3717aa65d02d4 HTTP/1.1" 200 None 2026-03-25 00:41:16,770 - lava_callback - INFO - Completed processing callback for node 69c32d8518a3717aa65d02d4 2026-03-25 00:41:20,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:20,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-25 00:41:22,435 - lava_callback - INFO - Received callback for node 69c32d6418a3717aa65d0275 INFO: 172.21.0.1:52182 - "POST /node/69c32d6418a3717aa65d0275 HTTP/1.0" 202 Accepted 2026-03-25 00:41:22,441 - lava_callback - INFO - Starting async job submit for node 69c32d6418a3717aa65d0275 2026-03-25 00:41:22,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:22,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6418a3717aa65d0275 HTTP/1.1" 200 None 2026-03-25 00:41:22,478 - lava_callback - DEBUG - Processing node 69c32d6418a3717aa65d0275: baseline-arm-clabbe 2026-03-25 00:41:22,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:41:22,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:41:22,803 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d6418a3717aa65d0275/log.txt.gz 2026-03-25 00:41:22,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:41:23,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:41:23,117 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d6418a3717aa65d0275/lava-logs.yaml 2026-03-25 00:41:23,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:41:23,259 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:41:23,260 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c32d6418a3717aa65d0275/lava_callback.json.gz 2026-03-25 00:41:23,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:23,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6418a3717aa65d0275 HTTP/1.1" 200 None 2026-03-25 00:41:23,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:23,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:41:23,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:23,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d6418a3717aa65d0275 HTTP/1.1" 200 None 2026-03-25 00:41:23,342 - lava_callback - INFO - Completed processing callback for node 69c32d6418a3717aa65d0275 INFO: 172.21.0.1:52184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43728 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:41:31,124 - lava_callback - INFO - Received callback for node 69c08c26ce0c9bc43bccdcbc INFO: 172.21.0.1:43744 - "POST /node/69c08c26ce0c9bc43bccdcbc HTTP/1.0" 202 Accepted 2026-03-25 00:41:31,129 - lava_callback - INFO - Starting async job submit for node 69c08c26ce0c9bc43bccdcbc 2026-03-25 00:41:31,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:31,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c26ce0c9bc43bccdcbc HTTP/1.1" 200 None 2026-03-25 00:41:31,145 - lava_callback - DEBUG - Processing node 69c08c26ce0c9bc43bccdcbc: baseline-arm-baylibre 2026-03-25 00:41:31,146 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c26ce0c9bc43bccdcbc 2026-03-25 00:41:31,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:31,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c26ce0c9bc43bccdcbc HTTP/1.1" 200 None 2026-03-25 00:41:31,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:31,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:31,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:31,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c26ce0c9bc43bccdcbc HTTP/1.1" 200 None 2026-03-25 00:41:31,199 - lava_callback - ERROR - Error processing callback for node 69c08c26ce0c9bc43bccdcbc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 00:41:31,789 - lava_callback - INFO - Received callback for node 69c08c29ce0c9bc43bccdcda INFO: 172.21.0.1:43758 - "POST /node/69c08c29ce0c9bc43bccdcda HTTP/1.0" 202 Accepted 2026-03-25 00:41:31,795 - lava_callback - INFO - Starting async job submit for node 69c08c29ce0c9bc43bccdcda 2026-03-25 00:41:31,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:31,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c29ce0c9bc43bccdcda HTTP/1.1" 200 None 2026-03-25 00:41:31,810 - lava_callback - DEBUG - Processing node 69c08c29ce0c9bc43bccdcda: baseline-arm-baylibre 2026-03-25 00:41:31,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c29ce0c9bc43bccdcda 2026-03-25 00:41:31,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:31,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c29ce0c9bc43bccdcda HTTP/1.1" 200 None 2026-03-25 00:41:31,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:31,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:31,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:31,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c29ce0c9bc43bccdcda HTTP/1.1" 200 None 2026-03-25 00:41:31,869 - lava_callback - ERROR - Error processing callback for node 69c08c29ce0c9bc43bccdcda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 00:41:32,448 - lava_callback - INFO - Received callback for node 69c08c2bce0c9bc43bccdcdd INFO: 172.21.0.1:43760 - "POST /node/69c08c2bce0c9bc43bccdcdd HTTP/1.0" 202 Accepted 2026-03-25 00:41:32,453 - lava_callback - INFO - Starting async job submit for node 69c08c2bce0c9bc43bccdcdd 2026-03-25 00:41:32,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:32,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c2bce0c9bc43bccdcdd HTTP/1.1" 200 None 2026-03-25 00:41:32,469 - lava_callback - DEBUG - Processing node 69c08c2bce0c9bc43bccdcdd: baseline-arm-baylibre 2026-03-25 00:41:32,469 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c2bce0c9bc43bccdcdd 2026-03-25 00:41:32,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:32,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c2bce0c9bc43bccdcdd HTTP/1.1" 200 None 2026-03-25 00:41:32,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:32,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:32,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:32,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c2bce0c9bc43bccdcdd HTTP/1.1" 200 None 2026-03-25 00:41:32,535 - lava_callback - ERROR - Error processing callback for node 69c08c2bce0c9bc43bccdcdd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 00:41:36,071 - lava_callback - INFO - Received callback for node 69c08c32ce0c9bc43bccdcf2 INFO: 172.21.0.1:43762 - "POST /node/69c08c32ce0c9bc43bccdcf2 HTTP/1.0" 202 Accepted 2026-03-25 00:41:36,077 - lava_callback - INFO - Starting async job submit for node 69c08c32ce0c9bc43bccdcf2 2026-03-25 00:41:36,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:36,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c32ce0c9bc43bccdcf2 HTTP/1.1" 200 None 2026-03-25 00:41:36,110 - lava_callback - DEBUG - Processing node 69c08c32ce0c9bc43bccdcf2: baseline-arm-clabbe 2026-03-25 00:41:36,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c32ce0c9bc43bccdcf2 2026-03-25 00:41:36,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:36,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c32ce0c9bc43bccdcf2 HTTP/1.1" 200 None 2026-03-25 00:41:36,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:36,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:36,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:36,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c32ce0c9bc43bccdcf2 HTTP/1.1" 200 None 2026-03-25 00:41:36,167 - lava_callback - ERROR - Error processing callback for node 69c08c32ce0c9bc43bccdcf2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:41:36,739 - lava_callback - INFO - Received callback for node 69c08c36ce0c9bc43bccdd01 INFO: 172.21.0.1:43782 - "POST /node/69c08c36ce0c9bc43bccdd01 HTTP/1.0" 202 Accepted 2026-03-25 00:41:36,745 - lava_callback - INFO - Starting async job submit for node 69c08c36ce0c9bc43bccdd01 2026-03-25 00:41:36,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:36,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c36ce0c9bc43bccdd01 HTTP/1.1" 200 None 2026-03-25 00:41:36,760 - lava_callback - DEBUG - Processing node 69c08c36ce0c9bc43bccdd01: baseline-arm-clabbe 2026-03-25 00:41:36,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c36ce0c9bc43bccdd01 2026-03-25 00:41:36,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:36,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c36ce0c9bc43bccdd01 HTTP/1.1" 200 None 2026-03-25 00:41:36,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:36,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:36,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:36,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c36ce0c9bc43bccdd01 HTTP/1.1" 200 None 2026-03-25 00:41:36,820 - lava_callback - ERROR - Error processing callback for node 69c08c36ce0c9bc43bccdd01: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 00:41:37,403 - lava_callback - INFO - Received callback for node 69c08c37ce0c9bc43bccdd04 INFO: 172.21.0.1:43794 - "POST /node/69c08c37ce0c9bc43bccdd04 HTTP/1.0" 202 Accepted 2026-03-25 00:41:37,409 - lava_callback - INFO - Starting async job submit for node 69c08c37ce0c9bc43bccdd04 2026-03-25 00:41:37,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:37,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c37ce0c9bc43bccdd04 HTTP/1.1" 200 None 2026-03-25 00:41:37,425 - lava_callback - DEBUG - Processing node 69c08c37ce0c9bc43bccdd04: baseline-arm-clabbe 2026-03-25 00:41:37,426 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c37ce0c9bc43bccdd04 2026-03-25 00:41:37,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:37,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c37ce0c9bc43bccdd04 HTTP/1.1" 200 None 2026-03-25 00:41:37,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:37,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:37,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:37,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c37ce0c9bc43bccdd04 HTTP/1.1" 200 None 2026-03-25 00:41:37,479 - lava_callback - ERROR - Error processing callback for node 69c08c37ce0c9bc43bccdd04: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 00:41:38,068 - lava_callback - INFO - Received callback for node 69c08c38ce0c9bc43bccdd07 INFO: 172.21.0.1:43810 - "POST /node/69c08c38ce0c9bc43bccdd07 HTTP/1.0" 202 Accepted 2026-03-25 00:41:38,074 - lava_callback - INFO - Starting async job submit for node 69c08c38ce0c9bc43bccdd07 2026-03-25 00:41:38,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:38,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c38ce0c9bc43bccdd07 HTTP/1.1" 200 None 2026-03-25 00:41:38,090 - lava_callback - DEBUG - Processing node 69c08c38ce0c9bc43bccdd07: baseline-arm-clabbe 2026-03-25 00:41:38,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c38ce0c9bc43bccdd07 2026-03-25 00:41:38,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:38,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c38ce0c9bc43bccdd07 HTTP/1.1" 200 None 2026-03-25 00:41:38,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:38,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:38,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:38,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c38ce0c9bc43bccdd07 HTTP/1.1" 200 None 2026-03-25 00:41:38,146 - lava_callback - ERROR - Error processing callback for node 69c08c38ce0c9bc43bccdd07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 00:41:38,764 - lava_callback - INFO - Received callback for node 69c08c3bce0c9bc43bccdd10 INFO: 172.21.0.1:43816 - "POST /node/69c08c3bce0c9bc43bccdd10 HTTP/1.0" 202 Accepted 2026-03-25 00:41:38,772 - lava_callback - INFO - Starting async job submit for node 69c08c3bce0c9bc43bccdd10 2026-03-25 00:41:38,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:38,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c3bce0c9bc43bccdd10 HTTP/1.1" 200 None 2026-03-25 00:41:38,788 - lava_callback - DEBUG - Processing node 69c08c3bce0c9bc43bccdd10: baseline-arm-clabbe 2026-03-25 00:41:38,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c3bce0c9bc43bccdd10 2026-03-25 00:41:38,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:38,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c3bce0c9bc43bccdd10 HTTP/1.1" 200 None 2026-03-25 00:41:38,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:38,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 00:41:38,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:38,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c3bce0c9bc43bccdd10 HTTP/1.1" 200 None 2026-03-25 00:41:38,851 - lava_callback - ERROR - Error processing callback for node 69c08c3bce0c9bc43bccdd10: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36560 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:41:50,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:50,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-25 00:41:55,402 - lava_callback - INFO - Received callback for node 69c32d7e18a3717aa65d02c8 INFO: 172.21.0.1:58924 - "POST /node/69c32d7e18a3717aa65d02c8 HTTP/1.0" 202 Accepted 2026-03-25 00:41:55,408 - lava_callback - INFO - Starting async job submit for node 69c32d7e18a3717aa65d02c8 2026-03-25 00:41:55,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:55,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d7e18a3717aa65d02c8 HTTP/1.1" 200 None 2026-03-25 00:41:55,449 - lava_callback - DEBUG - Processing node 69c32d7e18a3717aa65d02c8: ltp-containers 2026-03-25 00:41:55,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:41:55,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:41:55,997 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c32d7e18a3717aa65d02c8/log.txt.gz 2026-03-25 00:41:56,070 - lava_callback - INFO - Received callback for node 69c08c3fce0c9bc43bccdd22 INFO: 172.21.0.1:58940 - "POST /node/69c08c3fce0c9bc43bccdd22 HTTP/1.0" 202 Accepted 2026-03-25 00:41:56,085 - lava_callback - INFO - Starting async job submit for node 69c08c3fce0c9bc43bccdd22 2026-03-25 00:41:56,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:56,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c3fce0c9bc43bccdd22 HTTP/1.1" 200 None 2026-03-25 00:41:56,175 - lava_callback - DEBUG - Processing node 69c08c3fce0c9bc43bccdd22: baseline-arm-clabbe 2026-03-25 00:41:56,181 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c3fce0c9bc43bccdd22 2026-03-25 00:41:56,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:56,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:41:56,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c3fce0c9bc43bccdd22 HTTP/1.1" 200 None 2026-03-25 00:41:56,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:56,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 00:41:56,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:56,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c3fce0c9bc43bccdd22 HTTP/1.1" 200 None 2026-03-25 00:41:56,316 - lava_callback - ERROR - Error processing callback for node 69c08c3fce0c9bc43bccdd22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58956 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:41:56,602 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:41:56,603 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c32d7e18a3717aa65d02c8/lava-logs.yaml 2026-03-25 00:41:56,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:41:56,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:41:56,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c32d7e18a3717aa65d02c8/lava_callback.json.gz 2026-03-25 00:41:56,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:56,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d7e18a3717aa65d02c8 HTTP/1.1" 200 None 2026-03-25 00:41:56,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:56,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:41:56,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:41:56,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d7e18a3717aa65d02c8 HTTP/1.1" 200 None 2026-03-25 00:41:56,856 - lava_callback - INFO - Completed processing callback for node 69c32d7e18a3717aa65d02c8 INFO: 172.21.0.1:60338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38972 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 00:42:16,063 - lava_callback - INFO - Received callback for node 69c08c64ce0c9bc43bccdd96 INFO: 172.21.0.1:38978 - "POST /node/69c08c64ce0c9bc43bccdd96 HTTP/1.0" 202 Accepted 2026-03-25 00:42:16,068 - lava_callback - INFO - Starting async job submit for node 69c08c64ce0c9bc43bccdd96 2026-03-25 00:42:16,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:42:16,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c64ce0c9bc43bccdd96 HTTP/1.1" 200 None 2026-03-25 00:42:16,095 - lava_callback - DEBUG - Processing node 69c08c64ce0c9bc43bccdd96: baseline-arm-clabbe 2026-03-25 00:42:16,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08c64ce0c9bc43bccdd96 2026-03-25 00:42:16,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:42:16,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08c64ce0c9bc43bccdd96 HTTP/1.1" 200 None 2026-03-25 00:42:16,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:42:16,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 00:42:16,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:42:16,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08c64ce0c9bc43bccdd96 HTTP/1.1" 200 None 2026-03-25 00:42:16,146 - lava_callback - ERROR - Error processing callback for node 69c08c64ce0c9bc43bccdd96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38988 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:42:18,100 - lava_callback - INFO - Received callback for node 69c32f2918a3717aa65d29bb INFO: 172.21.0.1:39002 - "POST /node/69c32f2918a3717aa65d29bb HTTP/1.0" 202 Accepted 2026-03-25 00:42:18,105 - lava_callback - INFO - Starting async job submit for node 69c32f2918a3717aa65d29bb 2026-03-25 00:42:18,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:42:18,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32f2918a3717aa65d29bb HTTP/1.1" 200 None 2026-03-25 00:42:18,143 - lava_callback - DEBUG - Processing node 69c32f2918a3717aa65d29bb: baseline-arm-pengutronix 2026-03-25 00:42:18,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:42:18,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:42:18,429 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32f2918a3717aa65d29bb/log.txt.gz 2026-03-25 00:42:18,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:42:18,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:42:18,722 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32f2918a3717aa65d29bb/lava-logs.yaml 2026-03-25 00:42:18,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:42:18,860 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:42:18,862 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32f2918a3717aa65d29bb/lava_callback.json.gz 2026-03-25 00:42:18,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:42:18,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32f2918a3717aa65d29bb HTTP/1.1" 200 None 2026-03-25 00:42:18,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:42:18,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:42:18,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:42:18,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32f2918a3717aa65d29bb HTTP/1.1" 200 None 2026-03-25 00:42:18,944 - lava_callback - INFO - Completed processing callback for node 69c32f2918a3717aa65d29bb 2026-03-25 00:42:20,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:42:20,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50382 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:43:24,419 - lava_callback - INFO - Received callback for node 69c32d8718a3717aa65d02d7 INFO: 172.21.0.1:43176 - "POST /node/69c32d8718a3717aa65d02d7 HTTP/1.0" 202 Accepted 2026-03-25 00:43:24,425 - lava_callback - INFO - Starting async job submit for node 69c32d8718a3717aa65d02d7 2026-03-25 00:43:24,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:43:24,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8718a3717aa65d02d7 HTTP/1.1" 200 None 2026-03-25 00:43:24,465 - lava_callback - DEBUG - Processing node 69c32d8718a3717aa65d02d7: ltp-fsx 2026-03-25 00:43:24,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:43:24,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:43:24,991 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c32d8718a3717aa65d02d7/log.txt.gz 2026-03-25 00:43:25,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:43:25,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:43:25,401 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c32d8718a3717aa65d02d7/lava-logs.yaml 2026-03-25 00:43:25,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:43:25,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:43:25,561 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c32d8718a3717aa65d02d7/lava_callback.json.gz 2026-03-25 00:43:25,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:43:25,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8718a3717aa65d02d7 HTTP/1.1" 200 None 2026-03-25 00:43:25,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:43:25,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:43:25,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:43:25,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d8718a3717aa65d02d7 HTTP/1.1" 200 None 2026-03-25 00:43:25,650 - lava_callback - INFO - Completed processing callback for node 69c32d8718a3717aa65d02d7 INFO: 172.21.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38768 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:43:36,066 - lava_callback - INFO - Received callback for node 69c32d8818a3717aa65d02ec INFO: 172.21.0.1:38780 - "POST /node/69c32d8818a3717aa65d02ec HTTP/1.0" 202 Accepted 2026-03-25 00:43:36,072 - lava_callback - INFO - Starting async job submit for node 69c32d8818a3717aa65d02ec 2026-03-25 00:43:36,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:43:36,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8818a3717aa65d02ec HTTP/1.1" 200 None 2026-03-25 00:43:36,112 - lava_callback - DEBUG - Processing node 69c32d8818a3717aa65d02ec: ltp-input 2026-03-25 00:43:36,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:38784 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:43:36,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:43:36,626 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c32d8818a3717aa65d02ec/log.txt.gz 2026-03-25 00:43:36,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:43:37,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:43:37,017 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c32d8818a3717aa65d02ec/lava-logs.yaml 2026-03-25 00:43:37,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:43:37,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:43:37,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c32d8818a3717aa65d02ec/lava_callback.json.gz 2026-03-25 00:43:37,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:43:37,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8818a3717aa65d02ec HTTP/1.1" 200 None 2026-03-25 00:43:37,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:43:37,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:43:37,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:43:37,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d8818a3717aa65d02ec HTTP/1.1" 200 None 2026-03-25 00:43:37,256 - lava_callback - INFO - Completed processing callback for node 69c32d8818a3717aa65d02ec INFO: 172.21.0.1:44480 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:43:50,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:43:50,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41792 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:44:23,583 - lava_callback - INFO - Received callback for node 69c32fe018a3717aa65d2a30 INFO: 172.21.0.1:59082 - "POST /node/69c32fe018a3717aa65d2a30 HTTP/1.0" 202 Accepted 2026-03-25 00:44:23,589 - lava_callback - INFO - Starting async job submit for node 69c32fe018a3717aa65d2a30 2026-03-25 00:44:23,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:44:23,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32fe018a3717aa65d2a30 HTTP/1.1" 200 None 2026-03-25 00:44:23,608 - lava_callback - DEBUG - Processing node 69c32fe018a3717aa65d2a30: baseline-arm 2026-03-25 00:44:23,613 - lava_callback - WARNING - Failed to upload log 2026-03-25 00:44:23,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:44:23,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:44:23,731 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c32fe018a3717aa65d2a30/lava-logs.yaml 2026-03-25 00:44:23,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:44:23,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:44:23,841 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c32fe018a3717aa65d2a30/lava_callback.json.gz 2026-03-25 00:44:23,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:44:23,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32fe018a3717aa65d2a30 HTTP/1.1" 200 None 2026-03-25 00:44:23,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:44:23,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:44:23,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:44:23,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32fe018a3717aa65d2a30 HTTP/1.1" 200 None 2026-03-25 00:44:23,927 - lava_callback - INFO - Completed processing callback for node 69c32fe018a3717aa65d2a30 INFO: 172.21.0.1:59090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44404 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:44:50,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:44:50,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59858 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:51:22,809 - lava_callback - INFO - Received callback for node 69c32d4718a3717aa65d0216 INFO: 172.21.0.1:52098 - "POST /node/69c32d4718a3717aa65d0216 HTTP/1.0" 202 Accepted 2026-03-25 00:51:22,818 - lava_callback - INFO - Starting async job submit for node 69c32d4718a3717aa65d0216 2026-03-25 00:51:22,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:51:22,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4718a3717aa65d0216 HTTP/1.1" 200 None 2026-03-25 00:51:22,906 - lava_callback - DEBUG - Processing node 69c32d4718a3717aa65d0216: baseline-arm 2026-03-25 00:51:23,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:51:24,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:51:24,252 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4718a3717aa65d0216/log.txt.gz 2026-03-25 00:51:24,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:51:25,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:51:25,553 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4718a3717aa65d0216/lava-logs.yaml 2026-03-25 00:51:25,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:51:26,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:51:26,145 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c32d4718a3717aa65d0216/lava_callback.json.gz DEBUG: Setup failed for 69c32d4718a3717aa65d0216. Transitting job node result: incomplete -> fail 2026-03-25 00:51:26,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:51:26,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d4718a3717aa65d0216 HTTP/1.1" 200 None 2026-03-25 00:51:26,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:51:26,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:51:26,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:52108 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:51:26,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d4718a3717aa65d0216 HTTP/1.1" 200 None 2026-03-25 00:51:26,690 - lava_callback - INFO - Completed processing callback for node 69c32d4718a3717aa65d0216 INFO: 172.21.0.1:40470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48144 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:51:50,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:51:50,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39628 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:53:25,695 - lava_callback - INFO - Received callback for node 69c32d6b18a3717aa65d02ae INFO: 172.21.0.1:60452 - "POST /node/69c32d6b18a3717aa65d02ae HTTP/1.0" 202 Accepted 2026-03-25 00:53:25,741 - lava_callback - INFO - Starting async job submit for node 69c32d6b18a3717aa65d02ae 2026-03-25 00:53:25,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:53:25,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6b18a3717aa65d02ae HTTP/1.1" 200 None 2026-03-25 00:53:25,781 - lava_callback - DEBUG - Processing node 69c32d6b18a3717aa65d02ae: baseline-arm-pengutronix 2026-03-25 00:53:25,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:53:26,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:53:26,035 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6b18a3717aa65d02ae/log.txt.gz 2026-03-25 00:53:26,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:53:26,294 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:53:26,296 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6b18a3717aa65d02ae/lava-logs.yaml 2026-03-25 00:53:26,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:53:26,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:53:26,431 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c32d6b18a3717aa65d02ae/lava_callback.json.gz DEBUG: Setup failed for 69c32d6b18a3717aa65d02ae. Transitting job node result: incomplete -> fail 2026-03-25 00:53:26,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:53:26,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d6b18a3717aa65d02ae HTTP/1.1" 200 None 2026-03-25 00:53:26,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:53:26,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 00:53:26,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:60468 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:53:26,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d6b18a3717aa65d02ae HTTP/1.1" 200 None 2026-03-25 00:53:26,604 - lava_callback - INFO - Completed processing callback for node 69c32d6b18a3717aa65d02ae INFO: 172.21.0.1:51060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59210 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:53:50,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:53:50,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59370 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:54:04,563 - lava_callback - INFO - Received callback for node 69c32d9818a3717aa65d0318 INFO: 172.21.0.1:48104 - "POST /node/69c32d9818a3717aa65d0318 HTTP/1.0" 202 Accepted 2026-03-25 00:54:04,570 - lava_callback - INFO - Starting async job submit for node 69c32d9818a3717aa65d0318 2026-03-25 00:54:04,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:54:04,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9818a3717aa65d0318 HTTP/1.1" 200 None 2026-03-25 00:54:04,654 - lava_callback - DEBUG - Processing node 69c32d9818a3717aa65d0318: ltp-smoke 2026-03-25 00:54:05,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:54:05,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:54:05,302 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c32d9818a3717aa65d0318/log.txt.gz 2026-03-25 00:54:05,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:54:05,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:54:05,794 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c32d9818a3717aa65d0318/lava-logs.yaml 2026-03-25 00:54:05,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:54:05,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:54:05,970 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c32d9818a3717aa65d0318/lava_callback.json.gz 2026-03-25 00:54:06,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:54:06,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9818a3717aa65d0318 HTTP/1.1" 200 None 2026-03-25 00:54:06,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:54:06,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3294718a3717aa65d01b3 HTTP/1.1" 200 None 2026-03-25 00:54:06,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:54:06,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d9818a3717aa65d0318 HTTP/1.1" 200 None 2026-03-25 00:54:06,284 - lava_callback - INFO - Completed processing callback for node 69c32d9818a3717aa65d0318 INFO: 172.21.0.1:48110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42488 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:54:20,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:54:20,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41266 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:55:32,252 - lava_callback - INFO - Received callback for node 69c08f7ece0c9bc43bccdf71 INFO: 172.21.0.1:41278 - "POST /node/69c08f7ece0c9bc43bccdf71 HTTP/1.0" 202 Accepted 2026-03-25 00:55:32,258 - lava_callback - INFO - Starting async job submit for node 69c08f7ece0c9bc43bccdf71 2026-03-25 00:55:32,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:55:32,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08f7ece0c9bc43bccdf71 HTTP/1.1" 200 None 2026-03-25 00:55:32,274 - lava_callback - DEBUG - Processing node 69c08f7ece0c9bc43bccdf71: baseline-arm-baylibre 2026-03-25 00:55:32,274 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08f7ece0c9bc43bccdf71 2026-03-25 00:55:32,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:55:32,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08f7ece0c9bc43bccdf71 HTTP/1.1" 200 None 2026-03-25 00:55:32,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:55:32,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 00:55:32,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:55:32,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08f7ece0c9bc43bccdf71 HTTP/1.1" 200 None 2026-03-25 00:55:32,339 - lava_callback - ERROR - Error processing callback for node 69c08f7ece0c9bc43bccdf71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53640 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:55:54,823 - lava_callback - INFO - Received callback for node 69c3328118a3717aa65d2b0b INFO: 172.21.0.1:57870 - "POST /node/69c3328118a3717aa65d2b0b HTTP/1.0" 202 Accepted 2026-03-25 00:55:54,829 - lava_callback - INFO - Starting async job submit for node 69c3328118a3717aa65d2b0b 2026-03-25 00:55:54,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:55:54,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3328118a3717aa65d2b0b HTTP/1.1" 200 None 2026-03-25 00:55:54,863 - lava_callback - DEBUG - Processing node 69c3328118a3717aa65d2b0b: baseline-x86 2026-03-25 00:55:55,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:55:55,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:55:55,198 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328118a3717aa65d2b0b/log.txt.gz 2026-03-25 00:55:55,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:55:55,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:55:55,542 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328118a3717aa65d2b0b/lava-logs.yaml 2026-03-25 00:55:55,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:55:55,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:55:55,721 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328118a3717aa65d2b0b/lava_callback.json.gz 2026-03-25 00:55:55,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:55:55,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3328118a3717aa65d2b0b HTTP/1.1" 200 None 2026-03-25 00:55:55,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:55:55,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3296e18a3717aa65d01dd HTTP/1.1" 200 None 2026-03-25 00:55:55,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:55:55,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3328118a3717aa65d2b0b HTTP/1.1" 200 None 2026-03-25 00:55:55,797 - lava_callback - INFO - Completed processing callback for node 69c3328118a3717aa65d2b0b INFO: 172.21.0.1:57876 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:56:02,001 - lava_callback - INFO - Received callback for node 69c3328318a3717aa65d2b15 INFO: 172.21.0.1:35010 - "POST /node/69c3328318a3717aa65d2b15 HTTP/1.0" 202 Accepted 2026-03-25 00:56:02,007 - lava_callback - INFO - Starting async job submit for node 69c3328318a3717aa65d2b15 2026-03-25 00:56:02,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:02,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3328318a3717aa65d2b15 HTTP/1.1" 200 None 2026-03-25 00:56:02,039 - lava_callback - DEBUG - Processing node 69c3328318a3717aa65d2b15: baseline-x86 2026-03-25 00:56:02,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:56:02,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:56:02,350 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328318a3717aa65d2b15/log.txt.gz 2026-03-25 00:56:02,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:56:02,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:56:02,702 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328318a3717aa65d2b15/lava-logs.yaml 2026-03-25 00:56:02,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:56:02,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:56:02,843 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328318a3717aa65d2b15/lava_callback.json.gz 2026-03-25 00:56:02,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:02,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3328318a3717aa65d2b15 HTTP/1.1" 200 None 2026-03-25 00:56:02,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:02,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3296e18a3717aa65d01dd HTTP/1.1" 200 None 2026-03-25 00:56:02,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:02,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3328318a3717aa65d2b15 HTTP/1.1" 200 None 2026-03-25 00:56:02,918 - lava_callback - INFO - Completed processing callback for node 69c3328318a3717aa65d2b15 INFO: 172.21.0.1:35014 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:56:11,958 - lava_callback - INFO - Received callback for node 69c08fa2ce0c9bc43bccdf7f INFO: 172.21.0.1:41336 - "POST /node/69c08fa2ce0c9bc43bccdf7f HTTP/1.0" 202 Accepted 2026-03-25 00:56:11,964 - lava_callback - INFO - Starting async job submit for node 69c08fa2ce0c9bc43bccdf7f 2026-03-25 00:56:11,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:11,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08fa2ce0c9bc43bccdf7f HTTP/1.1" 200 None 2026-03-25 00:56:11,980 - lava_callback - DEBUG - Processing node 69c08fa2ce0c9bc43bccdf7f: baseline-arm-baylibre 2026-03-25 00:56:11,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08fa2ce0c9bc43bccdf7f 2026-03-25 00:56:11,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:11,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08fa2ce0c9bc43bccdf7f HTTP/1.1" 200 None 2026-03-25 00:56:11,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:12,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 00:56:12,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:12,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08fa2ce0c9bc43bccdf7f HTTP/1.1" 200 None 2026-03-25 00:56:12,042 - lava_callback - ERROR - Error processing callback for node 69c08fa2ce0c9bc43bccdf7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41340 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:56:20,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:20,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56250 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:56:47,841 - lava_callback - INFO - Received callback for node 69c3328218a3717aa65d2b12 INFO: 172.21.0.1:56252 - "POST /node/69c3328218a3717aa65d2b12 HTTP/1.0" 202 Accepted 2026-03-25 00:56:47,847 - lava_callback - INFO - Starting async job submit for node 69c3328218a3717aa65d2b12 2026-03-25 00:56:47,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:47,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3328218a3717aa65d2b12 HTTP/1.1" 200 None 2026-03-25 00:56:47,908 - lava_callback - DEBUG - Processing node 69c3328218a3717aa65d2b12: baseline-x86 2026-03-25 00:56:48,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:56:48,293 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:56:48,295 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328218a3717aa65d2b12/log.txt.gz 2026-03-25 00:56:48,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:56:48,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:56:48,648 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328218a3717aa65d2b12/lava-logs.yaml 2026-03-25 00:56:48,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:56:48,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:56:48,786 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328218a3717aa65d2b12/lava_callback.json.gz 2026-03-25 00:56:48,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:48,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3328218a3717aa65d2b12 HTTP/1.1" 200 None 2026-03-25 00:56:48,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:48,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3296e18a3717aa65d01dd HTTP/1.1" 200 None 2026-03-25 00:56:48,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:48,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3328218a3717aa65d2b12 HTTP/1.1" 200 None 2026-03-25 00:56:48,879 - lava_callback - INFO - Completed processing callback for node 69c3328218a3717aa65d2b12 2026-03-25 00:56:50,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:50,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-25 00:56:52,025 - lava_callback - INFO - Received callback for node 69c08fc6ce0c9bc43bccdf8a INFO: 172.21.0.1:35366 - "POST /node/69c08fc6ce0c9bc43bccdf8a HTTP/1.0" 202 Accepted 2026-03-25 00:56:52,031 - lava_callback - INFO - Starting async job submit for node 69c08fc6ce0c9bc43bccdf8a 2026-03-25 00:56:52,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:52,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08fc6ce0c9bc43bccdf8a HTTP/1.1" 200 None 2026-03-25 00:56:52,047 - lava_callback - DEBUG - Processing node 69c08fc6ce0c9bc43bccdf8a: baseline-arm-baylibre 2026-03-25 00:56:52,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08fc6ce0c9bc43bccdf8a 2026-03-25 00:56:52,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:52,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08fc6ce0c9bc43bccdf8a HTTP/1.1" 200 None 2026-03-25 00:56:52,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:52,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 00:56:52,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:52,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08fc6ce0c9bc43bccdf8a HTTP/1.1" 200 None 2026-03-25 00:56:52,104 - lava_callback - ERROR - Error processing callback for node 69c08fc6ce0c9bc43bccdf8a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 00:56:53,292 - lava_callback - INFO - Received callback for node 69c3328118a3717aa65d2b0e INFO: 172.21.0.1:35372 - "POST /node/69c3328118a3717aa65d2b0e HTTP/1.0" 202 Accepted 2026-03-25 00:56:53,298 - lava_callback - INFO - Starting async job submit for node 69c3328118a3717aa65d2b0e 2026-03-25 00:56:53,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:53,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3328118a3717aa65d2b0e HTTP/1.1" 200 None 2026-03-25 00:56:53,334 - lava_callback - DEBUG - Processing node 69c3328118a3717aa65d2b0e: baseline-x86 2026-03-25 00:56:53,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:56:53,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:56:53,770 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328118a3717aa65d2b0e/log.txt.gz 2026-03-25 00:56:53,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:56:54,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:56:54,141 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328118a3717aa65d2b0e/lava-logs.yaml 2026-03-25 00:56:54,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 00:56:54,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 00:56:54,287 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c3328118a3717aa65d2b0e/lava_callback.json.gz 2026-03-25 00:56:54,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:54,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3328118a3717aa65d2b0e HTTP/1.1" 200 None 2026-03-25 00:56:54,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:54,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3296e18a3717aa65d01dd HTTP/1.1" 200 None 2026-03-25 00:56:54,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:56:54,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3328118a3717aa65d2b0e HTTP/1.1" 200 None 2026-03-25 00:56:54,364 - lava_callback - INFO - Completed processing callback for node 69c3328118a3717aa65d2b0e INFO: 172.21.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52014 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:57:20,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:57:20,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51670 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:57:32,036 - lava_callback - INFO - Received callback for node 69c08febce0c9bc43bccdf95 INFO: 172.21.0.1:51676 - "POST /node/69c08febce0c9bc43bccdf95 HTTP/1.0" 202 Accepted 2026-03-25 00:57:32,042 - lava_callback - INFO - Starting async job submit for node 69c08febce0c9bc43bccdf95 2026-03-25 00:57:32,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:57:32,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08febce0c9bc43bccdf95 HTTP/1.1" 200 None 2026-03-25 00:57:32,058 - lava_callback - DEBUG - Processing node 69c08febce0c9bc43bccdf95: baseline-arm-baylibre 2026-03-25 00:57:32,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c08febce0c9bc43bccdf95 2026-03-25 00:57:32,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:57:32,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c08febce0c9bc43bccdf95 HTTP/1.1" 200 None 2026-03-25 00:57:32,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:57:32,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 00:57:32,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:57:32,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c08febce0c9bc43bccdf95 HTTP/1.1" 200 None 2026-03-25 00:57:32,131 - lava_callback - ERROR - Error processing callback for node 69c08febce0c9bc43bccdf95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51334 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:58:12,067 - lava_callback - INFO - Received callback for node 69c09013ce0c9bc43bccdfa8 INFO: 172.21.0.1:36334 - "POST /node/69c09013ce0c9bc43bccdfa8 HTTP/1.0" 202 Accepted 2026-03-25 00:58:12,073 - lava_callback - INFO - Starting async job submit for node 69c09013ce0c9bc43bccdfa8 2026-03-25 00:58:12,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:58:12,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09013ce0c9bc43bccdfa8 HTTP/1.1" 200 None 2026-03-25 00:58:12,095 - lava_callback - DEBUG - Processing node 69c09013ce0c9bc43bccdfa8: baseline-arm-baylibre 2026-03-25 00:58:12,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09013ce0c9bc43bccdfa8 2026-03-25 00:58:12,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:58:12,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09013ce0c9bc43bccdfa8 HTTP/1.1" 200 None 2026-03-25 00:58:12,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:58:12,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 00:58:12,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:58:12,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09013ce0c9bc43bccdfa8 HTTP/1.1" 200 None 2026-03-25 00:58:12,149 - lava_callback - ERROR - Error processing callback for node 69c09013ce0c9bc43bccdfa8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42618 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:58:56,887 - lava_callback - INFO - Received callback for node 69c0903bce0c9bc43bccdfbc INFO: 172.21.0.1:42632 - "POST /node/69c0903bce0c9bc43bccdfbc HTTP/1.0" 202 Accepted 2026-03-25 00:58:56,893 - lava_callback - INFO - Starting async job submit for node 69c0903bce0c9bc43bccdfbc 2026-03-25 00:58:56,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:58:56,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0903bce0c9bc43bccdfbc HTTP/1.1" 200 None 2026-03-25 00:58:56,908 - lava_callback - DEBUG - Processing node 69c0903bce0c9bc43bccdfbc: baseline-arm-clabbe 2026-03-25 00:58:56,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0903bce0c9bc43bccdfbc 2026-03-25 00:58:56,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:58:56,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0903bce0c9bc43bccdfbc HTTP/1.1" 200 None 2026-03-25 00:58:56,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:58:56,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 00:58:56,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:58:56,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0903bce0c9bc43bccdfbc HTTP/1.1" 200 None 2026-03-25 00:58:56,960 - lava_callback - ERROR - Error processing callback for node 69c0903bce0c9bc43bccdfbc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34008 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:59:16,879 - lava_callback - INFO - Received callback for node 69c09061ce0c9bc43bccdfc9 INFO: 172.21.0.1:34010 - "POST /node/69c09061ce0c9bc43bccdfc9 HTTP/1.0" 202 Accepted 2026-03-25 00:59:16,885 - lava_callback - INFO - Starting async job submit for node 69c09061ce0c9bc43bccdfc9 2026-03-25 00:59:16,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:59:16,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09061ce0c9bc43bccdfc9 HTTP/1.1" 200 None 2026-03-25 00:59:16,900 - lava_callback - DEBUG - Processing node 69c09061ce0c9bc43bccdfc9: baseline-arm-clabbe 2026-03-25 00:59:16,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09061ce0c9bc43bccdfc9 2026-03-25 00:59:16,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:59:16,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09061ce0c9bc43bccdfc9 HTTP/1.1" 200 None 2026-03-25 00:59:16,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:59:16,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 00:59:16,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:59:16,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09061ce0c9bc43bccdfc9 HTTP/1.1" 200 None 2026-03-25 00:59:16,964 - lava_callback - ERROR - Error processing callback for node 69c09061ce0c9bc43bccdfc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51554 - "GET / HTTP/1.0" 200 OK 2026-03-25 00:59:56,893 - lava_callback - INFO - Received callback for node 69c09088ce0c9bc43bccdfd1 INFO: 172.21.0.1:51566 - "POST /node/69c09088ce0c9bc43bccdfd1 HTTP/1.0" 202 Accepted 2026-03-25 00:59:56,899 - lava_callback - INFO - Starting async job submit for node 69c09088ce0c9bc43bccdfd1 2026-03-25 00:59:56,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:59:56,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09088ce0c9bc43bccdfd1 HTTP/1.1" 200 None 2026-03-25 00:59:56,916 - lava_callback - DEBUG - Processing node 69c09088ce0c9bc43bccdfd1: baseline-arm-clabbe 2026-03-25 00:59:56,916 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09088ce0c9bc43bccdfd1 2026-03-25 00:59:56,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:59:56,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09088ce0c9bc43bccdfd1 HTTP/1.1" 200 None 2026-03-25 00:59:56,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:59:56,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 00:59:56,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 00:59:56,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09088ce0c9bc43bccdfd1 HTTP/1.1" 200 None 2026-03-25 00:59:56,976 - lava_callback - ERROR - Error processing callback for node 69c09088ce0c9bc43bccdfd1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44398 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:00:32,230 - lava_callback - INFO - Received callback for node 69c090acce0c9bc43bccdfdc INFO: 172.21.0.1:44402 - "POST /node/69c090acce0c9bc43bccdfdc HTTP/1.0" 202 Accepted 2026-03-25 01:00:32,235 - lava_callback - INFO - Starting async job submit for node 69c090acce0c9bc43bccdfdc 2026-03-25 01:00:32,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:00:32,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c090acce0c9bc43bccdfdc HTTP/1.1" 200 None 2026-03-25 01:00:32,257 - lava_callback - DEBUG - Processing node 69c090acce0c9bc43bccdfdc: baseline-arm-baylibre 2026-03-25 01:00:32,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c090acce0c9bc43bccdfdc 2026-03-25 01:00:32,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:00:32,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c090acce0c9bc43bccdfdc HTTP/1.1" 200 None 2026-03-25 01:00:32,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:00:32,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 01:00:32,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:00:32,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c090acce0c9bc43bccdfdc HTTP/1.1" 200 None 2026-03-25 01:00:32,327 - lava_callback - ERROR - Error processing callback for node 69c090acce0c9bc43bccdfdc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49636 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:00:56,037 - lava_callback - INFO - Received callback for node 69c32d8a18a3717aa65d02ef INFO: 172.21.0.1:36464 - "POST /node/69c32d8a18a3717aa65d02ef HTTP/1.0" 202 Accepted 2026-03-25 01:00:56,043 - lava_callback - INFO - Starting async job submit for node 69c32d8a18a3717aa65d02ef 2026-03-25 01:00:56,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:00:56,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8a18a3717aa65d02ef HTTP/1.1" 200 None 2026-03-25 01:00:56,083 - lava_callback - DEBUG - Processing node 69c32d8a18a3717aa65d02ef: ltp-ipc 2026-03-25 01:00:56,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:36468 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:00:56,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:00:56,562 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c32d8a18a3717aa65d02ef/log.txt.gz 2026-03-25 01:00:56,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:00:57,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:00:57,020 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c32d8a18a3717aa65d02ef/lava-logs.yaml 2026-03-25 01:00:57,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:00:57,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:00:57,207 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c32d8a18a3717aa65d02ef/lava_callback.json.gz 2026-03-25 01:00:57,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:00:57,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8a18a3717aa65d02ef HTTP/1.1" 200 None 2026-03-25 01:00:57,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:00:57,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 01:00:57,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:00:57,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d8a18a3717aa65d02ef HTTP/1.1" 200 None 2026-03-25 01:00:57,294 - lava_callback - INFO - Completed processing callback for node 69c32d8a18a3717aa65d02ef INFO: 172.21.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45776 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:01:16,960 - lava_callback - INFO - Received callback for node 69c090d2ce0c9bc43bccdfe9 INFO: 172.21.0.1:45780 - "POST /node/69c090d2ce0c9bc43bccdfe9 HTTP/1.0" 202 Accepted 2026-03-25 01:01:16,966 - lava_callback - INFO - Starting async job submit for node 69c090d2ce0c9bc43bccdfe9 2026-03-25 01:01:16,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:01:16,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c090d2ce0c9bc43bccdfe9 HTTP/1.1" 200 None 2026-03-25 01:01:16,982 - lava_callback - DEBUG - Processing node 69c090d2ce0c9bc43bccdfe9: baseline-arm-clabbe 2026-03-25 01:01:16,982 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c090d2ce0c9bc43bccdfe9 2026-03-25 01:01:16,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:01:17,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c090d2ce0c9bc43bccdfe9 HTTP/1.1" 200 None 2026-03-25 01:01:17,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:01:17,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 01:01:17,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:01:17,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c090d2ce0c9bc43bccdfe9 HTTP/1.1" 200 None 2026-03-25 01:01:17,060 - lava_callback - ERROR - Error processing callback for node 69c090d2ce0c9bc43bccdfe9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:01:20,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:01:20,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56548 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:01:36,341 - lava_callback - INFO - Received callback for node 69c32d8b18a3717aa65d02f2 INFO: 172.21.0.1:56558 - "POST /node/69c32d8b18a3717aa65d02f2 HTTP/1.0" 202 Accepted 2026-03-25 01:01:36,347 - lava_callback - INFO - Starting async job submit for node 69c32d8b18a3717aa65d02f2 2026-03-25 01:01:36,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:01:36,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8b18a3717aa65d02f2 HTTP/1.1" 200 None 2026-03-25 01:01:36,388 - lava_callback - DEBUG - Processing node 69c32d8b18a3717aa65d02f2: ltp-pty INFO: 172.21.0.1:56566 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:01:36,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:01:36,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:01:36,885 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c32d8b18a3717aa65d02f2/log.txt.gz 2026-03-25 01:01:37,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:01:37,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:01:37,306 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c32d8b18a3717aa65d02f2/lava-logs.yaml 2026-03-25 01:01:37,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:01:37,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:01:37,492 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c32d8b18a3717aa65d02f2/lava_callback.json.gz 2026-03-25 01:01:37,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:01:37,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8b18a3717aa65d02f2 HTTP/1.1" 200 None 2026-03-25 01:01:37,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:01:37,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 01:01:37,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:01:37,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d8b18a3717aa65d02f2 HTTP/1.1" 200 None 2026-03-25 01:01:37,589 - lava_callback - INFO - Completed processing callback for node 69c32d8b18a3717aa65d02f2 INFO: 172.21.0.1:58418 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:01:50,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:01:50,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58798 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:02:37,049 - lava_callback - INFO - Received callback for node 69c0911bce0c9bc43bcce006 INFO: 172.21.0.1:58806 - "POST /node/69c0911bce0c9bc43bcce006 HTTP/1.0" 202 Accepted 2026-03-25 01:02:37,054 - lava_callback - INFO - Starting async job submit for node 69c0911bce0c9bc43bcce006 2026-03-25 01:02:37,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:02:37,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0911bce0c9bc43bcce006 HTTP/1.1" 200 None 2026-03-25 01:02:37,072 - lava_callback - DEBUG - Processing node 69c0911bce0c9bc43bcce006: baseline-arm-clabbe 2026-03-25 01:02:37,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0911bce0c9bc43bcce006 2026-03-25 01:02:37,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:02:37,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0911bce0c9bc43bcce006 HTTP/1.1" 200 None 2026-03-25 01:02:37,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:02:37,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 01:02:37,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:02:37,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0911bce0c9bc43bcce006 HTTP/1.1" 200 None 2026-03-25 01:02:37,130 - lava_callback - ERROR - Error processing callback for node 69c0911bce0c9bc43bcce006: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33842 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:02:56,112 - lava_callback - INFO - Received callback for node 69c32d8d18a3717aa65d02fb INFO: 172.21.0.1:37898 - "POST /node/69c32d8d18a3717aa65d02fb HTTP/1.0" 202 Accepted 2026-03-25 01:02:56,119 - lava_callback - INFO - Starting async job submit for node 69c32d8d18a3717aa65d02fb 2026-03-25 01:02:56,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:02:56,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8d18a3717aa65d02fb HTTP/1.1" 200 None 2026-03-25 01:02:56,162 - lava_callback - DEBUG - Processing node 69c32d8d18a3717aa65d02fb: ltp-sched INFO: 172.21.0.1:37910 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:02:56,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:02:56,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:02:56,708 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c32d8d18a3717aa65d02fb/log.txt.gz 2026-03-25 01:02:56,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:02:57,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:02:57,204 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c32d8d18a3717aa65d02fb/lava-logs.yaml 2026-03-25 01:02:57,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:02:57,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:02:57,382 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c32d8d18a3717aa65d02fb/lava_callback.json.gz 2026-03-25 01:02:57,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:02:57,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8d18a3717aa65d02fb HTTP/1.1" 200 None 2026-03-25 01:02:57,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:02:57,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 01:02:57,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:02:57,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d8d18a3717aa65d02fb HTTP/1.1" 200 None 2026-03-25 01:02:57,481 - lava_callback - INFO - Completed processing callback for node 69c32d8d18a3717aa65d02fb INFO: 172.21.0.1:45656 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:03:12,419 - lava_callback - INFO - Received callback for node 69c0913bce0c9bc43bcce00e INFO: 172.21.0.1:59716 - "POST /node/69c0913bce0c9bc43bcce00e HTTP/1.0" 202 Accepted 2026-03-25 01:03:12,425 - lava_callback - INFO - Starting async job submit for node 69c0913bce0c9bc43bcce00e 2026-03-25 01:03:12,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:03:12,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0913bce0c9bc43bcce00e HTTP/1.1" 200 None 2026-03-25 01:03:12,441 - lava_callback - DEBUG - Processing node 69c0913bce0c9bc43bcce00e: baseline-arm-baylibre 2026-03-25 01:03:12,441 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0913bce0c9bc43bcce00e 2026-03-25 01:03:12,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:03:12,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0913bce0c9bc43bcce00e HTTP/1.1" 200 None 2026-03-25 01:03:12,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:03:12,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:03:12,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:03:12,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0913bce0c9bc43bcce00e HTTP/1.1" 200 None 2026-03-25 01:03:12,520 - lava_callback - ERROR - Error processing callback for node 69c0913bce0c9bc43bcce00e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59742 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:03:20,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:03:20,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39792 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:03:32,372 - lava_callback - INFO - Received callback for node 69c0915fce0c9bc43bcce019 INFO: 172.21.0.1:39808 - "POST /node/69c0915fce0c9bc43bcce019 HTTP/1.0" 202 Accepted 2026-03-25 01:03:32,378 - lava_callback - INFO - Starting async job submit for node 69c0915fce0c9bc43bcce019 2026-03-25 01:03:32,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:03:32,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0915fce0c9bc43bcce019 HTTP/1.1" 200 None 2026-03-25 01:03:32,394 - lava_callback - DEBUG - Processing node 69c0915fce0c9bc43bcce019: baseline-arm-baylibre 2026-03-25 01:03:32,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0915fce0c9bc43bcce019 2026-03-25 01:03:32,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:03:32,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0915fce0c9bc43bcce019 HTTP/1.1" 200 None 2026-03-25 01:03:32,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:03:32,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:03:32,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:03:32,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0915fce0c9bc43bcce019 HTTP/1.1" 200 None 2026-03-25 01:03:32,458 - lava_callback - ERROR - Error processing callback for node 69c0915fce0c9bc43bcce019: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41970 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:04:52,504 - lava_callback - INFO - Received callback for node 69c091a6ce0c9bc43bcce030 INFO: 172.21.0.1:43866 - "POST /node/69c091a6ce0c9bc43bcce030 HTTP/1.0" 202 Accepted 2026-03-25 01:04:52,509 - lava_callback - INFO - Starting async job submit for node 69c091a6ce0c9bc43bcce030 2026-03-25 01:04:52,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:04:52,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c091a6ce0c9bc43bcce030 HTTP/1.1" 200 None 2026-03-25 01:04:52,525 - lava_callback - DEBUG - Processing node 69c091a6ce0c9bc43bcce030: baseline-arm-baylibre 2026-03-25 01:04:52,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c091a6ce0c9bc43bcce030 2026-03-25 01:04:52,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:04:52,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c091a6ce0c9bc43bcce030 HTTP/1.1" 200 None 2026-03-25 01:04:52,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:04:52,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:04:52,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:04:52,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c091a6ce0c9bc43bcce030 HTTP/1.1" 200 None 2026-03-25 01:04:52,577 - lava_callback - ERROR - Error processing callback for node 69c091a6ce0c9bc43bcce030: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:04:56,379 - lava_callback - INFO - Received callback for node 69c32d8e18a3717aa65d02fe INFO: 172.21.0.1:43874 - "POST /node/69c32d8e18a3717aa65d02fe HTTP/1.0" 202 Accepted 2026-03-25 01:04:56,385 - lava_callback - INFO - Starting async job submit for node 69c32d8e18a3717aa65d02fe 2026-03-25 01:04:56,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:04:56,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8e18a3717aa65d02fe HTTP/1.1" 200 None 2026-03-25 01:04:56,424 - lava_callback - DEBUG - Processing node 69c32d8e18a3717aa65d02fe: ltp-smoke INFO: 172.21.0.1:43890 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:04:56,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:04:56,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:04:56,940 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c32d8e18a3717aa65d02fe/log.txt.gz 2026-03-25 01:04:57,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:04:57,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:04:57,360 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c32d8e18a3717aa65d02fe/lava-logs.yaml 2026-03-25 01:04:57,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:04:57,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:04:57,527 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c32d8e18a3717aa65d02fe/lava_callback.json.gz 2026-03-25 01:04:57,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:04:57,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d8e18a3717aa65d02fe HTTP/1.1" 200 None 2026-03-25 01:04:57,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:04:57,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 01:04:57,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:04:57,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d8e18a3717aa65d02fe HTTP/1.1" 200 None 2026-03-25 01:04:57,611 - lava_callback - INFO - Completed processing callback for node 69c32d8e18a3717aa65d02fe 2026-03-25 01:05:04,877 - lava_callback - INFO - Received callback for node 69c32d9218a3717aa65d0304 INFO: 172.21.0.1:35512 - "POST /node/69c32d9218a3717aa65d0304 HTTP/1.0" 202 Accepted 2026-03-25 01:05:04,883 - lava_callback - INFO - Starting async job submit for node 69c32d9218a3717aa65d0304 2026-03-25 01:05:04,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:04,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9218a3717aa65d0304 HTTP/1.1" 200 None 2026-03-25 01:05:04,929 - lava_callback - DEBUG - Processing node 69c32d9218a3717aa65d0304: ltp-syscalls-ipc 2026-03-25 01:05:05,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:05:05,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:05:05,404 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c32d9218a3717aa65d0304/log.txt.gz 2026-03-25 01:05:05,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:05:05,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:05:05,829 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c32d9218a3717aa65d0304/lava-logs.yaml 2026-03-25 01:05:05,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:05:05,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:05:05,989 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c32d9218a3717aa65d0304/lava_callback.json.gz 2026-03-25 01:05:06,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:06,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9218a3717aa65d0304 HTTP/1.1" 200 None 2026-03-25 01:05:06,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:06,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 01:05:06,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:06,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d9218a3717aa65d0304 HTTP/1.1" 200 None 2026-03-25 01:05:06,075 - lava_callback - INFO - Completed processing callback for node 69c32d9218a3717aa65d0304 INFO: 172.21.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43058 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:05:20,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:20,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39836 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:05:32,460 - lava_callback - INFO - Received callback for node 69c091cace0c9bc43bcce041 INFO: 172.21.0.1:39846 - "POST /node/69c091cace0c9bc43bcce041 HTTP/1.0" 202 Accepted 2026-03-25 01:05:32,465 - lava_callback - INFO - Starting async job submit for node 69c091cace0c9bc43bcce041 2026-03-25 01:05:32,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:32,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c091cace0c9bc43bcce041 HTTP/1.1" 200 None 2026-03-25 01:05:32,481 - lava_callback - DEBUG - Processing node 69c091cace0c9bc43bcce041: baseline-arm-baylibre 2026-03-25 01:05:32,481 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c091cace0c9bc43bcce041 2026-03-25 01:05:32,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:32,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c091cace0c9bc43bcce041 HTTP/1.1" 200 None 2026-03-25 01:05:32,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:32,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:05:32,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:32,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c091cace0c9bc43bcce041 HTTP/1.1" 200 None 2026-03-25 01:05:32,542 - lava_callback - ERROR - Error processing callback for node 69c091cace0c9bc43bcce041: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:05:36,386 - lava_callback - INFO - Received callback for node 69c32d9318a3717aa65d030b INFO: 172.21.0.1:39854 - "POST /node/69c32d9318a3717aa65d030b HTTP/1.0" 202 Accepted 2026-03-25 01:05:36,392 - lava_callback - INFO - Starting async job submit for node 69c32d9318a3717aa65d030b 2026-03-25 01:05:36,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:36,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9318a3717aa65d030b HTTP/1.1" 200 None 2026-03-25 01:05:36,433 - lava_callback - DEBUG - Processing node 69c32d9318a3717aa65d030b: ltp-timers INFO: 172.21.0.1:39868 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:05:36,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:05:36,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:05:36,956 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c32d9318a3717aa65d030b/log.txt.gz 2026-03-25 01:05:37,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:05:37,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:05:37,372 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c32d9318a3717aa65d030b/lava-logs.yaml 2026-03-25 01:05:37,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:05:37,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:05:37,546 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c32d9318a3717aa65d030b/lava_callback.json.gz 2026-03-25 01:05:37,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:37,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9318a3717aa65d030b HTTP/1.1" 200 None 2026-03-25 01:05:37,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:37,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 01:05:37,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:37,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d9318a3717aa65d030b HTTP/1.1" 200 None 2026-03-25 01:05:37,641 - lava_callback - INFO - Completed processing callback for node 69c32d9318a3717aa65d030b INFO: 172.21.0.1:35926 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:05:50,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:50,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-25 01:05:52,470 - lava_callback - INFO - Received callback for node 69c091eece0c9bc43bcce04c INFO: 172.21.0.1:45822 - "POST /node/69c091eece0c9bc43bcce04c HTTP/1.0" 202 Accepted 2026-03-25 01:05:52,476 - lava_callback - INFO - Starting async job submit for node 69c091eece0c9bc43bcce04c 2026-03-25 01:05:52,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:52,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c091eece0c9bc43bcce04c HTTP/1.1" 200 None 2026-03-25 01:05:52,495 - lava_callback - DEBUG - Processing node 69c091eece0c9bc43bcce04c: baseline-arm-baylibre 2026-03-25 01:05:52,495 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c091eece0c9bc43bcce04c 2026-03-25 01:05:52,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:52,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c091eece0c9bc43bcce04c HTTP/1.1" 200 None 2026-03-25 01:05:52,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:52,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:05:52,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:05:52,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c091eece0c9bc43bcce04c HTTP/1.1" 200 None 2026-03-25 01:05:52,558 - lava_callback - ERROR - Error processing callback for node 69c091eece0c9bc43bcce04c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43604 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:06:37,372 - lava_callback - INFO - Received callback for node 69c09213ce0c9bc43bcce057 INFO: 172.21.0.1:43618 - "POST /node/69c09213ce0c9bc43bcce057 HTTP/1.0" 202 Accepted 2026-03-25 01:06:37,377 - lava_callback - INFO - Starting async job submit for node 69c09213ce0c9bc43bcce057 2026-03-25 01:06:37,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:06:37,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09213ce0c9bc43bcce057 HTTP/1.1" 200 None 2026-03-25 01:06:37,394 - lava_callback - DEBUG - Processing node 69c09213ce0c9bc43bcce057: baseline-arm-clabbe 2026-03-25 01:06:37,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09213ce0c9bc43bcce057 2026-03-25 01:06:37,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:06:37,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09213ce0c9bc43bcce057 HTTP/1.1" 200 None 2026-03-25 01:06:37,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:06:37,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:06:37,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:06:37,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09213ce0c9bc43bcce057 HTTP/1.1" 200 None 2026-03-25 01:06:37,449 - lava_callback - ERROR - Error processing callback for node 69c09213ce0c9bc43bcce057: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57518 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 01:07:16,237 - lava_callback - INFO - Received callback for node 69c32d9518a3717aa65d030e INFO: 172.21.0.1:57524 - "POST /node/69c32d9518a3717aa65d030e HTTP/1.0" 202 Accepted 2026-03-25 01:07:16,244 - lava_callback - INFO - Starting async job submit for node 69c32d9518a3717aa65d030e 2026-03-25 01:07:16,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:07:16,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9518a3717aa65d030e HTTP/1.1" 200 None 2026-03-25 01:07:16,310 - lava_callback - DEBUG - Processing node 69c32d9518a3717aa65d030e: ltp-watchqueue INFO: 172.21.0.1:57532 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:07:16,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:07:16,851 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:07:16,853 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c32d9518a3717aa65d030e/log.txt.gz 2026-03-25 01:07:17,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:07:17,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:07:17,305 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c32d9518a3717aa65d030e/lava-logs.yaml 2026-03-25 01:07:17,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:07:17,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:07:17,513 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c32d9518a3717aa65d030e/lava_callback.json.gz 2026-03-25 01:07:17,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:07:17,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c32d9518a3717aa65d030e HTTP/1.1" 200 None 2026-03-25 01:07:17,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:07:17,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 01:07:17,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:07:17,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c32d9518a3717aa65d030e HTTP/1.1" 200 None 2026-03-25 01:07:17,630 - lava_callback - INFO - Completed processing callback for node 69c32d9518a3717aa65d030e 2026-03-25 01:07:20,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:07:20,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49872 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:07:57,328 - lava_callback - INFO - Received callback for node 69c0925ace0c9bc43bcce06d INFO: 172.21.0.1:49882 - "POST /node/69c0925ace0c9bc43bcce06d HTTP/1.0" 202 Accepted 2026-03-25 01:07:57,338 - lava_callback - INFO - Starting async job submit for node 69c0925ace0c9bc43bcce06d 2026-03-25 01:07:57,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:07:57,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0925ace0c9bc43bcce06d HTTP/1.1" 200 None 2026-03-25 01:07:57,356 - lava_callback - DEBUG - Processing node 69c0925ace0c9bc43bcce06d: baseline-arm-clabbe 2026-03-25 01:07:57,357 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0925ace0c9bc43bcce06d 2026-03-25 01:07:57,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:07:57,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0925ace0c9bc43bcce06d HTTP/1.1" 200 None 2026-03-25 01:07:57,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:07:57,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:07:57,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:07:57,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0925ace0c9bc43bcce06d HTTP/1.1" 200 None 2026-03-25 01:07:57,459 - lava_callback - ERROR - Error processing callback for node 69c0925ace0c9bc43bcce06d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50600 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:08:32,686 - lava_callback - INFO - Received callback for node 69c0927cce0c9bc43bcce07b INFO: 172.21.0.1:50616 - "POST /node/69c0927cce0c9bc43bcce07b HTTP/1.0" 202 Accepted 2026-03-25 01:08:32,692 - lava_callback - INFO - Starting async job submit for node 69c0927cce0c9bc43bcce07b 2026-03-25 01:08:32,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:08:32,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0927cce0c9bc43bcce07b HTTP/1.1" 200 None 2026-03-25 01:08:32,731 - lava_callback - DEBUG - Processing node 69c0927cce0c9bc43bcce07b: baseline-arm-baylibre 2026-03-25 01:08:32,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0927cce0c9bc43bcce07b 2026-03-25 01:08:32,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:08:32,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0927cce0c9bc43bcce07b HTTP/1.1" 200 None 2026-03-25 01:08:32,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:08:32,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:08:32,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:08:32,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0927cce0c9bc43bcce07b HTTP/1.1" 200 None 2026-03-25 01:08:32,800 - lava_callback - ERROR - Error processing callback for node 69c0927cce0c9bc43bcce07b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45882 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:08:57,497 - lava_callback - INFO - Received callback for node 69c092a1ce0c9bc43bcce085 INFO: 172.21.0.1:45884 - "POST /node/69c092a1ce0c9bc43bcce085 HTTP/1.0" 202 Accepted 2026-03-25 01:08:57,505 - lava_callback - INFO - Starting async job submit for node 69c092a1ce0c9bc43bcce085 2026-03-25 01:08:57,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:08:57,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c092a1ce0c9bc43bcce085 HTTP/1.1" 200 None 2026-03-25 01:08:57,535 - lava_callback - DEBUG - Processing node 69c092a1ce0c9bc43bcce085: baseline-arm-clabbe 2026-03-25 01:08:57,536 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c092a1ce0c9bc43bcce085 2026-03-25 01:08:57,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:08:57,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c092a1ce0c9bc43bcce085 HTTP/1.1" 200 None 2026-03-25 01:08:57,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:08:57,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:08:57,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:08:57,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c092a1ce0c9bc43bcce085 HTTP/1.1" 200 None 2026-03-25 01:08:57,631 - lava_callback - ERROR - Error processing callback for node 69c092a1ce0c9bc43bcce085: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52868 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:09:37,463 - lava_callback - INFO - Received callback for node 69c092c5ce0c9bc43bcce08e INFO: 172.21.0.1:52874 - "POST /node/69c092c5ce0c9bc43bcce08e HTTP/1.0" 202 Accepted 2026-03-25 01:09:37,472 - lava_callback - INFO - Starting async job submit for node 69c092c5ce0c9bc43bcce08e 2026-03-25 01:09:37,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:09:37,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c092c5ce0c9bc43bcce08e HTTP/1.1" 200 None 2026-03-25 01:09:37,503 - lava_callback - DEBUG - Processing node 69c092c5ce0c9bc43bcce08e: baseline-arm-clabbe 2026-03-25 01:09:37,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c092c5ce0c9bc43bcce08e 2026-03-25 01:09:37,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:09:37,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c092c5ce0c9bc43bcce08e HTTP/1.1" 200 None 2026-03-25 01:09:37,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:09:37,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:09:37,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:09:37,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c092c5ce0c9bc43bcce08e HTTP/1.1" 200 None 2026-03-25 01:09:37,626 - lava_callback - ERROR - Error processing callback for node 69c092c5ce0c9bc43bcce08e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55852 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:10:17,381 - lava_callback - INFO - Received callback for node 69c092eace0c9bc43bcce09a INFO: 172.21.0.1:55862 - "POST /node/69c092eace0c9bc43bcce09a HTTP/1.0" 202 Accepted 2026-03-25 01:10:17,387 - lava_callback - INFO - Starting async job submit for node 69c092eace0c9bc43bcce09a 2026-03-25 01:10:17,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:10:17,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c092eace0c9bc43bcce09a HTTP/1.1" 200 None 2026-03-25 01:10:17,419 - lava_callback - DEBUG - Processing node 69c092eace0c9bc43bcce09a: baseline-arm-clabbe 2026-03-25 01:10:17,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c092eace0c9bc43bcce09a 2026-03-25 01:10:17,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:10:17,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c092eace0c9bc43bcce09a HTTP/1.1" 200 None 2026-03-25 01:10:17,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:10:17,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89da7fbe29c53ae85cce2 HTTP/1.1" 200 None 2026-03-25 01:10:17,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:10:17,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c092eace0c9bc43bcce09a HTTP/1.1" 200 None 2026-03-25 01:10:17,475 - lava_callback - ERROR - Error processing callback for node 69c092eace0c9bc43bcce09a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48226 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:10:53,107 - lava_callback - INFO - Received callback for node 69c09314ce0c9bc43bcce0a6 INFO: 172.21.0.1:54060 - "POST /node/69c09314ce0c9bc43bcce0a6 HTTP/1.0" 202 Accepted 2026-03-25 01:10:53,113 - lava_callback - INFO - Starting async job submit for node 69c09314ce0c9bc43bcce0a6 2026-03-25 01:10:53,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:10:53,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09314ce0c9bc43bcce0a6 HTTP/1.1" 200 None 2026-03-25 01:10:53,129 - lava_callback - DEBUG - Processing node 69c09314ce0c9bc43bcce0a6: baseline-arm64-baylibre 2026-03-25 01:10:53,129 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09314ce0c9bc43bcce0a6 2026-03-25 01:10:53,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:10:53,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09314ce0c9bc43bcce0a6 HTTP/1.1" 200 None 2026-03-25 01:10:53,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:10:53,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:10:53,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:10:53,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09314ce0c9bc43bcce0a6 HTTP/1.1" 200 None 2026-03-25 01:10:53,195 - lava_callback - ERROR - Error processing callback for node 69c09314ce0c9bc43bcce0a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34738 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:12:12,852 - lava_callback - INFO - Received callback for node 69c0935fce0c9bc43bcce0b6 INFO: 172.21.0.1:37864 - "POST /node/69c0935fce0c9bc43bcce0b6 HTTP/1.0" 202 Accepted 2026-03-25 01:12:12,858 - lava_callback - INFO - Starting async job submit for node 69c0935fce0c9bc43bcce0b6 2026-03-25 01:12:12,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:12:12,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0935fce0c9bc43bcce0b6 HTTP/1.1" 200 None 2026-03-25 01:12:12,876 - lava_callback - DEBUG - Processing node 69c0935fce0c9bc43bcce0b6: baseline-arm64-baylibre 2026-03-25 01:12:12,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0935fce0c9bc43bcce0b6 2026-03-25 01:12:12,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:12:12,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0935fce0c9bc43bcce0b6 HTTP/1.1" 200 None 2026-03-25 01:12:12,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:12:12,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:12:12,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:12:12,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0935fce0c9bc43bcce0b6 HTTP/1.1" 200 None 2026-03-25 01:12:12,949 - lava_callback - ERROR - Error processing callback for node 69c0935fce0c9bc43bcce0b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55376 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:13:32,967 - lava_callback - INFO - Received callback for node 69c093aace0c9bc43bcce0cf INFO: 172.21.0.1:55390 - "POST /node/69c093aace0c9bc43bcce0cf HTTP/1.0" 202 Accepted 2026-03-25 01:13:32,973 - lava_callback - INFO - Starting async job submit for node 69c093aace0c9bc43bcce0cf 2026-03-25 01:13:32,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:13:33,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c093aace0c9bc43bcce0cf HTTP/1.1" 200 None 2026-03-25 01:13:33,010 - lava_callback - DEBUG - Processing node 69c093aace0c9bc43bcce0cf: baseline-arm64-baylibre 2026-03-25 01:13:33,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c093aace0c9bc43bcce0cf 2026-03-25 01:13:33,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:13:33,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c093aace0c9bc43bcce0cf HTTP/1.1" 200 None 2026-03-25 01:13:33,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:13:33,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:13:33,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:13:33,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c093aace0c9bc43bcce0cf HTTP/1.1" 200 None 2026-03-25 01:13:33,079 - lava_callback - ERROR - Error processing callback for node 69c093aace0c9bc43bcce0cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51940 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:14:32,998 - lava_callback - INFO - Received callback for node 69c093f6ce0c9bc43bcce0d6 INFO: 172.21.0.1:51952 - "POST /node/69c093f6ce0c9bc43bcce0d6 HTTP/1.0" 202 Accepted 2026-03-25 01:14:33,005 - lava_callback - INFO - Starting async job submit for node 69c093f6ce0c9bc43bcce0d6 2026-03-25 01:14:33,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:14:33,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c093f6ce0c9bc43bcce0d6 HTTP/1.1" 200 None 2026-03-25 01:14:33,025 - lava_callback - DEBUG - Processing node 69c093f6ce0c9bc43bcce0d6: baseline-arm64-baylibre 2026-03-25 01:14:33,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c093f6ce0c9bc43bcce0d6 2026-03-25 01:14:33,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:14:33,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c093f6ce0c9bc43bcce0d6 HTTP/1.1" 200 None 2026-03-25 01:14:33,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:14:33,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:14:33,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:14:33,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c093f6ce0c9bc43bcce0d6 HTTP/1.1" 200 None 2026-03-25 01:14:33,109 - lava_callback - ERROR - Error processing callback for node 69c093f6ce0c9bc43bcce0d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51954 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:14:41,100 - lava_callback - INFO - Received callback for node 69c3330f18a3717aa65d2b69 INFO: 172.21.0.1:49986 - "POST /node/69c3330f18a3717aa65d2b69 HTTP/1.0" 202 Accepted 2026-03-25 01:14:41,111 - lava_callback - INFO - Starting async job submit for node 69c3330f18a3717aa65d2b69 2026-03-25 01:14:41,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:14:41,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3330f18a3717aa65d2b69 HTTP/1.1" 200 None 2026-03-25 01:14:41,228 - lava_callback - DEBUG - Processing node 69c3330f18a3717aa65d2b69: baseline-arm 2026-03-25 01:14:42,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:14:43,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:14:43,131 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c3330f18a3717aa65d2b69/log.txt.gz 2026-03-25 01:14:43,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:14:44,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:14:44,995 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c3330f18a3717aa65d2b69/lava-logs.yaml 2026-03-25 01:14:45,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:14:46,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:14:46,068 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c3330f18a3717aa65d2b69/lava_callback.json.gz DEBUG: Setup failed for 69c3330f18a3717aa65d2b69. Transitting job node result: incomplete -> fail 2026-03-25 01:14:46,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:14:46,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3330f18a3717aa65d2b69 HTTP/1.1" 200 None 2026-03-25 01:14:46,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:14:46,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 01:14:46,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:14:46,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3330f18a3717aa65d2b69 HTTP/1.1" 200 None 2026-03-25 01:14:46,443 - lava_callback - INFO - Completed processing callback for node 69c3330f18a3717aa65d2b69 INFO: 172.21.0.1:49988 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:14:50,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:14:50,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40522 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:15:53,099 - lava_callback - INFO - Received callback for node 69c09443ce0c9bc43bcce0e8 INFO: 172.21.0.1:56578 - "POST /node/69c09443ce0c9bc43bcce0e8 HTTP/1.0" 202 Accepted 2026-03-25 01:15:53,107 - lava_callback - INFO - Starting async job submit for node 69c09443ce0c9bc43bcce0e8 2026-03-25 01:15:53,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:15:53,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09443ce0c9bc43bcce0e8 HTTP/1.1" 200 None 2026-03-25 01:15:53,125 - lava_callback - DEBUG - Processing node 69c09443ce0c9bc43bcce0e8: baseline-arm64-baylibre 2026-03-25 01:15:53,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09443ce0c9bc43bcce0e8 2026-03-25 01:15:53,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:15:53,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09443ce0c9bc43bcce0e8 HTTP/1.1" 200 None 2026-03-25 01:15:53,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:15:53,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:15:53,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:15:53,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09443ce0c9bc43bcce0e8 HTTP/1.1" 200 None 2026-03-25 01:15:53,223 - lava_callback - ERROR - Error processing callback for node 69c09443ce0c9bc43bcce0e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46804 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:16:26,283 - lava_callback - INFO - Received callback for node 69c3335818a3717aa65d2b7f INFO: 172.21.0.1:41912 - "POST /node/69c3335818a3717aa65d2b7f HTTP/1.0" 202 Accepted 2026-03-25 01:16:26,291 - lava_callback - INFO - Starting async job submit for node 69c3335818a3717aa65d2b7f 2026-03-25 01:16:26,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:16:26,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3335818a3717aa65d2b7f HTTP/1.1" 200 None 2026-03-25 01:16:26,430 - lava_callback - DEBUG - Processing node 69c3335818a3717aa65d2b7f: ltp-smoke INFO: 172.21.0.1:41916 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:16:27,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:16:27,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:16:27,951 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c3335818a3717aa65d2b7f/log.txt.gz 2026-03-25 01:16:28,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:16:29,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:16:29,253 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c3335818a3717aa65d2b7f/lava-logs.yaml 2026-03-25 01:16:29,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:16:30,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:16:30,132 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c3335818a3717aa65d2b7f/lava_callback.json.gz 2026-03-25 01:16:30,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:16:30,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3335818a3717aa65d2b7f HTTP/1.1" 200 None 2026-03-25 01:16:30,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:16:30,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3294718a3717aa65d01b3 HTTP/1.1" 200 None 2026-03-25 01:16:30,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:16:30,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3335818a3717aa65d2b7f HTTP/1.1" 200 None 2026-03-25 01:16:30,615 - lava_callback - INFO - Completed processing callback for node 69c3335818a3717aa65d2b7f INFO: 172.21.0.1:35126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50062 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:16:50,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:16:50,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42176 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:17:13,182 - lava_callback - INFO - Received callback for node 69c0948ece0c9bc43bcce0f4 INFO: 172.21.0.1:57700 - "POST /node/69c0948ece0c9bc43bcce0f4 HTTP/1.0" 202 Accepted 2026-03-25 01:17:13,188 - lava_callback - INFO - Starting async job submit for node 69c0948ece0c9bc43bcce0f4 2026-03-25 01:17:13,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:17:13,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0948ece0c9bc43bcce0f4 HTTP/1.1" 200 None 2026-03-25 01:17:13,204 - lava_callback - DEBUG - Processing node 69c0948ece0c9bc43bcce0f4: baseline-arm64-baylibre 2026-03-25 01:17:13,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0948ece0c9bc43bcce0f4 2026-03-25 01:17:13,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:17:13,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0948ece0c9bc43bcce0f4 HTTP/1.1" 200 None 2026-03-25 01:17:13,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:17:13,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:17:13,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:17:13,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0948ece0c9bc43bcce0f4 HTTP/1.1" 200 None 2026-03-25 01:17:13,260 - lava_callback - ERROR - Error processing callback for node 69c0948ece0c9bc43bcce0f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41082 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:17:47,365 - lava_callback - INFO - Received callback for node 69c337af18a3717aa65d2d09 INFO: 172.21.0.1:41088 - "POST /node/69c337af18a3717aa65d2d09 HTTP/1.0" 202 Accepted 2026-03-25 01:17:47,371 - lava_callback - INFO - Starting async job submit for node 69c337af18a3717aa65d2d09 2026-03-25 01:17:47,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:17:47,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337af18a3717aa65d2d09 HTTP/1.1" 200 None 2026-03-25 01:17:47,388 - lava_callback - DEBUG - Processing node 69c337af18a3717aa65d2d09: baseline-nfs-arm64 2026-03-25 01:17:47,389 - lava_callback - WARNING - Failed to upload log 2026-03-25 01:17:47,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:17:47,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:17:47,514 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337af18a3717aa65d2d09/lava-logs.yaml 2026-03-25 01:17:47,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:17:47,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:17:47,689 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337af18a3717aa65d2d09/lava_callback.json.gz 2026-03-25 01:17:47,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:17:47,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337af18a3717aa65d2d09 HTTP/1.1" 200 None 2026-03-25 01:17:47,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:17:47,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:17:47,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:17:47,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337af18a3717aa65d2d09 HTTP/1.1" 200 None 2026-03-25 01:17:47,755 - lava_callback - INFO - Completed processing callback for node 69c337af18a3717aa65d2d09 2026-03-25 01:17:50,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:17:50,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46762 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:17:59,091 - lava_callback - INFO - Received callback for node 69c3379218a3717aa65d2ca8 INFO: 172.21.0.1:46778 - "POST /node/69c3379218a3717aa65d2ca8 HTTP/1.0" 202 Accepted 2026-03-25 01:17:59,097 - lava_callback - INFO - Starting async job submit for node 69c3379218a3717aa65d2ca8 2026-03-25 01:17:59,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:17:59,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379218a3717aa65d2ca8 HTTP/1.1" 200 None 2026-03-25 01:17:59,137 - lava_callback - DEBUG - Processing node 69c3379218a3717aa65d2ca8: baseline-arm64 2026-03-25 01:17:59,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:17:59,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:17:59,485 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379218a3717aa65d2ca8/log.txt.gz 2026-03-25 01:17:59,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:17:59,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:17:59,856 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379218a3717aa65d2ca8/lava-logs.yaml 2026-03-25 01:17:59,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:00,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:00,022 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379218a3717aa65d2ca8/lava_callback.json.gz 2026-03-25 01:18:00,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:00,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379218a3717aa65d2ca8 HTTP/1.1" 200 None 2026-03-25 01:18:00,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:00,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:18:00,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:00,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379218a3717aa65d2ca8 HTTP/1.1" 200 None 2026-03-25 01:18:00,120 - lava_callback - INFO - Completed processing callback for node 69c3379218a3717aa65d2ca8 2026-03-25 01:18:06,135 - lava_callback - INFO - Received callback for node 69c3379e18a3717aa65d2cdf INFO: 172.21.0.1:42292 - "POST /node/69c3379e18a3717aa65d2cdf HTTP/1.0" 202 Accepted 2026-03-25 01:18:06,142 - lava_callback - INFO - Starting async job submit for node 69c3379e18a3717aa65d2cdf 2026-03-25 01:18:06,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:06,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379e18a3717aa65d2cdf HTTP/1.1" 200 None 2026-03-25 01:18:06,183 - lava_callback - DEBUG - Processing node 69c3379e18a3717aa65d2cdf: baseline-arm64 2026-03-25 01:18:06,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:06,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:06,466 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379e18a3717aa65d2cdf/log.txt.gz INFO: 172.21.0.1:42306 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:18:06,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:06,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:06,807 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379e18a3717aa65d2cdf/lava-logs.yaml 2026-03-25 01:18:06,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:06,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:06,983 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379e18a3717aa65d2cdf/lava_callback.json.gz 2026-03-25 01:18:07,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:07,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379e18a3717aa65d2cdf HTTP/1.1" 200 None 2026-03-25 01:18:07,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:07,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:18:07,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:07,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379e18a3717aa65d2cdf HTTP/1.1" 200 None 2026-03-25 01:18:07,076 - lava_callback - INFO - Completed processing callback for node 69c3379e18a3717aa65d2cdf INFO: 172.21.0.1:45280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45284 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:18:17,697 - lava_callback - INFO - Received callback for node 69c3379618a3717aa65d2cb4 INFO: 172.21.0.1:45294 - "POST /node/69c3379618a3717aa65d2cb4 HTTP/1.0" 202 Accepted 2026-03-25 01:18:17,704 - lava_callback - INFO - Starting async job submit for node 69c3379618a3717aa65d2cb4 2026-03-25 01:18:17,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:17,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379618a3717aa65d2cb4 HTTP/1.1" 200 None 2026-03-25 01:18:17,749 - lava_callback - DEBUG - Processing node 69c3379618a3717aa65d2cb4: baseline-arm64 2026-03-25 01:18:17,917 - lava_callback - WARNING - Failed to upload log 2026-03-25 01:18:17,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:18,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:18,146 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379618a3717aa65d2cb4/lava-logs.yaml 2026-03-25 01:18:18,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:18,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:18,312 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379618a3717aa65d2cb4/lava_callback.json.gz 2026-03-25 01:18:18,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:18,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379618a3717aa65d2cb4 HTTP/1.1" 200 None 2026-03-25 01:18:18,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:18,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:18:18,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:18,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379618a3717aa65d2cb4 HTTP/1.1" 200 None 2026-03-25 01:18:18,471 - lava_callback - INFO - Completed processing callback for node 69c3379618a3717aa65d2cb4 2026-03-25 01:18:18,868 - lava_callback - INFO - Received callback for node 69c3379318a3717aa65d2cae INFO: 172.21.0.1:45302 - "POST /node/69c3379318a3717aa65d2cae HTTP/1.0" 202 Accepted 2026-03-25 01:18:18,879 - lava_callback - INFO - Starting async job submit for node 69c3379318a3717aa65d2cae 2026-03-25 01:18:18,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:18,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379318a3717aa65d2cae HTTP/1.1" 200 None 2026-03-25 01:18:18,945 - lava_callback - DEBUG - Processing node 69c3379318a3717aa65d2cae: baseline-arm64 2026-03-25 01:18:19,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:19,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:19,410 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379318a3717aa65d2cae/log.txt.gz 2026-03-25 01:18:19,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:19,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:19,865 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379318a3717aa65d2cae/lava-logs.yaml 2026-03-25 01:18:19,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:20,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:20,003 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379318a3717aa65d2cae/lava_callback.json.gz 2026-03-25 01:18:20,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:20,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379318a3717aa65d2cae HTTP/1.1" 200 None 2026-03-25 01:18:20,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:20,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:18:20,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:20,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379318a3717aa65d2cae HTTP/1.1" 200 None 2026-03-25 01:18:20,093 - lava_callback - INFO - Completed processing callback for node 69c3379318a3717aa65d2cae 2026-03-25 01:18:20,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:20,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:56354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48166 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:18:33,018 - lava_callback - INFO - Received callback for node 69c337a218a3717aa65d2ceb INFO: 172.21.0.1:48178 - "POST /node/69c337a218a3717aa65d2ceb HTTP/1.0" 202 Accepted 2026-03-25 01:18:33,024 - lava_callback - INFO - Starting async job submit for node 69c337a218a3717aa65d2ceb 2026-03-25 01:18:33,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:33,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a218a3717aa65d2ceb HTTP/1.1" 200 None 2026-03-25 01:18:33,069 - lava_callback - DEBUG - Processing node 69c337a218a3717aa65d2ceb: baseline-arm64 2026-03-25 01:18:33,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:33,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:33,470 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a218a3717aa65d2ceb/log.txt.gz 2026-03-25 01:18:33,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:33,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:33,841 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a218a3717aa65d2ceb/lava-logs.yaml 2026-03-25 01:18:33,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:33,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:33,994 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a218a3717aa65d2ceb/lava_callback.json.gz 2026-03-25 01:18:34,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:34,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a218a3717aa65d2ceb HTTP/1.1" 200 None 2026-03-25 01:18:34,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:34,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:18:34,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:34,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337a218a3717aa65d2ceb HTTP/1.1" 200 None 2026-03-25 01:18:34,074 - lava_callback - INFO - Completed processing callback for node 69c337a218a3717aa65d2ceb 2026-03-25 01:18:36,394 - lava_callback - INFO - Received callback for node 69c3379918a3717aa65d2cbf INFO: 172.21.0.1:48188 - "POST /node/69c3379918a3717aa65d2cbf HTTP/1.0" 202 Accepted 2026-03-25 01:18:36,401 - lava_callback - INFO - Starting async job submit for node 69c3379918a3717aa65d2cbf 2026-03-25 01:18:36,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:36,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379918a3717aa65d2cbf HTTP/1.1" 200 None 2026-03-25 01:18:36,438 - lava_callback - DEBUG - Processing node 69c3379918a3717aa65d2cbf: baseline-arm64 INFO: 172.21.0.1:48200 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:18:36,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:36,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:36,807 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379918a3717aa65d2cbf/log.txt.gz 2026-03-25 01:18:36,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:37,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:37,255 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379918a3717aa65d2cbf/lava-logs.yaml 2026-03-25 01:18:37,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:37,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:37,398 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379918a3717aa65d2cbf/lava_callback.json.gz 2026-03-25 01:18:37,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:37,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379918a3717aa65d2cbf HTTP/1.1" 200 None 2026-03-25 01:18:37,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:37,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:18:37,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:37,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379918a3717aa65d2cbf HTTP/1.1" 200 None 2026-03-25 01:18:37,477 - lava_callback - INFO - Completed processing callback for node 69c3379918a3717aa65d2cbf 2026-03-25 01:18:37,585 - lava_callback - INFO - Received callback for node 69c094e1ce0c9bc43bcce106 INFO: 172.21.0.1:48212 - "POST /node/69c094e1ce0c9bc43bcce106 HTTP/1.0" 202 Accepted 2026-03-25 01:18:37,595 - lava_callback - INFO - Starting async job submit for node 69c094e1ce0c9bc43bcce106 2026-03-25 01:18:37,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:37,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c094e1ce0c9bc43bcce106 HTTP/1.1" 200 None 2026-03-25 01:18:37,619 - lava_callback - DEBUG - Processing node 69c094e1ce0c9bc43bcce106: baseline-arm64-clabbe 2026-03-25 01:18:37,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c094e1ce0c9bc43bcce106 2026-03-25 01:18:37,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:37,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c094e1ce0c9bc43bcce106 HTTP/1.1" 200 None 2026-03-25 01:18:37,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:37,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:18:37,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:37,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c094e1ce0c9bc43bcce106 HTTP/1.1" 200 None 2026-03-25 01:18:37,698 - lava_callback - ERROR - Error processing callback for node 69c094e1ce0c9bc43bcce106: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:18:43,287 - lava_callback - INFO - Received callback for node 69c337a318a3717aa65d2cee INFO: 172.21.0.1:57064 - "POST /node/69c337a318a3717aa65d2cee HTTP/1.0" 202 Accepted 2026-03-25 01:18:43,295 - lava_callback - INFO - Starting async job submit for node 69c337a318a3717aa65d2cee 2026-03-25 01:18:43,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:43,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a318a3717aa65d2cee HTTP/1.1" 200 None 2026-03-25 01:18:43,333 - lava_callback - DEBUG - Processing node 69c337a318a3717aa65d2cee: baseline-arm64 2026-03-25 01:18:43,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:43,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:43,731 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a318a3717aa65d2cee/log.txt.gz 2026-03-25 01:18:43,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:44,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:44,086 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a318a3717aa65d2cee/lava-logs.yaml 2026-03-25 01:18:44,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:44,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:44,250 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a318a3717aa65d2cee/lava_callback.json.gz 2026-03-25 01:18:44,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:44,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a318a3717aa65d2cee HTTP/1.1" 200 None 2026-03-25 01:18:44,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:44,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:18:44,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:44,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337a318a3717aa65d2cee HTTP/1.1" 200 None 2026-03-25 01:18:44,339 - lava_callback - INFO - Completed processing callback for node 69c337a318a3717aa65d2cee 2026-03-25 01:18:44,776 - lava_callback - INFO - Received callback for node 69c3379318a3717aa65d2cab INFO: 172.21.0.1:57074 - "POST /node/69c3379318a3717aa65d2cab HTTP/1.0" 202 Accepted 2026-03-25 01:18:44,783 - lava_callback - INFO - Starting async job submit for node 69c3379318a3717aa65d2cab 2026-03-25 01:18:44,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:44,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379318a3717aa65d2cab HTTP/1.1" 200 None 2026-03-25 01:18:44,822 - lava_callback - DEBUG - Processing node 69c3379318a3717aa65d2cab: baseline-arm64 2026-03-25 01:18:45,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:45,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:45,281 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379318a3717aa65d2cab/log.txt.gz 2026-03-25 01:18:45,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:45,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:45,610 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379318a3717aa65d2cab/lava-logs.yaml 2026-03-25 01:18:45,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:18:45,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:18:45,784 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379318a3717aa65d2cab/lava_callback.json.gz 2026-03-25 01:18:45,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:45,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379318a3717aa65d2cab HTTP/1.1" 200 None 2026-03-25 01:18:45,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:45,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:18:45,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:45,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379318a3717aa65d2cab HTTP/1.1" 200 None 2026-03-25 01:18:45,878 - lava_callback - INFO - Completed processing callback for node 69c3379318a3717aa65d2cab INFO: 172.21.0.1:57080 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:18:50,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:18:50,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:57034 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:19:03,955 - lava_callback - INFO - Received callback for node 69c337aa18a3717aa65d2cfd INFO: 172.21.0.1:34178 - "POST /node/69c337aa18a3717aa65d2cfd HTTP/1.0" 202 Accepted 2026-03-25 01:19:03,965 - lava_callback - INFO - Starting async job submit for node 69c337aa18a3717aa65d2cfd 2026-03-25 01:19:04,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:04,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337aa18a3717aa65d2cfd HTTP/1.1" 200 None 2026-03-25 01:19:04,062 - lava_callback - DEBUG - Processing node 69c337aa18a3717aa65d2cfd: baseline-nfs-arm64 2026-03-25 01:19:04,480 - lava_callback - INFO - Received callback for node 69c337ac18a3717aa65d2d03 INFO: 172.21.0.1:34184 - "POST /node/69c337ac18a3717aa65d2d03 HTTP/1.0" 202 Accepted 2026-03-25 01:19:04,502 - lava_callback - INFO - Starting async job submit for node 69c337ac18a3717aa65d2d03 2026-03-25 01:19:04,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:04,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:04,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337ac18a3717aa65d2d03 HTTP/1.1" 200 None 2026-03-25 01:19:04,581 - lava_callback - DEBUG - Processing node 69c337ac18a3717aa65d2d03: baseline-nfs-arm64 2026-03-25 01:19:04,874 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:04,922 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337aa18a3717aa65d2cfd/log.txt.gz 2026-03-25 01:19:05,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:05,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:05,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:05,969 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337ac18a3717aa65d2d03/log.txt.gz 2026-03-25 01:19:06,012 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:06,015 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337aa18a3717aa65d2cfd/lava-logs.yaml 2026-03-25 01:19:06,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:06,259 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:06,271 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337aa18a3717aa65d2cfd/lava_callback.json.gz 2026-03-25 01:19:06,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:06,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337aa18a3717aa65d2cfd HTTP/1.1" 200 None 2026-03-25 01:19:06,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:06,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:06,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:06,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:06,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337aa18a3717aa65d2cfd HTTP/1.1" 200 None 2026-03-25 01:19:06,510 - lava_callback - INFO - Completed processing callback for node 69c337aa18a3717aa65d2cfd INFO: 172.21.0.1:34192 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:19:06,845 - lava_callback - INFO - Received callback for node 69c3379f18a3717aa65d2ce2 INFO: 172.21.0.1:34208 - "POST /node/69c3379f18a3717aa65d2ce2 HTTP/1.0" 202 Accepted 2026-03-25 01:19:06,856 - lava_callback - INFO - Starting async job submit for node 69c3379f18a3717aa65d2ce2 2026-03-25 01:19:06,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:06,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379f18a3717aa65d2ce2 HTTP/1.1" 200 None 2026-03-25 01:19:06,926 - lava_callback - DEBUG - Processing node 69c3379f18a3717aa65d2ce2: baseline-arm64 2026-03-25 01:19:07,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:07,061 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337ac18a3717aa65d2d03/lava-logs.yaml 2026-03-25 01:19:07,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:07,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:07,484 - lava_callback - INFO - Received callback for node 69c3379c18a3717aa65d2cc5 2026-03-25 01:19:07,495 - lava_callback - INFO - Starting async job submit for node 69c3379c18a3717aa65d2cc5 INFO: 172.21.0.1:34220 - "POST /node/69c3379c18a3717aa65d2cc5 HTTP/1.0" 202 Accepted 2026-03-25 01:19:07,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:07,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379c18a3717aa65d2cc5 HTTP/1.1" 200 None 2026-03-25 01:19:07,590 - lava_callback - DEBUG - Processing node 69c3379c18a3717aa65d2cc5: baseline-arm64 2026-03-25 01:19:07,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:07,710 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337ac18a3717aa65d2d03/lava_callback.json.gz 2026-03-25 01:19:07,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:07,718 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379f18a3717aa65d2ce2/log.txt.gz 2026-03-25 01:19:07,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:08,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337ac18a3717aa65d2d03 HTTP/1.1" 200 None 2026-03-25 01:19:08,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:08,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:08,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:08,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:08,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337ac18a3717aa65d2d03 HTTP/1.1" 200 None 2026-03-25 01:19:08,354 - lava_callback - INFO - Completed processing callback for node 69c337ac18a3717aa65d2d03 2026-03-25 01:19:08,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:08,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:08,822 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379f18a3717aa65d2ce2/lava-logs.yaml 2026-03-25 01:19:08,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:08,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:08,844 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379c18a3717aa65d2cc5/log.txt.gz 2026-03-25 01:19:09,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:09,086 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379f18a3717aa65d2ce2/lava_callback.json.gz 2026-03-25 01:19:09,147 - lava_callback - INFO - Received callback for node 69c337a518a3717aa65d2cf1 2026-03-25 01:19:09,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:34232 - "POST /node/69c337a518a3717aa65d2cf1 HTTP/1.0" 202 Accepted 2026-03-25 01:19:09,174 - lava_callback - INFO - Starting async job submit for node 69c337a518a3717aa65d2cf1 2026-03-25 01:19:09,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:09,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379f18a3717aa65d2ce2 HTTP/1.1" 200 None 2026-03-25 01:19:09,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:09,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:09,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a518a3717aa65d2cf1 HTTP/1.1" 200 None 2026-03-25 01:19:09,294 - lava_callback - DEBUG - Processing node 69c337a518a3717aa65d2cf1: baseline-arm64 2026-03-25 01:19:09,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:09,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:09,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379f18a3717aa65d2ce2 HTTP/1.1" 200 None 2026-03-25 01:19:09,399 - lava_callback - INFO - Completed processing callback for node 69c3379f18a3717aa65d2ce2 2026-03-25 01:19:09,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:09,849 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379c18a3717aa65d2cc5/lava-logs.yaml 2026-03-25 01:19:09,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:09,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:10,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:10,498 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a518a3717aa65d2cf1/log.txt.gz 2026-03-25 01:19:10,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:10,563 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379c18a3717aa65d2cc5/lava_callback.json.gz 2026-03-25 01:19:10,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:10,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379c18a3717aa65d2cc5 HTTP/1.1" 200 None 2026-03-25 01:19:10,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:10,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:10,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:10,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:10,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379c18a3717aa65d2cc5 HTTP/1.1" 200 None 2026-03-25 01:19:10,860 - lava_callback - INFO - Completed processing callback for node 69c3379c18a3717aa65d2cc5 2026-03-25 01:19:11,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:11,406 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a518a3717aa65d2cf1/lava-logs.yaml 2026-03-25 01:19:11,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:11,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:11,582 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a518a3717aa65d2cf1/lava_callback.json.gz 2026-03-25 01:19:11,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:11,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a518a3717aa65d2cf1 HTTP/1.1" 200 None 2026-03-25 01:19:11,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:11,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:11,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:11,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337a518a3717aa65d2cf1 HTTP/1.1" 200 None 2026-03-25 01:19:11,729 - lava_callback - INFO - Completed processing callback for node 69c337a518a3717aa65d2cf1 2026-03-25 01:19:12,507 - lava_callback - INFO - Received callback for node 69c337b518a3717aa65d2d15 INFO: 172.21.0.1:54564 - "POST /node/69c337b518a3717aa65d2d15 HTTP/1.0" 202 Accepted 2026-03-25 01:19:12,515 - lava_callback - INFO - Starting async job submit for node 69c337b518a3717aa65d2d15 2026-03-25 01:19:12,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:12,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337b518a3717aa65d2d15 HTTP/1.1" 200 None 2026-03-25 01:19:12,567 - lava_callback - DEBUG - Processing node 69c337b518a3717aa65d2d15: baseline-nfs-arm64 2026-03-25 01:19:13,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:13,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:13,267 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b518a3717aa65d2d15/log.txt.gz 2026-03-25 01:19:13,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:14,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:14,023 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b518a3717aa65d2d15/lava-logs.yaml 2026-03-25 01:19:14,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:14,170 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:14,172 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b518a3717aa65d2d15/lava_callback.json.gz 2026-03-25 01:19:14,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:14,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337b518a3717aa65d2d15 HTTP/1.1" 200 None 2026-03-25 01:19:14,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:14,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:14,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:14,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337b518a3717aa65d2d15 HTTP/1.1" 200 None 2026-03-25 01:19:14,257 - lava_callback - INFO - Completed processing callback for node 69c337b518a3717aa65d2d15 INFO: 172.21.0.1:54576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54580 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:19:17,556 - lava_callback - INFO - Received callback for node 69c337b618a3717aa65d2d18 INFO: 172.21.0.1:54590 - "POST /node/69c337b618a3717aa65d2d18 HTTP/1.0" 202 Accepted 2026-03-25 01:19:17,563 - lava_callback - INFO - Starting async job submit for node 69c337b618a3717aa65d2d18 2026-03-25 01:19:17,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:17,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337b618a3717aa65d2d18 HTTP/1.1" 200 None 2026-03-25 01:19:17,603 - lava_callback - DEBUG - Processing node 69c337b618a3717aa65d2d18: baseline-nfs-arm64 2026-03-25 01:19:18,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:18,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:18,387 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b618a3717aa65d2d18/log.txt.gz 2026-03-25 01:19:18,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:19,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:19,110 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b618a3717aa65d2d18/lava-logs.yaml 2026-03-25 01:19:19,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:19,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:19,314 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b618a3717aa65d2d18/lava_callback.json.gz 2026-03-25 01:19:19,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:19,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337b618a3717aa65d2d18 HTTP/1.1" 200 None 2026-03-25 01:19:19,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:19,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:19,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:19,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337b618a3717aa65d2d18 HTTP/1.1" 200 None 2026-03-25 01:19:19,459 - lava_callback - INFO - Completed processing callback for node 69c337b618a3717aa65d2d18 2026-03-25 01:19:20,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:20,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-25 01:19:25,076 - lava_callback - INFO - Received callback for node 69c337dd18a3717aa65d2db9 INFO: 172.21.0.1:43472 - "POST /node/69c337dd18a3717aa65d2db9 HTTP/1.0" 202 Accepted 2026-03-25 01:19:25,086 - lava_callback - INFO - Starting async job submit for node 69c337dd18a3717aa65d2db9 2026-03-25 01:19:25,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:25,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337dd18a3717aa65d2db9 HTTP/1.1" 200 None 2026-03-25 01:19:25,173 - lava_callback - DEBUG - Processing node 69c337dd18a3717aa65d2db9: baseline-arm64-foundriesio 2026-03-25 01:19:25,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:25,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:25,844 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c337dd18a3717aa65d2db9/log.txt.gz 2026-03-25 01:19:25,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:26,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:26,432 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c337dd18a3717aa65d2db9/lava-logs.yaml 2026-03-25 01:19:26,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:43482 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:19:26,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:26,617 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c337dd18a3717aa65d2db9/lava_callback.json.gz 2026-03-25 01:19:26,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:26,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337dd18a3717aa65d2db9 HTTP/1.1" 200 None 2026-03-25 01:19:26,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:26,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:26,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:26,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337dd18a3717aa65d2db9 HTTP/1.1" 200 None 2026-03-25 01:19:26,696 - lava_callback - INFO - Completed processing callback for node 69c337dd18a3717aa65d2db9 2026-03-25 01:19:26,803 - lava_callback - INFO - Received callback for node 69c337db18a3717aa65d2da5 INFO: 172.21.0.1:43490 - "POST /node/69c337db18a3717aa65d2da5 HTTP/1.0" 202 Accepted 2026-03-25 01:19:26,813 - lava_callback - INFO - Starting async job submit for node 69c337db18a3717aa65d2da5 2026-03-25 01:19:26,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:26,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337db18a3717aa65d2da5 HTTP/1.1" 200 None 2026-03-25 01:19:26,883 - lava_callback - DEBUG - Processing node 69c337db18a3717aa65d2da5: baseline-arm64-pengutronix 2026-03-25 01:19:27,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:27,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:27,490 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c337db18a3717aa65d2da5/log.txt.gz 2026-03-25 01:19:27,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:27,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:27,987 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c337db18a3717aa65d2da5/lava-logs.yaml 2026-03-25 01:19:27,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:28,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:28,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c337db18a3717aa65d2da5/lava_callback.json.gz 2026-03-25 01:19:28,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:28,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337db18a3717aa65d2da5 HTTP/1.1" 200 None 2026-03-25 01:19:28,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:28,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:28,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:28,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337db18a3717aa65d2da5 HTTP/1.1" 200 None 2026-03-25 01:19:28,264 - lava_callback - INFO - Completed processing callback for node 69c337db18a3717aa65d2da5 2026-03-25 01:19:28,635 - lava_callback - INFO - Received callback for node 69c337cc18a3717aa65d2d64 INFO: 172.21.0.1:43496 - "POST /node/69c337cc18a3717aa65d2d64 HTTP/1.0" 202 Accepted 2026-03-25 01:19:28,645 - lava_callback - INFO - Starting async job submit for node 69c337cc18a3717aa65d2d64 2026-03-25 01:19:28,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:28,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337cc18a3717aa65d2d64 HTTP/1.1" 200 None 2026-03-25 01:19:28,714 - lava_callback - DEBUG - Processing node 69c337cc18a3717aa65d2d64: baseline-arm64-broonie 2026-03-25 01:19:29,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:29,397 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:29,399 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337cc18a3717aa65d2d64/log.txt.gz 2026-03-25 01:19:29,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:29,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:29,856 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337cc18a3717aa65d2d64/lava-logs.yaml 2026-03-25 01:19:29,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:30,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:30,054 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337cc18a3717aa65d2d64/lava_callback.json.gz 2026-03-25 01:19:30,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:30,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337cc18a3717aa65d2d64 HTTP/1.1" 200 None 2026-03-25 01:19:30,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:30,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:30,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:30,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337cc18a3717aa65d2d64 HTTP/1.1" 200 None 2026-03-25 01:19:30,190 - lava_callback - INFO - Completed processing callback for node 69c337cc18a3717aa65d2d64 INFO: 172.21.0.1:59718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59722 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:19:36,545 - lava_callback - INFO - Received callback for node 69c337dc18a3717aa65d2dac INFO: 172.21.0.1:59732 - "POST /node/69c337dc18a3717aa65d2dac HTTP/1.0" 202 Accepted 2026-03-25 01:19:36,554 - lava_callback - INFO - Starting async job submit for node 69c337dc18a3717aa65d2dac 2026-03-25 01:19:36,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:36,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337dc18a3717aa65d2dac HTTP/1.1" 200 None 2026-03-25 01:19:36,616 - lava_callback - DEBUG - Processing node 69c337dc18a3717aa65d2dac: baseline-arm64-pengutronix 2026-03-25 01:19:36,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:37,146 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:37,149 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c337dc18a3717aa65d2dac/log.txt.gz 2026-03-25 01:19:37,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:37,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:37,595 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c337dc18a3717aa65d2dac/lava-logs.yaml 2026-03-25 01:19:37,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:37,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:37,923 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c337dc18a3717aa65d2dac/lava_callback.json.gz 2026-03-25 01:19:37,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:37,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337dc18a3717aa65d2dac HTTP/1.1" 200 None 2026-03-25 01:19:37,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:38,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:38,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:38,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337dc18a3717aa65d2dac HTTP/1.1" 200 None 2026-03-25 01:19:38,046 - lava_callback - INFO - Completed processing callback for node 69c337dc18a3717aa65d2dac INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:19:49,325 - lava_callback - INFO - Received callback for node 69c3379d18a3717aa65d2cdc INFO: 172.21.0.1:48298 - "POST /node/69c3379d18a3717aa65d2cdc HTTP/1.0" 202 Accepted 2026-03-25 01:19:49,336 - lava_callback - INFO - Starting async job submit for node 69c3379d18a3717aa65d2cdc 2026-03-25 01:19:49,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:49,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379d18a3717aa65d2cdc HTTP/1.1" 200 None 2026-03-25 01:19:49,388 - lava_callback - DEBUG - Processing node 69c3379d18a3717aa65d2cdc: baseline-arm64 2026-03-25 01:19:50,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:50,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:50,310 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379d18a3717aa65d2cdc/log.txt.gz 2026-03-25 01:19:50,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:50,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:50,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-25 01:19:51,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:51,034 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379d18a3717aa65d2cdc/lava-logs.yaml 2026-03-25 01:19:51,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:51,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:51,224 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379d18a3717aa65d2cdc/lava_callback.json.gz 2026-03-25 01:19:51,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:51,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379d18a3717aa65d2cdc HTTP/1.1" 200 None 2026-03-25 01:19:51,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:51,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:51,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:51,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379d18a3717aa65d2cdc HTTP/1.1" 200 None 2026-03-25 01:19:51,455 - lava_callback - INFO - Completed processing callback for node 69c3379d18a3717aa65d2cdc 2026-03-25 01:19:53,255 - lava_callback - INFO - Received callback for node 69c3379518a3717aa65d2cb1 INFO: 172.21.0.1:49350 - "POST /node/69c3379518a3717aa65d2cb1 HTTP/1.0" 202 Accepted 2026-03-25 01:19:53,264 - lava_callback - INFO - Starting async job submit for node 69c3379518a3717aa65d2cb1 2026-03-25 01:19:53,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:53,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379518a3717aa65d2cb1 HTTP/1.1" 200 None 2026-03-25 01:19:53,309 - lava_callback - DEBUG - Processing node 69c3379518a3717aa65d2cb1: baseline-arm64 2026-03-25 01:19:53,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:53,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:53,944 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379518a3717aa65d2cb1/log.txt.gz 2026-03-25 01:19:54,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:54,356 - lava_callback - INFO - Received callback for node 69c337de18a3717aa65d2dbc INFO: 172.21.0.1:49360 - "POST /node/69c337de18a3717aa65d2dbc HTTP/1.0" 202 Accepted 2026-03-25 01:19:54,378 - lava_callback - INFO - Starting async job submit for node 69c337de18a3717aa65d2dbc 2026-03-25 01:19:54,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:54,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337de18a3717aa65d2dbc HTTP/1.1" 200 None 2026-03-25 01:19:54,472 - lava_callback - DEBUG - Processing node 69c337de18a3717aa65d2dbc: baseline-arm64-foundriesio 2026-03-25 01:19:54,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:54,578 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379518a3717aa65d2cb1/lava-logs.yaml 2026-03-25 01:19:54,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:54,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:54,881 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379518a3717aa65d2cb1/lava_callback.json.gz 2026-03-25 01:19:55,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:55,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379518a3717aa65d2cb1 HTTP/1.1" 200 None 2026-03-25 01:19:55,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:55,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:55,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:55,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:55,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379518a3717aa65d2cb1 HTTP/1.1" 200 None 2026-03-25 01:19:55,300 - lava_callback - INFO - Completed processing callback for node 69c3379518a3717aa65d2cb1 2026-03-25 01:19:55,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:55,577 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c337de18a3717aa65d2dbc/log.txt.gz 2026-03-25 01:19:55,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:56,310 - lava_callback - INFO - Received callback for node 69c337b318a3717aa65d2d12 INFO: 172.21.0.1:49370 - "POST /node/69c337b318a3717aa65d2d12 HTTP/1.0" 202 Accepted 2026-03-25 01:19:56,324 - lava_callback - INFO - Starting async job submit for node 69c337b318a3717aa65d2d12 2026-03-25 01:19:56,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:56,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337b318a3717aa65d2d12 HTTP/1.1" 200 None 2026-03-25 01:19:56,385 - lava_callback - DEBUG - Processing node 69c337b318a3717aa65d2d12: baseline-nfs-arm64 2026-03-25 01:19:56,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:56,444 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c337de18a3717aa65d2dbc/lava-logs.yaml INFO: 172.21.0.1:49382 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:19:56,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:56,688 - lava_callback - INFO - Received callback for node 69c337cd18a3717aa65d2d67 INFO: 172.21.0.1:49398 - "POST /node/69c337cd18a3717aa65d2d67 HTTP/1.0" 202 Accepted 2026-03-25 01:19:56,700 - lava_callback - INFO - Starting async job submit for node 69c337cd18a3717aa65d2d67 2026-03-25 01:19:56,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:56,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337cd18a3717aa65d2d67 HTTP/1.1" 200 None 2026-03-25 01:19:56,818 - lava_callback - DEBUG - Processing node 69c337cd18a3717aa65d2d67: baseline-arm64-broonie 2026-03-25 01:19:56,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:57,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:57,196 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c337de18a3717aa65d2dbc/lava_callback.json.gz 2026-03-25 01:19:57,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:57,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:57,326 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b318a3717aa65d2d12/log.txt.gz 2026-03-25 01:19:57,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337de18a3717aa65d2dbc HTTP/1.1" 200 None 2026-03-25 01:19:57,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:57,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:57,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:57,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:57,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337de18a3717aa65d2dbc HTTP/1.1" 200 None 2026-03-25 01:19:57,589 - lava_callback - INFO - Completed processing callback for node 69c337de18a3717aa65d2dbc 2026-03-25 01:19:57,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:57,701 - lava_callback - INFO - Received callback for node 69c0952dce0c9bc43bcce115 INFO: 172.21.0.1:49408 - "POST /node/69c0952dce0c9bc43bcce115 HTTP/1.0" 202 Accepted 2026-03-25 01:19:57,715 - lava_callback - INFO - Starting async job submit for node 69c0952dce0c9bc43bcce115 2026-03-25 01:19:57,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:57,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0952dce0c9bc43bcce115 HTTP/1.1" 200 None 2026-03-25 01:19:57,753 - lava_callback - DEBUG - Processing node 69c0952dce0c9bc43bcce115: baseline-arm64-clabbe 2026-03-25 01:19:57,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0952dce0c9bc43bcce115 2026-03-25 01:19:57,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:57,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0952dce0c9bc43bcce115 HTTP/1.1" 200 None 2026-03-25 01:19:57,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:57,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:19:57,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:57,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0952dce0c9bc43bcce115 HTTP/1.1" 200 None 2026-03-25 01:19:57,874 - lava_callback - ERROR - Error processing callback for node 69c0952dce0c9bc43bcce115: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:19:58,174 - lava_callback - INFO - Received callback for node 69c337cf18a3717aa65d2d6e INFO: 172.21.0.1:49422 - "POST /node/69c337cf18a3717aa65d2d6e HTTP/1.0" 202 Accepted 2026-03-25 01:19:58,182 - lava_callback - INFO - Starting async job submit for node 69c337cf18a3717aa65d2d6e 2026-03-25 01:19:58,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:58,190 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337cd18a3717aa65d2d67/log.txt.gz 2026-03-25 01:19:58,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:58,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:58,343 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b318a3717aa65d2d12/lava-logs.yaml 2026-03-25 01:19:58,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337cf18a3717aa65d2d6e HTTP/1.1" 200 None 2026-03-25 01:19:58,349 - lava_callback - DEBUG - Processing node 69c337cf18a3717aa65d2d6e: baseline-arm64-broonie 2026-03-25 01:19:58,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:58,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:58,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:59,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:59,045 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b318a3717aa65d2d12/lava_callback.json.gz 2026-03-25 01:19:59,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:59,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:59,124 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337cd18a3717aa65d2d67/lava-logs.yaml 2026-03-25 01:19:59,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:59,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337b318a3717aa65d2d12 HTTP/1.1" 200 None 2026-03-25 01:19:59,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:59,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:59,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:59,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337b318a3717aa65d2d12 HTTP/1.1" 200 None 2026-03-25 01:19:59,216 - lava_callback - INFO - Completed processing callback for node 69c337b318a3717aa65d2d12 2026-03-25 01:19:59,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:59,345 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337cf18a3717aa65d2d6e/log.txt.gz 2026-03-25 01:19:59,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:19:59,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:19:59,614 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337cd18a3717aa65d2d67/lava_callback.json.gz 2026-03-25 01:19:59,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:59,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337cd18a3717aa65d2d67 HTTP/1.1" 200 None 2026-03-25 01:19:59,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:59,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:19:59,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:59,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337cd18a3717aa65d2d67 HTTP/1.1" 200 None 2026-03-25 01:19:59,773 - lava_callback - INFO - Completed processing callback for node 69c337cd18a3717aa65d2d67 2026-03-25 01:19:59,822 - lava_callback - INFO - Received callback for node 69c337d618a3717aa65d2d80 INFO: 172.21.0.1:42846 - "POST /node/69c337d618a3717aa65d2d80 HTTP/1.0" 202 Accepted 2026-03-25 01:19:59,835 - lava_callback - INFO - Starting async job submit for node 69c337d618a3717aa65d2d80 2026-03-25 01:19:59,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:19:59,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d618a3717aa65d2d80 HTTP/1.1" 200 None 2026-03-25 01:19:59,976 - lava_callback - DEBUG - Processing node 69c337d618a3717aa65d2d80: baseline-arm64-clabbe 2026-03-25 01:20:00,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:00,186 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337cf18a3717aa65d2d6e/lava-logs.yaml 2026-03-25 01:20:00,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:00,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:00,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:00,793 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c337d618a3717aa65d2d80/log.txt.gz 2026-03-25 01:20:00,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:00,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337cf18a3717aa65d2d6e/lava_callback.json.gz 2026-03-25 01:20:00,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:00,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337cf18a3717aa65d2d6e HTTP/1.1" 200 None 2026-03-25 01:20:00,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:01,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:20:01,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:01,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:01,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337cf18a3717aa65d2d6e HTTP/1.1" 200 None 2026-03-25 01:20:01,126 - lava_callback - INFO - Completed processing callback for node 69c337cf18a3717aa65d2d6e 2026-03-25 01:20:01,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:01,500 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c337d618a3717aa65d2d80/lava-logs.yaml 2026-03-25 01:20:01,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:01,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:01,656 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c337d618a3717aa65d2d80/lava_callback.json.gz 2026-03-25 01:20:01,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:01,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d618a3717aa65d2d80 HTTP/1.1" 200 None 2026-03-25 01:20:01,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:01,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:20:01,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:01,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337d618a3717aa65d2d80 HTTP/1.1" 200 None 2026-03-25 01:20:01,765 - lava_callback - INFO - Completed processing callback for node 69c337d618a3717aa65d2d80 INFO: 172.21.0.1:42858 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:20:07,683 - lava_callback - INFO - Received callback for node 69c3383f18a3717aa65d2e51 INFO: 172.21.0.1:42864 - "POST /node/69c3383f18a3717aa65d2e51 HTTP/1.0" 202 Accepted 2026-03-25 01:20:07,689 - lava_callback - INFO - Starting async job submit for node 69c3383f18a3717aa65d2e51 2026-03-25 01:20:07,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:07,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3383f18a3717aa65d2e51 HTTP/1.1" 200 None 2026-03-25 01:20:07,711 - lava_callback - DEBUG - Processing node 69c3383f18a3717aa65d2e51: baseline-nfs-arm64 2026-03-25 01:20:07,712 - lava_callback - WARNING - Failed to upload log 2026-03-25 01:20:07,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:07,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:07,915 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3383f18a3717aa65d2e51/lava-logs.yaml 2026-03-25 01:20:07,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:08,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:08,254 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3383f18a3717aa65d2e51/lava_callback.json.gz 2026-03-25 01:20:08,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:08,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3383f18a3717aa65d2e51 HTTP/1.1" 200 None 2026-03-25 01:20:08,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:08,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:20:08,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:08,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3383f18a3717aa65d2e51 HTTP/1.1" 200 None 2026-03-25 01:20:08,346 - lava_callback - INFO - Completed processing callback for node 69c3383f18a3717aa65d2e51 INFO: 172.21.0.1:46168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46184 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:20:20,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:20,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-25 01:20:25,339 - lava_callback - INFO - Received callback for node 69c337d018a3717aa65d2d71 INFO: 172.21.0.1:50068 - "POST /node/69c337d018a3717aa65d2d71 HTTP/1.0" 202 Accepted 2026-03-25 01:20:25,349 - lava_callback - INFO - Starting async job submit for node 69c337d018a3717aa65d2d71 2026-03-25 01:20:25,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:25,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d018a3717aa65d2d71 HTTP/1.1" 200 None 2026-03-25 01:20:25,430 - lava_callback - DEBUG - Processing node 69c337d018a3717aa65d2d71: baseline-arm64-broonie 2026-03-25 01:20:25,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:26,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:26,024 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337d018a3717aa65d2d71/log.txt.gz 2026-03-25 01:20:26,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:50084 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:20:26,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:26,606 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337d018a3717aa65d2d71/lava-logs.yaml 2026-03-25 01:20:26,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:26,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:26,793 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337d018a3717aa65d2d71/lava_callback.json.gz 2026-03-25 01:20:26,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:26,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337d018a3717aa65d2d71 HTTP/1.1" 200 None 2026-03-25 01:20:26,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:26,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:20:26,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:26,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337d018a3717aa65d2d71 HTTP/1.1" 200 None 2026-03-25 01:20:26,912 - lava_callback - INFO - Completed processing callback for node 69c337d018a3717aa65d2d71 INFO: 172.21.0.1:46930 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:20:31,095 - lava_callback - INFO - Received callback for node 69c337b118a3717aa65d2d0c INFO: 172.21.0.1:46938 - "POST /node/69c337b118a3717aa65d2d0c HTTP/1.0" 202 Accepted 2026-03-25 01:20:31,106 - lava_callback - INFO - Starting async job submit for node 69c337b118a3717aa65d2d0c 2026-03-25 01:20:31,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:31,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337b118a3717aa65d2d0c HTTP/1.1" 200 None 2026-03-25 01:20:31,150 - lava_callback - DEBUG - Processing node 69c337b118a3717aa65d2d0c: baseline-nfs-arm64 2026-03-25 01:20:31,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:31,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:31,803 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b118a3717aa65d2d0c/log.txt.gz 2026-03-25 01:20:32,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:32,054 - lava_callback - INFO - Received callback for node 69c337da18a3717aa65d2da0 INFO: 172.21.0.1:46940 - "POST /node/69c337da18a3717aa65d2da0 HTTP/1.0" 202 Accepted 2026-03-25 01:20:32,061 - lava_callback - INFO - Starting async job submit for node 69c337da18a3717aa65d2da0 2026-03-25 01:20:32,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:32,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337da18a3717aa65d2da0 HTTP/1.1" 200 None 2026-03-25 01:20:32,098 - lava_callback - DEBUG - Processing node 69c337da18a3717aa65d2da0: baseline-arm64-pengutronix 2026-03-25 01:20:32,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:32,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:32,418 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b118a3717aa65d2d0c/lava-logs.yaml 2026-03-25 01:20:32,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:32,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:32,656 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c337da18a3717aa65d2da0/log.txt.gz 2026-03-25 01:20:32,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:32,750 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337b118a3717aa65d2d0c/lava_callback.json.gz 2026-03-25 01:20:32,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:32,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:32,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337b118a3717aa65d2d0c HTTP/1.1" 200 None 2026-03-25 01:20:32,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:32,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:20:32,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:32,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337b118a3717aa65d2d0c HTTP/1.1" 200 None 2026-03-25 01:20:32,884 - lava_callback - INFO - Completed processing callback for node 69c337b118a3717aa65d2d0c 2026-03-25 01:20:33,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:33,395 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c337da18a3717aa65d2da0/lava-logs.yaml 2026-03-25 01:20:33,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:33,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:33,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c337da18a3717aa65d2da0/lava_callback.json.gz 2026-03-25 01:20:33,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:33,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337da18a3717aa65d2da0 HTTP/1.1" 200 None 2026-03-25 01:20:33,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:33,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:20:33,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:33,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337da18a3717aa65d2da0 HTTP/1.1" 200 None 2026-03-25 01:20:33,789 - lava_callback - INFO - Completed processing callback for node 69c337da18a3717aa65d2da0 INFO: 172.21.0.1:46946 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:20:38,207 - lava_callback - INFO - Received callback for node 69c337ae18a3717aa65d2d06 INFO: 172.21.0.1:46956 - "POST /node/69c337ae18a3717aa65d2d06 HTTP/1.0" 202 Accepted 2026-03-25 01:20:38,218 - lava_callback - INFO - Starting async job submit for node 69c337ae18a3717aa65d2d06 2026-03-25 01:20:38,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:38,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337ae18a3717aa65d2d06 HTTP/1.1" 200 None 2026-03-25 01:20:38,292 - lava_callback - DEBUG - Processing node 69c337ae18a3717aa65d2d06: baseline-nfs-arm64 2026-03-25 01:20:39,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:39,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:39,326 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337ae18a3717aa65d2d06/log.txt.gz 2026-03-25 01:20:39,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:40,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:40,065 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337ae18a3717aa65d2d06/lava-logs.yaml 2026-03-25 01:20:40,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:40,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:40,255 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337ae18a3717aa65d2d06/lava_callback.json.gz 2026-03-25 01:20:40,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:40,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337ae18a3717aa65d2d06 HTTP/1.1" 200 None 2026-03-25 01:20:40,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:40,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:20:40,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:40,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337ae18a3717aa65d2d06 HTTP/1.1" 200 None 2026-03-25 01:20:40,341 - lava_callback - INFO - Completed processing callback for node 69c337ae18a3717aa65d2d06 2026-03-25 01:20:44,934 - lava_callback - INFO - Received callback for node 69c3379718a3717aa65d2cb7 INFO: 172.21.0.1:51436 - "POST /node/69c3379718a3717aa65d2cb7 HTTP/1.0" 202 Accepted 2026-03-25 01:20:44,941 - lava_callback - INFO - Starting async job submit for node 69c3379718a3717aa65d2cb7 2026-03-25 01:20:44,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:44,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379718a3717aa65d2cb7 HTTP/1.1" 200 None 2026-03-25 01:20:44,989 - lava_callback - DEBUG - Processing node 69c3379718a3717aa65d2cb7: baseline-arm64 2026-03-25 01:20:45,189 - lava_callback - WARNING - Failed to upload log 2026-03-25 01:20:45,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:45,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:45,685 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379718a3717aa65d2cb7/lava-logs.yaml 2026-03-25 01:20:45,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:20:45,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:20:45,807 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379718a3717aa65d2cb7/lava_callback.json.gz 2026-03-25 01:20:45,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:45,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379718a3717aa65d2cb7 HTTP/1.1" 200 None 2026-03-25 01:20:45,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:45,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:20:45,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:45,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379718a3717aa65d2cb7 HTTP/1.1" 200 None 2026-03-25 01:20:45,984 - lava_callback - INFO - Completed processing callback for node 69c3379718a3717aa65d2cb7 INFO: 172.21.0.1:51450 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:20:50,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:20:50,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44798 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:21:17,816 - lava_callback - INFO - Received callback for node 69c0957cce0c9bc43bcce126 INFO: 172.21.0.1:44800 - "POST /node/69c0957cce0c9bc43bcce126 HTTP/1.0" 202 Accepted 2026-03-25 01:21:17,822 - lava_callback - INFO - Starting async job submit for node 69c0957cce0c9bc43bcce126 2026-03-25 01:21:17,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:17,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0957cce0c9bc43bcce126 HTTP/1.1" 200 None 2026-03-25 01:21:17,838 - lava_callback - DEBUG - Processing node 69c0957cce0c9bc43bcce126: baseline-arm64-clabbe 2026-03-25 01:21:17,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0957cce0c9bc43bcce126 2026-03-25 01:21:17,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:17,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0957cce0c9bc43bcce126 HTTP/1.1" 200 None 2026-03-25 01:21:17,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:17,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:21:17,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:17,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0957cce0c9bc43bcce126 HTTP/1.1" 200 None 2026-03-25 01:21:17,890 - lava_callback - ERROR - Error processing callback for node 69c0957cce0c9bc43bcce126: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37778 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:21:42,847 - lava_callback - INFO - Received callback for node 69c337a718a3717aa65d2cf7 INFO: 172.21.0.1:56494 - "POST /node/69c337a718a3717aa65d2cf7 HTTP/1.0" 202 Accepted 2026-03-25 01:21:42,854 - lava_callback - INFO - Starting async job submit for node 69c337a718a3717aa65d2cf7 2026-03-25 01:21:42,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:42,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a718a3717aa65d2cf7 HTTP/1.1" 200 None 2026-03-25 01:21:42,892 - lava_callback - DEBUG - Processing node 69c337a718a3717aa65d2cf7: baseline-nfs-arm64 2026-03-25 01:21:43,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:21:43,357 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:21:43,359 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337a718a3717aa65d2cf7/log.txt.gz 2026-03-25 01:21:43,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:21:43,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:21:43,783 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337a718a3717aa65d2cf7/lava-logs.yaml 2026-03-25 01:21:43,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:21:43,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:21:43,944 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337a718a3717aa65d2cf7/lava_callback.json.gz 2026-03-25 01:21:43,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:43,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a718a3717aa65d2cf7 HTTP/1.1" 200 None 2026-03-25 01:21:43,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:43,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:21:44,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:44,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337a718a3717aa65d2cf7 HTTP/1.1" 200 None 2026-03-25 01:21:44,030 - lava_callback - INFO - Completed processing callback for node 69c337a718a3717aa65d2cf7 2026-03-25 01:21:45,197 - lava_callback - INFO - Received callback for node 69c337ca18a3717aa65d2d61 INFO: 172.21.0.1:56502 - "POST /node/69c337ca18a3717aa65d2d61 HTTP/1.0" 202 Accepted 2026-03-25 01:21:45,203 - lava_callback - INFO - Starting async job submit for node 69c337ca18a3717aa65d2d61 2026-03-25 01:21:45,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:45,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337ca18a3717aa65d2d61 HTTP/1.1" 200 None 2026-03-25 01:21:45,241 - lava_callback - DEBUG - Processing node 69c337ca18a3717aa65d2d61: baseline-arm64-broonie 2026-03-25 01:21:45,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:21:45,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:21:45,614 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337ca18a3717aa65d2d61/log.txt.gz 2026-03-25 01:21:45,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:21:45,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:21:45,961 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337ca18a3717aa65d2d61/lava-logs.yaml 2026-03-25 01:21:45,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:21:46,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:21:46,122 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337ca18a3717aa65d2d61/lava_callback.json.gz 2026-03-25 01:21:46,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:46,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337ca18a3717aa65d2d61 HTTP/1.1" 200 None 2026-03-25 01:21:46,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:46,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:21:46,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:46,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337ca18a3717aa65d2d61 HTTP/1.1" 200 None 2026-03-25 01:21:46,210 - lava_callback - INFO - Completed processing callback for node 69c337ca18a3717aa65d2d61 INFO: 172.21.0.1:56514 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:21:50,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:21:50,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47800 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:22:57,899 - lava_callback - INFO - Received callback for node 69c095ecce0c9bc43bcce13c INFO: 172.21.0.1:47814 - "POST /node/69c095ecce0c9bc43bcce13c HTTP/1.0" 202 Accepted 2026-03-25 01:22:57,905 - lava_callback - INFO - Starting async job submit for node 69c095ecce0c9bc43bcce13c 2026-03-25 01:22:57,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:22:57,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c095ecce0c9bc43bcce13c HTTP/1.1" 200 None 2026-03-25 01:22:57,921 - lava_callback - DEBUG - Processing node 69c095ecce0c9bc43bcce13c: baseline-arm64-clabbe 2026-03-25 01:22:57,921 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c095ecce0c9bc43bcce13c 2026-03-25 01:22:57,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:22:57,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c095ecce0c9bc43bcce13c HTTP/1.1" 200 None 2026-03-25 01:22:57,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:22:57,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:22:57,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:22:57,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c095ecce0c9bc43bcce13c HTTP/1.1" 200 None 2026-03-25 01:22:57,981 - lava_callback - ERROR - Error processing callback for node 69c095ecce0c9bc43bcce13c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46944 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:23:28,468 - lava_callback - INFO - Received callback for node 69c338d918a3717aa65d2ebf INFO: 172.21.0.1:46954 - "POST /node/69c338d918a3717aa65d2ebf HTTP/1.0" 202 Accepted 2026-03-25 01:23:28,475 - lava_callback - INFO - Starting async job submit for node 69c338d918a3717aa65d2ebf 2026-03-25 01:23:28,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:23:28,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c338d918a3717aa65d2ebf HTTP/1.1" 200 None 2026-03-25 01:23:28,520 - lava_callback - DEBUG - Processing node 69c338d918a3717aa65d2ebf: baseline-arm64 2026-03-25 01:23:28,653 - lava_callback - WARNING - Failed to upload log 2026-03-25 01:23:28,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:23:28,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:23:28,912 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c338d918a3717aa65d2ebf/lava-logs.yaml 2026-03-25 01:23:28,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:23:29,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:23:29,029 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c338d918a3717aa65d2ebf/lava_callback.json.gz 2026-03-25 01:23:29,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:23:29,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c338d918a3717aa65d2ebf HTTP/1.1" 200 None 2026-03-25 01:23:29,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:23:29,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:23:29,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:23:29,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c338d918a3717aa65d2ebf HTTP/1.1" 200 None 2026-03-25 01:23:29,167 - lava_callback - INFO - Completed processing callback for node 69c338d918a3717aa65d2ebf INFO: 172.21.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44736 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:23:50,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:23:50,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49240 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:23:56,695 - lava_callback - INFO - Received callback for node 69c337c918a3717aa65d2d5e INFO: 172.21.0.1:49250 - "POST /node/69c337c918a3717aa65d2d5e HTTP/1.0" 202 Accepted 2026-03-25 01:23:56,701 - lava_callback - INFO - Starting async job submit for node 69c337c918a3717aa65d2d5e 2026-03-25 01:23:56,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:23:56,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337c918a3717aa65d2d5e HTTP/1.1" 200 None 2026-03-25 01:23:56,738 - lava_callback - DEBUG - Processing node 69c337c918a3717aa65d2d5e: baseline-arm64-broonie 2026-03-25 01:23:57,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:23:57,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:23:57,162 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c918a3717aa65d2d5e/log.txt.gz 2026-03-25 01:23:57,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:23:57,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:23:57,514 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c918a3717aa65d2d5e/lava-logs.yaml 2026-03-25 01:23:57,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:23:57,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:23:57,646 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c918a3717aa65d2d5e/lava_callback.json.gz 2026-03-25 01:23:57,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:23:57,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337c918a3717aa65d2d5e HTTP/1.1" 200 None 2026-03-25 01:23:57,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:23:57,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:23:57,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:23:57,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337c918a3717aa65d2d5e HTTP/1.1" 200 None 2026-03-25 01:23:57,738 - lava_callback - INFO - Completed processing callback for node 69c337c918a3717aa65d2d5e INFO: 172.21.0.1:41456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48692 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:24:17,956 - lava_callback - INFO - Received callback for node 69c09637ce0c9bc43bcce14a INFO: 172.21.0.1:48700 - "POST /node/69c09637ce0c9bc43bcce14a HTTP/1.0" 202 Accepted 2026-03-25 01:24:17,961 - lava_callback - INFO - Starting async job submit for node 69c09637ce0c9bc43bcce14a 2026-03-25 01:24:17,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:24:17,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09637ce0c9bc43bcce14a HTTP/1.1" 200 None 2026-03-25 01:24:17,978 - lava_callback - DEBUG - Processing node 69c09637ce0c9bc43bcce14a: baseline-arm64-clabbe 2026-03-25 01:24:17,978 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09637ce0c9bc43bcce14a 2026-03-25 01:24:17,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:24:17,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09637ce0c9bc43bcce14a HTTP/1.1" 200 None 2026-03-25 01:24:17,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:24:18,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:24:18,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:24:18,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09637ce0c9bc43bcce14a HTTP/1.1" 200 None 2026-03-25 01:24:18,030 - lava_callback - ERROR - Error processing callback for node 69c09637ce0c9bc43bcce14a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:24:20,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:24:20,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50582 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:25:38,280 - lava_callback - INFO - Received callback for node 69c09681ce0c9bc43bcce156 INFO: 172.21.0.1:50594 - "POST /node/69c09681ce0c9bc43bcce156 HTTP/1.0" 202 Accepted 2026-03-25 01:25:38,285 - lava_callback - INFO - Starting async job submit for node 69c09681ce0c9bc43bcce156 2026-03-25 01:25:38,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:25:38,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09681ce0c9bc43bcce156 HTTP/1.1" 200 None 2026-03-25 01:25:38,302 - lava_callback - DEBUG - Processing node 69c09681ce0c9bc43bcce156: baseline-arm64-clabbe 2026-03-25 01:25:38,302 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09681ce0c9bc43bcce156 2026-03-25 01:25:38,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:25:38,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09681ce0c9bc43bcce156 HTTP/1.1" 200 None 2026-03-25 01:25:38,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:25:38,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 01:25:38,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:25:38,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09681ce0c9bc43bcce156 HTTP/1.1" 200 None 2026-03-25 01:25:38,368 - lava_callback - ERROR - Error processing callback for node 69c09681ce0c9bc43bcce156: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39876 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:26:32,362 - lava_callback - INFO - Received callback for node 69c3398018a3717aa65d2ee3 INFO: 172.21.0.1:39888 - "POST /node/69c3398018a3717aa65d2ee3 HTTP/1.0" 202 Accepted 2026-03-25 01:26:32,369 - lava_callback - INFO - Starting async job submit for node 69c3398018a3717aa65d2ee3 2026-03-25 01:26:32,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:26:32,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3398018a3717aa65d2ee3 HTTP/1.1" 200 None 2026-03-25 01:26:32,406 - lava_callback - DEBUG - Processing node 69c3398018a3717aa65d2ee3: baseline-arm64 2026-03-25 01:26:32,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:26:32,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:26:32,840 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3398018a3717aa65d2ee3/log.txt.gz 2026-03-25 01:26:33,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:26:33,309 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:26:33,311 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3398018a3717aa65d2ee3/lava-logs.yaml 2026-03-25 01:26:33,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:26:33,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:26:33,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3398018a3717aa65d2ee3/lava_callback.json.gz 2026-03-25 01:26:33,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:26:33,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3398018a3717aa65d2ee3 HTTP/1.1" 200 None 2026-03-25 01:26:33,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:26:33,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:26:33,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:26:33,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3398018a3717aa65d2ee3 HTTP/1.1" 200 None 2026-03-25 01:26:33,620 - lava_callback - INFO - Completed processing callback for node 69c3398018a3717aa65d2ee3 INFO: 172.21.0.1:39900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59366 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:26:50,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:26:50,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51258 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:27:44,782 - lava_callback - INFO - Received callback for node 69c339cf18a3717aa65d2ef2 INFO: 172.21.0.1:60772 - "POST /node/69c339cf18a3717aa65d2ef2 HTTP/1.0" 202 Accepted 2026-03-25 01:27:44,788 - lava_callback - INFO - Starting async job submit for node 69c339cf18a3717aa65d2ef2 2026-03-25 01:27:44,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:27:44,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c339cf18a3717aa65d2ef2 HTTP/1.1" 200 None 2026-03-25 01:27:44,825 - lava_callback - DEBUG - Processing node 69c339cf18a3717aa65d2ef2: baseline-arm64 2026-03-25 01:27:45,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:27:45,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:27:45,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c339cf18a3717aa65d2ef2/log.txt.gz 2026-03-25 01:27:45,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:27:46,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:27:46,017 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c339cf18a3717aa65d2ef2/lava-logs.yaml 2026-03-25 01:27:46,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:27:46,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:27:46,202 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c339cf18a3717aa65d2ef2/lava_callback.json.gz 2026-03-25 01:27:46,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:27:46,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c339cf18a3717aa65d2ef2 HTTP/1.1" 200 None 2026-03-25 01:27:46,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:27:46,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:27:46,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:27:46,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c339cf18a3717aa65d2ef2 HTTP/1.1" 200 None 2026-03-25 01:27:46,308 - lava_callback - INFO - Completed processing callback for node 69c339cf18a3717aa65d2ef2 INFO: 172.21.0.1:60774 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:27:50,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:27:50,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35054 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:29:17,218 - lava_callback - INFO - Received callback for node 69c33a1c18a3717aa65d2f09 INFO: 172.21.0.1:35056 - "POST /node/69c33a1c18a3717aa65d2f09 HTTP/1.0" 202 Accepted 2026-03-25 01:29:17,224 - lava_callback - INFO - Starting async job submit for node 69c33a1c18a3717aa65d2f09 2026-03-25 01:29:17,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:29:17,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33a1c18a3717aa65d2f09 HTTP/1.1" 200 None 2026-03-25 01:29:17,260 - lava_callback - DEBUG - Processing node 69c33a1c18a3717aa65d2f09: baseline-arm64 2026-03-25 01:29:17,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:29:17,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:29:17,648 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33a1c18a3717aa65d2f09/log.txt.gz 2026-03-25 01:29:17,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:29:17,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:29:17,995 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33a1c18a3717aa65d2f09/lava-logs.yaml 2026-03-25 01:29:18,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:29:18,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:29:18,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33a1c18a3717aa65d2f09/lava_callback.json.gz 2026-03-25 01:29:18,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:29:18,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33a1c18a3717aa65d2f09 HTTP/1.1" 200 None 2026-03-25 01:29:18,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:29:18,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:29:18,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:29:18,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33a1c18a3717aa65d2f09 HTTP/1.1" 200 None 2026-03-25 01:29:18,387 - lava_callback - INFO - Completed processing callback for node 69c33a1c18a3717aa65d2f09 2026-03-25 01:29:20,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:29:21,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60248 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:30:34,757 - lava_callback - INFO - Received callback for node 69c3376d18a3717aa65d2c7b INFO: 172.21.0.1:60252 - "POST /node/69c3376d18a3717aa65d2c7b HTTP/1.0" 202 Accepted 2026-03-25 01:30:34,763 - lava_callback - INFO - Starting async job submit for node 69c3376d18a3717aa65d2c7b 2026-03-25 01:30:34,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:30:34,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3376d18a3717aa65d2c7b HTTP/1.1" 200 None 2026-03-25 01:30:34,826 - lava_callback - DEBUG - Processing node 69c3376d18a3717aa65d2c7b: baseline-arm 2026-03-25 01:30:35,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:30:35,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:30:35,419 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c3376d18a3717aa65d2c7b/log.txt.gz 2026-03-25 01:30:35,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:30:35,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:30:35,929 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c3376d18a3717aa65d2c7b/lava-logs.yaml 2026-03-25 01:30:35,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:30:36,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:30:36,111 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c3376d18a3717aa65d2c7b/lava_callback.json.gz DEBUG: Setup failed for 69c3376d18a3717aa65d2c7b. Transitting job node result: incomplete -> fail 2026-03-25 01:30:36,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:30:36,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3376d18a3717aa65d2c7b HTTP/1.1" 200 None 2026-03-25 01:30:36,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:30:36,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 01:30:36,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:30:36,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3376d18a3717aa65d2c7b HTTP/1.1" 200 None 2026-03-25 01:30:36,356 - lava_callback - INFO - Completed processing callback for node 69c3376d18a3717aa65d2c7b INFO: 172.21.0.1:60264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37570 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:30:51,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:30:51,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-25 01:30:54,241 - lava_callback - INFO - Received callback for node 69c097cbce0c9bc43bcce17b INFO: 172.21.0.1:52774 - "POST /node/69c097cbce0c9bc43bcce17b HTTP/1.0" 202 Accepted 2026-03-25 01:30:54,247 - lava_callback - INFO - Starting async job submit for node 69c097cbce0c9bc43bcce17b 2026-03-25 01:30:54,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:30:54,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c097cbce0c9bc43bcce17b HTTP/1.1" 200 None 2026-03-25 01:30:54,262 - lava_callback - DEBUG - Processing node 69c097cbce0c9bc43bcce17b: baseline-arm64-baylibre 2026-03-25 01:30:54,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c097cbce0c9bc43bcce17b 2026-03-25 01:30:54,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:30:54,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c097cbce0c9bc43bcce17b HTTP/1.1" 200 None 2026-03-25 01:30:54,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:30:54,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:30:54,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:30:54,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c097cbce0c9bc43bcce17b HTTP/1.1" 200 None 2026-03-25 01:30:54,320 - lava_callback - ERROR - Error processing callback for node 69c097cbce0c9bc43bcce17b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49884 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:31:08,112 - lava_callback - INFO - Received callback for node 69c33ac618a3717aa65d2f24 INFO: 172.21.0.1:49898 - "POST /node/69c33ac618a3717aa65d2f24 HTTP/1.0" 202 Accepted 2026-03-25 01:31:08,118 - lava_callback - INFO - Starting async job submit for node 69c33ac618a3717aa65d2f24 2026-03-25 01:31:08,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:31:08,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33ac618a3717aa65d2f24 HTTP/1.1" 200 None 2026-03-25 01:31:08,136 - lava_callback - DEBUG - Processing node 69c33ac618a3717aa65d2f24: baseline-nfs-arm64 2026-03-25 01:31:08,137 - lava_callback - WARNING - Failed to upload log 2026-03-25 01:31:08,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:31:08,290 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:31:08,291 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c33ac618a3717aa65d2f24/lava-logs.yaml 2026-03-25 01:31:08,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:31:08,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:31:08,429 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c33ac618a3717aa65d2f24/lava_callback.json.gz 2026-03-25 01:31:08,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:31:08,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33ac618a3717aa65d2f24 HTTP/1.1" 200 None 2026-03-25 01:31:08,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:31:08,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:31:08,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:31:08,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33ac618a3717aa65d2f24 HTTP/1.1" 200 None 2026-03-25 01:31:08,493 - lava_callback - INFO - Completed processing callback for node 69c33ac618a3717aa65d2f24 INFO: 172.21.0.1:58166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58168 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:31:21,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:31:21,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42514 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:32:13,956 - lava_callback - INFO - Received callback for node 69c09818ce0c9bc43bcce18c INFO: 172.21.0.1:37424 - "POST /node/69c09818ce0c9bc43bcce18c HTTP/1.0" 202 Accepted 2026-03-25 01:32:13,961 - lava_callback - INFO - Starting async job submit for node 69c09818ce0c9bc43bcce18c 2026-03-25 01:32:13,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:32:13,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09818ce0c9bc43bcce18c HTTP/1.1" 200 None 2026-03-25 01:32:13,978 - lava_callback - DEBUG - Processing node 69c09818ce0c9bc43bcce18c: baseline-arm64-baylibre 2026-03-25 01:32:13,978 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09818ce0c9bc43bcce18c 2026-03-25 01:32:13,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:32:13,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09818ce0c9bc43bcce18c HTTP/1.1" 200 None 2026-03-25 01:32:13,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:32:14,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:32:14,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:32:14,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09818ce0c9bc43bcce18c HTTP/1.1" 200 None 2026-03-25 01:32:14,043 - lava_callback - ERROR - Error processing callback for node 69c09818ce0c9bc43bcce18c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37440 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:32:21,972 - lava_callback - INFO - Received callback for node 69c3333e18a3717aa65d2b74 INFO: 172.21.0.1:60536 - "POST /node/69c3333e18a3717aa65d2b74 HTTP/1.0" 202 Accepted 2026-03-25 01:32:22,022 - lava_callback - INFO - Starting async job submit for node 69c3333e18a3717aa65d2b74 2026-03-25 01:32:22,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:32:22,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3333e18a3717aa65d2b74 HTTP/1.1" 200 None 2026-03-25 01:32:22,087 - lava_callback - DEBUG - Processing node 69c3333e18a3717aa65d2b74: baseline-arm-pengutronix 2026-03-25 01:32:22,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:32:22,397 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:32:22,399 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c3333e18a3717aa65d2b74/log.txt.gz 2026-03-25 01:32:22,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:32:22,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:32:22,641 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c3333e18a3717aa65d2b74/lava-logs.yaml 2026-03-25 01:32:22,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:32:22,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:32:22,806 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c3333e18a3717aa65d2b74/lava_callback.json.gz DEBUG: Setup failed for 69c3333e18a3717aa65d2b74. Transitting job node result: incomplete -> fail 2026-03-25 01:32:22,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:32:22,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3333e18a3717aa65d2b74 HTTP/1.1" 200 None 2026-03-25 01:32:22,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:32:22,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 01:32:22,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:32:22,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3333e18a3717aa65d2b74 HTTP/1.1" 200 None 2026-03-25 01:32:22,975 - lava_callback - INFO - Completed processing callback for node 69c3333e18a3717aa65d2b74 INFO: 172.21.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52230 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:32:51,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:32:51,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58332 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:33:12,807 - lava_callback - INFO - Received callback for node 69c33b2818a3717aa65d2f40 INFO: 172.21.0.1:44894 - "POST /node/69c33b2818a3717aa65d2f40 HTTP/1.0" 202 Accepted 2026-03-25 01:33:12,815 - lava_callback - INFO - Starting async job submit for node 69c33b2818a3717aa65d2f40 2026-03-25 01:33:12,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:12,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33b2818a3717aa65d2f40 HTTP/1.1" 200 None 2026-03-25 01:33:12,882 - lava_callback - DEBUG - Processing node 69c33b2818a3717aa65d2f40: baseline-arm64-pengutronix 2026-03-25 01:33:13,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:33:13,301 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:33:13,303 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c33b2818a3717aa65d2f40/log.txt.gz 2026-03-25 01:33:13,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:33:13,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:33:13,571 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c33b2818a3717aa65d2f40/lava-logs.yaml 2026-03-25 01:33:13,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:33:13,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:33:13,747 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c33b2818a3717aa65d2f40/lava_callback.json.gz 2026-03-25 01:33:13,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:13,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33b2818a3717aa65d2f40 HTTP/1.1" 200 None 2026-03-25 01:33:13,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:13,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:33:13,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:13,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33b2818a3717aa65d2f40 HTTP/1.1" 200 None 2026-03-25 01:33:13,849 - lava_callback - INFO - Completed processing callback for node 69c33b2818a3717aa65d2f40 INFO: 172.21.0.1:44900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44902 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:33:21,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:21,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57558 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:33:34,045 - lava_callback - INFO - Received callback for node 69c09864ce0c9bc43bcce19a INFO: 172.21.0.1:57560 - "POST /node/69c09864ce0c9bc43bcce19a HTTP/1.0" 202 Accepted 2026-03-25 01:33:34,050 - lava_callback - INFO - Starting async job submit for node 69c09864ce0c9bc43bcce19a 2026-03-25 01:33:34,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:34,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09864ce0c9bc43bcce19a HTTP/1.1" 200 None 2026-03-25 01:33:34,067 - lava_callback - DEBUG - Processing node 69c09864ce0c9bc43bcce19a: baseline-arm64-baylibre 2026-03-25 01:33:34,067 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09864ce0c9bc43bcce19a 2026-03-25 01:33:34,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:34,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09864ce0c9bc43bcce19a HTTP/1.1" 200 None 2026-03-25 01:33:34,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:34,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:33:34,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:34,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09864ce0c9bc43bcce19a HTTP/1.1" 200 None 2026-03-25 01:33:34,123 - lava_callback - ERROR - Error processing callback for node 69c09864ce0c9bc43bcce19a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57566 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:33:39,813 - lava_callback - INFO - Received callback for node 69c337a618a3717aa65d2cf4 INFO: 172.21.0.1:39808 - "POST /node/69c337a618a3717aa65d2cf4 HTTP/1.0" 202 Accepted 2026-03-25 01:33:39,820 - lava_callback - INFO - Starting async job submit for node 69c337a618a3717aa65d2cf4 2026-03-25 01:33:39,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:39,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a618a3717aa65d2cf4 HTTP/1.1" 200 None 2026-03-25 01:33:39,871 - lava_callback - DEBUG - Processing node 69c337a618a3717aa65d2cf4: baseline-arm64 2026-03-25 01:33:40,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:33:40,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:33:40,431 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a618a3717aa65d2cf4/log.txt.gz 2026-03-25 01:33:40,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:33:40,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:33:40,903 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a618a3717aa65d2cf4/lava-logs.yaml 2026-03-25 01:33:40,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:33:41,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:33:41,079 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c337a618a3717aa65d2cf4/lava_callback.json.gz DEBUG: Setup failed for 69c337a618a3717aa65d2cf4. Transitting job node result: incomplete -> fail 2026-03-25 01:33:41,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:41,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a618a3717aa65d2cf4 HTTP/1.1" 200 None 2026-03-25 01:33:41,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:41,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:33:41,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:41,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337a618a3717aa65d2cf4 HTTP/1.1" 200 None 2026-03-25 01:33:41,332 - lava_callback - INFO - Completed processing callback for node 69c337a618a3717aa65d2cf4 INFO: 172.21.0.1:39812 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:33:51,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:33:51,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34386 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:34:54,110 - lava_callback - INFO - Received callback for node 69c098abce0c9bc43bcce1a0 INFO: 172.21.0.1:54994 - "POST /node/69c098abce0c9bc43bcce1a0 HTTP/1.0" 202 Accepted 2026-03-25 01:34:54,116 - lava_callback - INFO - Starting async job submit for node 69c098abce0c9bc43bcce1a0 2026-03-25 01:34:54,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:34:54,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c098abce0c9bc43bcce1a0 HTTP/1.1" 200 None 2026-03-25 01:34:54,132 - lava_callback - DEBUG - Processing node 69c098abce0c9bc43bcce1a0: baseline-arm64-baylibre 2026-03-25 01:34:54,132 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c098abce0c9bc43bcce1a0 2026-03-25 01:34:54,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:34:54,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c098abce0c9bc43bcce1a0 HTTP/1.1" 200 None 2026-03-25 01:34:54,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:34:54,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:34:54,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:34:54,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c098abce0c9bc43bcce1a0 HTTP/1.1" 200 None 2026-03-25 01:34:54,192 - lava_callback - ERROR - Error processing callback for node 69c098abce0c9bc43bcce1a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55000 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:35:04,014 - lava_callback - INFO - Received callback for node 69c33b6218a3717aa65d2f50 INFO: 172.21.0.1:52168 - "POST /node/69c33b6218a3717aa65d2f50 HTTP/1.0" 202 Accepted 2026-03-25 01:35:04,021 - lava_callback - INFO - Starting async job submit for node 69c33b6218a3717aa65d2f50 2026-03-25 01:35:04,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:04,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33b6218a3717aa65d2f50 HTTP/1.1" 200 None 2026-03-25 01:35:04,068 - lava_callback - DEBUG - Processing node 69c33b6218a3717aa65d2f50: baseline-arm64 2026-03-25 01:35:04,274 - lava_callback - WARNING - Failed to upload log 2026-03-25 01:35:04,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:35:04,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:35:04,621 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33b6218a3717aa65d2f50/lava-logs.yaml 2026-03-25 01:35:04,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:35:04,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:35:04,827 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33b6218a3717aa65d2f50/lava_callback.json.gz 2026-03-25 01:35:04,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:04,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33b6218a3717aa65d2f50 HTTP/1.1" 200 None 2026-03-25 01:35:04,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:04,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:35:04,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:04,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33b6218a3717aa65d2f50 HTTP/1.1" 200 None 2026-03-25 01:35:05,016 - lava_callback - INFO - Completed processing callback for node 69c33b6218a3717aa65d2f50 INFO: 172.21.0.1:52170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37770 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:35:21,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:21,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-25 01:35:26,164 - lava_callback - INFO - Received callback for node 69c337c218a3717aa65d2d4e INFO: 172.21.0.1:53780 - "POST /node/69c337c218a3717aa65d2d4e HTTP/1.0" 202 Accepted 2026-03-25 01:35:26,170 - lava_callback - INFO - Starting async job submit for node 69c337c218a3717aa65d2d4e 2026-03-25 01:35:26,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:26,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337c218a3717aa65d2d4e HTTP/1.1" 200 None 2026-03-25 01:35:26,211 - lava_callback - DEBUG - Processing node 69c337c218a3717aa65d2d4e: baseline-arm64-broonie 2026-03-25 01:35:26,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:53786 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:35:26,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:35:26,622 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c218a3717aa65d2d4e/log.txt.gz 2026-03-25 01:35:26,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:35:26,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:35:26,971 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c218a3717aa65d2d4e/lava-logs.yaml 2026-03-25 01:35:26,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:35:27,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:35:27,157 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c218a3717aa65d2d4e/lava_callback.json.gz 2026-03-25 01:35:27,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:27,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337c218a3717aa65d2d4e HTTP/1.1" 200 None 2026-03-25 01:35:27,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:27,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:35:27,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:27,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337c218a3717aa65d2d4e HTTP/1.1" 200 None 2026-03-25 01:35:27,237 - lava_callback - INFO - Completed processing callback for node 69c337c218a3717aa65d2d4e INFO: 172.21.0.1:53962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51858 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:35:51,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:51,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-25 01:35:54,124 - lava_callback - INFO - Received callback for node 69c098f8ce0c9bc43bcce1b4 INFO: 172.21.0.1:44452 - "POST /node/69c098f8ce0c9bc43bcce1b4 HTTP/1.0" 202 Accepted 2026-03-25 01:35:54,129 - lava_callback - INFO - Starting async job submit for node 69c098f8ce0c9bc43bcce1b4 2026-03-25 01:35:54,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:54,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c098f8ce0c9bc43bcce1b4 HTTP/1.1" 200 None 2026-03-25 01:35:54,146 - lava_callback - DEBUG - Processing node 69c098f8ce0c9bc43bcce1b4: baseline-arm64-baylibre 2026-03-25 01:35:54,146 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c098f8ce0c9bc43bcce1b4 2026-03-25 01:35:54,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:54,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c098f8ce0c9bc43bcce1b4 HTTP/1.1" 200 None 2026-03-25 01:35:54,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:54,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:35:54,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:35:54,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c098f8ce0c9bc43bcce1b4 HTTP/1.1" 200 None 2026-03-25 01:35:54,203 - lava_callback - ERROR - Error processing callback for node 69c098f8ce0c9bc43bcce1b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52416 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:36:26,201 - lava_callback - INFO - Received callback for node 69c337e218a3717aa65d2dbf INFO: 172.21.0.1:41826 - "POST /node/69c337e218a3717aa65d2dbf HTTP/1.0" 202 Accepted 2026-03-25 01:36:26,208 - lava_callback - INFO - Starting async job submit for node 69c337e218a3717aa65d2dbf 2026-03-25 01:36:26,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:36:26,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337e218a3717aa65d2dbf HTTP/1.1" 200 None 2026-03-25 01:36:26,294 - lava_callback - DEBUG - Processing node 69c337e218a3717aa65d2dbf: ltp-smoke INFO: 172.21.0.1:41842 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:36:26,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:36:26,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:36:26,965 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c337e218a3717aa65d2dbf/log.txt.gz 2026-03-25 01:36:27,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:36:27,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:36:27,477 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c337e218a3717aa65d2dbf/lava-logs.yaml 2026-03-25 01:36:27,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:36:27,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:36:27,702 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c337e218a3717aa65d2dbf/lava_callback.json.gz 2026-03-25 01:36:27,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:36:27,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337e218a3717aa65d2dbf HTTP/1.1" 200 None 2026-03-25 01:36:27,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:36:27,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3294718a3717aa65d01b3 HTTP/1.1" 200 None 2026-03-25 01:36:27,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:36:27,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337e218a3717aa65d2dbf HTTP/1.1" 200 None 2026-03-25 01:36:28,018 - lava_callback - INFO - Completed processing callback for node 69c337e218a3717aa65d2dbf INFO: 172.21.0.1:48574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60464 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:36:51,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:36:51,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46466 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:37:14,272 - lava_callback - INFO - Received callback for node 69c09943ce0c9bc43bcce1b9 INFO: 172.21.0.1:56976 - "POST /node/69c09943ce0c9bc43bcce1b9 HTTP/1.0" 202 Accepted 2026-03-25 01:37:14,277 - lava_callback - INFO - Starting async job submit for node 69c09943ce0c9bc43bcce1b9 2026-03-25 01:37:14,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:37:14,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09943ce0c9bc43bcce1b9 HTTP/1.1" 200 None 2026-03-25 01:37:14,294 - lava_callback - DEBUG - Processing node 69c09943ce0c9bc43bcce1b9: baseline-arm64-baylibre 2026-03-25 01:37:14,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09943ce0c9bc43bcce1b9 2026-03-25 01:37:14,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:37:14,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09943ce0c9bc43bcce1b9 HTTP/1.1" 200 None 2026-03-25 01:37:14,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:37:14,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:37:14,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:37:14,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09943ce0c9bc43bcce1b9 HTTP/1.1" 200 None 2026-03-25 01:37:14,358 - lava_callback - ERROR - Error processing callback for node 69c09943ce0c9bc43bcce1b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57006 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:37:20,702 - lava_callback - INFO - Received callback for node 69c3379b18a3717aa65d2cc2 INFO: 172.21.0.1:34842 - "POST /node/69c3379b18a3717aa65d2cc2 HTTP/1.0" 202 Accepted 2026-03-25 01:37:20,708 - lava_callback - INFO - Starting async job submit for node 69c3379b18a3717aa65d2cc2 2026-03-25 01:37:20,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:37:20,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379b18a3717aa65d2cc2 HTTP/1.1" 200 None 2026-03-25 01:37:20,746 - lava_callback - DEBUG - Processing node 69c3379b18a3717aa65d2cc2: baseline-arm64 2026-03-25 01:37:21,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:37:21,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:37:21,240 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379b18a3717aa65d2cc2/log.txt.gz 2026-03-25 01:37:21,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:37:21,663 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:37:21,665 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379b18a3717aa65d2cc2/lava-logs.yaml 2026-03-25 01:37:21,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:37:21,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:37:21,837 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379b18a3717aa65d2cc2/lava_callback.json.gz 2026-03-25 01:37:21,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:37:21,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379b18a3717aa65d2cc2 HTTP/1.1" 200 None 2026-03-25 01:37:21,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:37:21,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:37:21,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:37:21,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379b18a3717aa65d2cc2 HTTP/1.1" 200 None 2026-03-25 01:37:21,936 - lava_callback - INFO - Completed processing callback for node 69c3379b18a3717aa65d2cc2 INFO: 172.21.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46290 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:37:51,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:37:51,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59450 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:38:38,629 - lava_callback - INFO - Received callback for node 69c09992ce0c9bc43bcce1c4 INFO: 172.21.0.1:59462 - "POST /node/69c09992ce0c9bc43bcce1c4 HTTP/1.0" 202 Accepted 2026-03-25 01:38:38,635 - lava_callback - INFO - Starting async job submit for node 69c09992ce0c9bc43bcce1c4 2026-03-25 01:38:38,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:38:38,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09992ce0c9bc43bcce1c4 HTTP/1.1" 200 None 2026-03-25 01:38:38,651 - lava_callback - DEBUG - Processing node 69c09992ce0c9bc43bcce1c4: baseline-arm64-clabbe 2026-03-25 01:38:38,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09992ce0c9bc43bcce1c4 2026-03-25 01:38:38,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:38:38,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09992ce0c9bc43bcce1c4 HTTP/1.1" 200 None 2026-03-25 01:38:38,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:38:38,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:38:38,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:38:38,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09992ce0c9bc43bcce1c4 HTTP/1.1" 200 None 2026-03-25 01:38:38,709 - lava_callback - ERROR - Error processing callback for node 69c09992ce0c9bc43bcce1c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38872 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:39:37,451 - lava_callback - INFO - Received callback for node 69c337c718a3717aa65d2d57 INFO: 172.21.0.1:38880 - "POST /node/69c337c718a3717aa65d2d57 HTTP/1.0" 202 Accepted 2026-03-25 01:39:37,457 - lava_callback - INFO - Starting async job submit for node 69c337c718a3717aa65d2d57 2026-03-25 01:39:37,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:37,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337c718a3717aa65d2d57 HTTP/1.1" 200 None 2026-03-25 01:39:37,495 - lava_callback - DEBUG - Processing node 69c337c718a3717aa65d2d57: baseline-arm64-broonie 2026-03-25 01:39:37,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:39:37,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:39:37,960 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c718a3717aa65d2d57/log.txt.gz 2026-03-25 01:39:38,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:39:38,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:39:38,324 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c718a3717aa65d2d57/lava-logs.yaml 2026-03-25 01:39:38,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:39:38,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:39:38,480 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c718a3717aa65d2d57/lava_callback.json.gz 2026-03-25 01:39:38,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:38,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337c718a3717aa65d2d57 HTTP/1.1" 200 None 2026-03-25 01:39:38,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:38,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:39:38,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:38,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337c718a3717aa65d2d57 HTTP/1.1" 200 None 2026-03-25 01:39:38,620 - lava_callback - INFO - Completed processing callback for node 69c337c718a3717aa65d2d57 INFO: 172.21.0.1:38210 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:39:49,294 - lava_callback - INFO - Received callback for node 69c33ca618a3717aa65d2f9e INFO: 172.21.0.1:38222 - "POST /node/69c33ca618a3717aa65d2f9e HTTP/1.0" 202 Accepted 2026-03-25 01:39:49,300 - lava_callback - INFO - Starting async job submit for node 69c33ca618a3717aa65d2f9e 2026-03-25 01:39:49,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:49,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33ca618a3717aa65d2f9e HTTP/1.1" 200 None 2026-03-25 01:39:49,361 - lava_callback - DEBUG - Processing node 69c33ca618a3717aa65d2f9e: baseline-arm64 2026-03-25 01:39:49,494 - lava_callback - WARNING - Failed to upload log 2026-03-25 01:39:49,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:39:49,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:39:49,768 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33ca618a3717aa65d2f9e/lava-logs.yaml 2026-03-25 01:39:49,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:39:49,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:39:49,910 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33ca618a3717aa65d2f9e/lava_callback.json.gz 2026-03-25 01:39:49,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:49,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33ca618a3717aa65d2f9e HTTP/1.1" 200 None 2026-03-25 01:39:49,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:49,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:39:49,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:49,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33ca618a3717aa65d2f9e HTTP/1.1" 200 None 2026-03-25 01:39:50,063 - lava_callback - INFO - Completed processing callback for node 69c33ca618a3717aa65d2f9e 2026-03-25 01:39:51,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:51,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39318 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:39:58,660 - lava_callback - INFO - Received callback for node 69c099dcce0c9bc43bcce1d2 INFO: 172.21.0.1:39330 - "POST /node/69c099dcce0c9bc43bcce1d2 HTTP/1.0" 202 Accepted 2026-03-25 01:39:58,665 - lava_callback - INFO - Starting async job submit for node 69c099dcce0c9bc43bcce1d2 2026-03-25 01:39:58,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:58,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c099dcce0c9bc43bcce1d2 HTTP/1.1" 200 None 2026-03-25 01:39:58,686 - lava_callback - DEBUG - Processing node 69c099dcce0c9bc43bcce1d2: baseline-arm64-clabbe 2026-03-25 01:39:58,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c099dcce0c9bc43bcce1d2 2026-03-25 01:39:58,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:58,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c099dcce0c9bc43bcce1d2 HTTP/1.1" 200 None 2026-03-25 01:39:58,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:58,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:39:58,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:39:58,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c099dcce0c9bc43bcce1d2 HTTP/1.1" 200 None 2026-03-25 01:39:58,746 - lava_callback - ERROR - Error processing callback for node 69c099dcce0c9bc43bcce1d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53772 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:40:58,683 - lava_callback - INFO - Received callback for node 69c09a27ce0c9bc43bcce1dd INFO: 172.21.0.1:53788 - "POST /node/69c09a27ce0c9bc43bcce1dd HTTP/1.0" 202 Accepted 2026-03-25 01:40:58,688 - lava_callback - INFO - Starting async job submit for node 69c09a27ce0c9bc43bcce1dd 2026-03-25 01:40:58,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:40:58,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09a27ce0c9bc43bcce1dd HTTP/1.1" 200 None 2026-03-25 01:40:58,704 - lava_callback - DEBUG - Processing node 69c09a27ce0c9bc43bcce1dd: baseline-arm64-clabbe 2026-03-25 01:40:58,704 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09a27ce0c9bc43bcce1dd 2026-03-25 01:40:58,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:40:58,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09a27ce0c9bc43bcce1dd HTTP/1.1" 200 None 2026-03-25 01:40:58,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:40:58,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:40:58,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:40:58,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09a27ce0c9bc43bcce1dd HTTP/1.1" 200 None 2026-03-25 01:40:58,765 - lava_callback - ERROR - Error processing callback for node 69c09a27ce0c9bc43bcce1dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55540 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:42:18,800 - lava_callback - INFO - Received callback for node 69c09a73ce0c9bc43bcce1e2 INFO: 172.21.0.1:55552 - "POST /node/69c09a73ce0c9bc43bcce1e2 HTTP/1.0" 202 Accepted 2026-03-25 01:42:18,806 - lava_callback - INFO - Starting async job submit for node 69c09a73ce0c9bc43bcce1e2 2026-03-25 01:42:18,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:42:18,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09a73ce0c9bc43bcce1e2 HTTP/1.1" 200 None 2026-03-25 01:42:18,832 - lava_callback - DEBUG - Processing node 69c09a73ce0c9bc43bcce1e2: baseline-arm64-clabbe 2026-03-25 01:42:18,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09a73ce0c9bc43bcce1e2 2026-03-25 01:42:18,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:42:18,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09a73ce0c9bc43bcce1e2 HTTP/1.1" 200 None 2026-03-25 01:42:18,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:42:18,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:42:18,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:42:18,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09a73ce0c9bc43bcce1e2 HTTP/1.1" 200 None 2026-03-25 01:42:18,889 - lava_callback - ERROR - Error processing callback for node 69c09a73ce0c9bc43bcce1e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56936 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:43:38,882 - lava_callback - INFO - Received callback for node 69c09ac3ce0c9bc43bcce1eb INFO: 172.21.0.1:56952 - "POST /node/69c09ac3ce0c9bc43bcce1eb HTTP/1.0" 202 Accepted 2026-03-25 01:43:38,888 - lava_callback - INFO - Starting async job submit for node 69c09ac3ce0c9bc43bcce1eb 2026-03-25 01:43:38,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:43:38,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09ac3ce0c9bc43bcce1eb HTTP/1.1" 200 None 2026-03-25 01:43:38,904 - lava_callback - DEBUG - Processing node 69c09ac3ce0c9bc43bcce1eb: baseline-arm64-clabbe 2026-03-25 01:43:38,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09ac3ce0c9bc43bcce1eb 2026-03-25 01:43:38,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:43:38,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09ac3ce0c9bc43bcce1eb HTTP/1.1" 200 None 2026-03-25 01:43:38,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:43:38,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:43:38,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:43:38,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09ac3ce0c9bc43bcce1eb HTTP/1.1" 200 None 2026-03-25 01:43:38,967 - lava_callback - ERROR - Error processing callback for node 69c09ac3ce0c9bc43bcce1eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54074 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:44:18,645 - lava_callback - INFO - Received callback for node 69c33da418a3717aa65d2fcb INFO: 172.21.0.1:54080 - "POST /node/69c33da418a3717aa65d2fcb HTTP/1.0" 202 Accepted 2026-03-25 01:44:18,651 - lava_callback - INFO - Starting async job submit for node 69c33da418a3717aa65d2fcb 2026-03-25 01:44:18,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:44:18,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33da418a3717aa65d2fcb HTTP/1.1" 200 None 2026-03-25 01:44:18,694 - lava_callback - DEBUG - Processing node 69c33da418a3717aa65d2fcb: baseline-arm64 2026-03-25 01:44:19,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:44:19,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:44:19,173 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33da418a3717aa65d2fcb/log.txt.gz 2026-03-25 01:44:19,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:44:19,653 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:44:19,655 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33da418a3717aa65d2fcb/lava-logs.yaml 2026-03-25 01:44:19,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:44:19,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:44:19,843 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33da418a3717aa65d2fcb/lava_callback.json.gz 2026-03-25 01:44:19,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:44:19,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33da418a3717aa65d2fcb HTTP/1.1" 200 None 2026-03-25 01:44:19,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:44:19,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:44:19,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:44:19,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33da418a3717aa65d2fcb HTTP/1.1" 200 None 2026-03-25 01:44:20,008 - lava_callback - INFO - Completed processing callback for node 69c33da418a3717aa65d2fcb 2026-03-25 01:44:21,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:44:21,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39228 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:44:58,891 - lava_callback - INFO - Received callback for node 69c09b10ce0c9bc43bcce259 INFO: 172.21.0.1:39230 - "POST /node/69c09b10ce0c9bc43bcce259 HTTP/1.0" 202 Accepted 2026-03-25 01:44:58,897 - lava_callback - INFO - Starting async job submit for node 69c09b10ce0c9bc43bcce259 2026-03-25 01:44:58,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:44:58,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b10ce0c9bc43bcce259 HTTP/1.1" 200 None 2026-03-25 01:44:58,913 - lava_callback - DEBUG - Processing node 69c09b10ce0c9bc43bcce259: baseline-arm64-clabbe 2026-03-25 01:44:58,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b10ce0c9bc43bcce259 2026-03-25 01:44:58,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:44:58,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b10ce0c9bc43bcce259 HTTP/1.1" 200 None 2026-03-25 01:44:58,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:44:58,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 01:44:58,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:44:58,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b10ce0c9bc43bcce259 HTTP/1.1" 200 None 2026-03-25 01:44:58,974 - lava_callback - ERROR - Error processing callback for node 69c09b10ce0c9bc43bcce259: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52076 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:45:14,795 - lava_callback - INFO - Received callback for node 69c09b1fce0c9bc43bcce281 INFO: 172.21.0.1:53148 - "POST /node/69c09b1fce0c9bc43bcce281 HTTP/1.0" 202 Accepted 2026-03-25 01:45:14,800 - lava_callback - INFO - Starting async job submit for node 69c09b1fce0c9bc43bcce281 2026-03-25 01:45:14,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:14,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b1fce0c9bc43bcce281 HTTP/1.1" 200 None 2026-03-25 01:45:14,829 - lava_callback - DEBUG - Processing node 69c09b1fce0c9bc43bcce281: baseline-arm64-baylibre 2026-03-25 01:45:14,829 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b1fce0c9bc43bcce281 2026-03-25 01:45:14,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:14,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b1fce0c9bc43bcce281 HTTP/1.1" 200 None 2026-03-25 01:45:14,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:14,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:14,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:14,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b1fce0c9bc43bcce281 HTTP/1.1" 200 None 2026-03-25 01:45:14,881 - lava_callback - ERROR - Error processing callback for node 69c09b1fce0c9bc43bcce281: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53150 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 01:45:15,455 - lava_callback - INFO - Received callback for node 69c09b1ece0c9bc43bcce27a INFO: 172.21.0.1:53152 - "POST /node/69c09b1ece0c9bc43bcce27a HTTP/1.0" 202 Accepted 2026-03-25 01:45:15,461 - lava_callback - INFO - Starting async job submit for node 69c09b1ece0c9bc43bcce27a 2026-03-25 01:45:15,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:15,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b1ece0c9bc43bcce27a HTTP/1.1" 200 None 2026-03-25 01:45:15,481 - lava_callback - DEBUG - Processing node 69c09b1ece0c9bc43bcce27a: baseline-arm64-baylibre 2026-03-25 01:45:15,481 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b1ece0c9bc43bcce27a 2026-03-25 01:45:15,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:15,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b1ece0c9bc43bcce27a HTTP/1.1" 200 None 2026-03-25 01:45:15,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:15,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:15,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:15,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b1ece0c9bc43bcce27a HTTP/1.1" 200 None 2026-03-25 01:45:15,541 - lava_callback - ERROR - Error processing callback for node 69c09b1ece0c9bc43bcce27a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:45:16,097 - lava_callback - INFO - Received callback for node 69c09b21ce0c9bc43bcce284 INFO: 172.21.0.1:53166 - "POST /node/69c09b21ce0c9bc43bcce284 HTTP/1.0" 202 Accepted 2026-03-25 01:45:16,103 - lava_callback - INFO - Starting async job submit for node 69c09b21ce0c9bc43bcce284 2026-03-25 01:45:16,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:16,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b21ce0c9bc43bcce284 HTTP/1.1" 200 None 2026-03-25 01:45:16,122 - lava_callback - DEBUG - Processing node 69c09b21ce0c9bc43bcce284: baseline-arm64-baylibre 2026-03-25 01:45:16,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b21ce0c9bc43bcce284 2026-03-25 01:45:16,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:16,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b21ce0c9bc43bcce284 HTTP/1.1" 200 None 2026-03-25 01:45:16,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:16,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:16,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:16,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b21ce0c9bc43bcce284 HTTP/1.1" 200 None 2026-03-25 01:45:16,184 - lava_callback - ERROR - Error processing callback for node 69c09b21ce0c9bc43bcce284: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:45:16,753 - lava_callback - INFO - Received callback for node 69c09b23ce0c9bc43bcce28b INFO: 172.21.0.1:53182 - "POST /node/69c09b23ce0c9bc43bcce28b HTTP/1.0" 202 Accepted 2026-03-25 01:45:16,759 - lava_callback - INFO - Starting async job submit for node 69c09b23ce0c9bc43bcce28b 2026-03-25 01:45:16,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:16,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b23ce0c9bc43bcce28b HTTP/1.1" 200 None 2026-03-25 01:45:16,781 - lava_callback - DEBUG - Processing node 69c09b23ce0c9bc43bcce28b: baseline-arm64-baylibre 2026-03-25 01:45:16,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b23ce0c9bc43bcce28b 2026-03-25 01:45:16,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:16,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b23ce0c9bc43bcce28b HTTP/1.1" 200 None 2026-03-25 01:45:16,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:16,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:16,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:16,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b23ce0c9bc43bcce28b HTTP/1.1" 200 None 2026-03-25 01:45:16,834 - lava_callback - ERROR - Error processing callback for node 69c09b23ce0c9bc43bcce28b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:45:17,414 - lava_callback - INFO - Received callback for node 69c09b25ce0c9bc43bcce28e INFO: 172.21.0.1:53186 - "POST /node/69c09b25ce0c9bc43bcce28e HTTP/1.0" 202 Accepted 2026-03-25 01:45:17,419 - lava_callback - INFO - Starting async job submit for node 69c09b25ce0c9bc43bcce28e 2026-03-25 01:45:17,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:17,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b25ce0c9bc43bcce28e HTTP/1.1" 200 None 2026-03-25 01:45:17,436 - lava_callback - DEBUG - Processing node 69c09b25ce0c9bc43bcce28e: baseline-arm64-baylibre 2026-03-25 01:45:17,436 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b25ce0c9bc43bcce28e 2026-03-25 01:45:17,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:17,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b25ce0c9bc43bcce28e HTTP/1.1" 200 None 2026-03-25 01:45:17,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:17,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:17,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:17,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b25ce0c9bc43bcce28e HTTP/1.1" 200 None 2026-03-25 01:45:17,497 - lava_callback - ERROR - Error processing callback for node 69c09b25ce0c9bc43bcce28e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:45:18,059 - lava_callback - INFO - Received callback for node 69c09b26ce0c9bc43bcce291 INFO: 172.21.0.1:53188 - "POST /node/69c09b26ce0c9bc43bcce291 HTTP/1.0" 202 Accepted 2026-03-25 01:45:18,064 - lava_callback - INFO - Starting async job submit for node 69c09b26ce0c9bc43bcce291 2026-03-25 01:45:18,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:18,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b26ce0c9bc43bcce291 HTTP/1.1" 200 None 2026-03-25 01:45:18,080 - lava_callback - DEBUG - Processing node 69c09b26ce0c9bc43bcce291: baseline-arm64-baylibre 2026-03-25 01:45:18,080 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b26ce0c9bc43bcce291 2026-03-25 01:45:18,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:18,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b26ce0c9bc43bcce291 HTTP/1.1" 200 None 2026-03-25 01:45:18,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:18,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:18,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:18,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b26ce0c9bc43bcce291 HTTP/1.1" 200 None 2026-03-25 01:45:18,141 - lava_callback - ERROR - Error processing callback for node 69c09b26ce0c9bc43bcce291: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36526 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:45:38,902 - lava_callback - INFO - Received callback for node 69c09b36ce0c9bc43bcce2d2 INFO: 172.21.0.1:36538 - "POST /node/69c09b36ce0c9bc43bcce2d2 HTTP/1.0" 202 Accepted 2026-03-25 01:45:38,908 - lava_callback - INFO - Starting async job submit for node 69c09b36ce0c9bc43bcce2d2 2026-03-25 01:45:38,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:38,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b36ce0c9bc43bcce2d2 HTTP/1.1" 200 None 2026-03-25 01:45:38,924 - lava_callback - DEBUG - Processing node 69c09b36ce0c9bc43bcce2d2: baseline-arm64-clabbe 2026-03-25 01:45:38,924 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b36ce0c9bc43bcce2d2 2026-03-25 01:45:38,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:38,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b36ce0c9bc43bcce2d2 HTTP/1.1" 200 None 2026-03-25 01:45:38,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:38,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:38,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:38,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b36ce0c9bc43bcce2d2 HTTP/1.1" 200 None 2026-03-25 01:45:38,987 - lava_callback - ERROR - Error processing callback for node 69c09b36ce0c9bc43bcce2d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:45:39,541 - lava_callback - INFO - Received callback for node 69c09b38ce0c9bc43bcce2d5 INFO: 172.21.0.1:36540 - "POST /node/69c09b38ce0c9bc43bcce2d5 HTTP/1.0" 202 Accepted 2026-03-25 01:45:39,546 - lava_callback - INFO - Starting async job submit for node 69c09b38ce0c9bc43bcce2d5 2026-03-25 01:45:39,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:39,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b38ce0c9bc43bcce2d5 HTTP/1.1" 200 None 2026-03-25 01:45:39,562 - lava_callback - DEBUG - Processing node 69c09b38ce0c9bc43bcce2d5: baseline-arm64-clabbe 2026-03-25 01:45:39,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b38ce0c9bc43bcce2d5 2026-03-25 01:45:39,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:39,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b38ce0c9bc43bcce2d5 HTTP/1.1" 200 None 2026-03-25 01:45:39,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:39,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:39,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:39,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b38ce0c9bc43bcce2d5 HTTP/1.1" 200 None 2026-03-25 01:45:39,621 - lava_callback - ERROR - Error processing callback for node 69c09b38ce0c9bc43bcce2d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:45:40,176 - lava_callback - INFO - Received callback for node 69c09b39ce0c9bc43bcce2d8 INFO: 172.21.0.1:53342 - "POST /node/69c09b39ce0c9bc43bcce2d8 HTTP/1.0" 202 Accepted 2026-03-25 01:45:40,181 - lava_callback - INFO - Starting async job submit for node 69c09b39ce0c9bc43bcce2d8 2026-03-25 01:45:40,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:40,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b39ce0c9bc43bcce2d8 HTTP/1.1" 200 None 2026-03-25 01:45:40,198 - lava_callback - DEBUG - Processing node 69c09b39ce0c9bc43bcce2d8: baseline-arm64-clabbe 2026-03-25 01:45:40,199 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b39ce0c9bc43bcce2d8 2026-03-25 01:45:40,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:40,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b39ce0c9bc43bcce2d8 HTTP/1.1" 200 None 2026-03-25 01:45:40,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:40,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:40,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:40,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b39ce0c9bc43bcce2d8 HTTP/1.1" 200 None 2026-03-25 01:45:40,267 - lava_callback - ERROR - Error processing callback for node 69c09b39ce0c9bc43bcce2d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:45:40,821 - lava_callback - INFO - Received callback for node 69c09b39ce0c9bc43bcce2db INFO: 172.21.0.1:53350 - "POST /node/69c09b39ce0c9bc43bcce2db HTTP/1.0" 202 Accepted 2026-03-25 01:45:40,826 - lava_callback - INFO - Starting async job submit for node 69c09b39ce0c9bc43bcce2db 2026-03-25 01:45:40,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:40,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b39ce0c9bc43bcce2db HTTP/1.1" 200 None 2026-03-25 01:45:40,842 - lava_callback - DEBUG - Processing node 69c09b39ce0c9bc43bcce2db: baseline-arm64-clabbe 2026-03-25 01:45:40,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b39ce0c9bc43bcce2db 2026-03-25 01:45:40,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:40,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b39ce0c9bc43bcce2db HTTP/1.1" 200 None 2026-03-25 01:45:40,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:40,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:40,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:40,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b39ce0c9bc43bcce2db HTTP/1.1" 200 None 2026-03-25 01:45:40,894 - lava_callback - ERROR - Error processing callback for node 69c09b39ce0c9bc43bcce2db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:45:41,463 - lava_callback - INFO - Received callback for node 69c09b3bce0c9bc43bcce2e1 INFO: 172.21.0.1:53360 - "POST /node/69c09b3bce0c9bc43bcce2e1 HTTP/1.0" 202 Accepted 2026-03-25 01:45:41,468 - lava_callback - INFO - Starting async job submit for node 69c09b3bce0c9bc43bcce2e1 2026-03-25 01:45:41,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:41,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b3bce0c9bc43bcce2e1 HTTP/1.1" 200 None 2026-03-25 01:45:41,487 - lava_callback - DEBUG - Processing node 69c09b3bce0c9bc43bcce2e1: baseline-arm64-clabbe 2026-03-25 01:45:41,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b3bce0c9bc43bcce2e1 2026-03-25 01:45:41,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:41,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b3bce0c9bc43bcce2e1 HTTP/1.1" 200 None 2026-03-25 01:45:41,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:41,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:41,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:41,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b3bce0c9bc43bcce2e1 HTTP/1.1" 200 None 2026-03-25 01:45:41,543 - lava_callback - ERROR - Error processing callback for node 69c09b3bce0c9bc43bcce2e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-25 01:45:42,090 - lava_callback - INFO - Received callback for node 69c09b3cce0c9bc43bcce2e4 INFO: 172.21.0.1:53366 - "POST /node/69c09b3cce0c9bc43bcce2e4 HTTP/1.0" 202 Accepted 2026-03-25 01:45:42,096 - lava_callback - INFO - Starting async job submit for node 69c09b3cce0c9bc43bcce2e4 2026-03-25 01:45:42,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:42,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b3cce0c9bc43bcce2e4 HTTP/1.1" 200 None 2026-03-25 01:45:42,111 - lava_callback - DEBUG - Processing node 69c09b3cce0c9bc43bcce2e4: baseline-arm64-clabbe 2026-03-25 01:45:42,111 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09b3cce0c9bc43bcce2e4 2026-03-25 01:45:42,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:42,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09b3cce0c9bc43bcce2e4 HTTP/1.1" 200 None 2026-03-25 01:45:42,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:42,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-25 01:45:42,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:45:42,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09b3cce0c9bc43bcce2e4 HTTP/1.1" 200 None 2026-03-25 01:45:42,172 - lava_callback - ERROR - Error processing callback for node 69c09b3cce0c9bc43bcce2e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41896 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:46:28,773 - lava_callback - INFO - Received callback for node 69c33e6b18a3717aa65d300e INFO: 172.21.0.1:41908 - "POST /node/69c33e6b18a3717aa65d300e HTTP/1.0" 202 Accepted 2026-03-25 01:46:28,780 - lava_callback - INFO - Starting async job submit for node 69c33e6b18a3717aa65d300e 2026-03-25 01:46:28,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:46:28,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33e6b18a3717aa65d300e HTTP/1.1" 200 None 2026-03-25 01:46:28,796 - lava_callback - DEBUG - Processing node 69c33e6b18a3717aa65d300e: baseline-nfs-arm64 2026-03-25 01:46:28,798 - lava_callback - WARNING - Failed to upload log 2026-03-25 01:46:28,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:46:28,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:46:28,953 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c33e6b18a3717aa65d300e/lava-logs.yaml 2026-03-25 01:46:28,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:46:29,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:46:29,308 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c33e6b18a3717aa65d300e/lava_callback.json.gz 2026-03-25 01:46:29,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:46:29,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33e6b18a3717aa65d300e HTTP/1.1" 200 None 2026-03-25 01:46:29,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:46:29,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:46:29,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:46:29,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33e6b18a3717aa65d300e HTTP/1.1" 200 None 2026-03-25 01:46:29,461 - lava_callback - INFO - Completed processing callback for node 69c33e6b18a3717aa65d300e INFO: 172.21.0.1:53406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47548 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:46:51,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:46:51,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34790 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:55:24,620 - lava_callback - INFO - Received callback for node 69c3401718a3717aa65d303f INFO: 172.21.0.1:33332 - "POST /node/69c3401718a3717aa65d303f HTTP/1.0" 202 Accepted 2026-03-25 01:55:24,627 - lava_callback - INFO - Starting async job submit for node 69c3401718a3717aa65d303f 2026-03-25 01:55:24,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:55:24,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3401718a3717aa65d303f HTTP/1.1" 200 None 2026-03-25 01:55:24,674 - lava_callback - DEBUG - Processing node 69c3401718a3717aa65d303f: baseline-arm64 2026-03-25 01:55:24,873 - lava_callback - WARNING - Failed to upload log 2026-03-25 01:55:24,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:55:25,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:55:25,202 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3401718a3717aa65d303f/lava-logs.yaml 2026-03-25 01:55:25,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:55:25,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:55:25,374 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3401718a3717aa65d303f/lava_callback.json.gz 2026-03-25 01:55:25,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:55:25,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3401718a3717aa65d303f HTTP/1.1" 200 None 2026-03-25 01:55:25,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:55:25,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:55:25,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:55:25,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3401718a3717aa65d303f HTTP/1.1" 200 None 2026-03-25 01:55:25,515 - lava_callback - INFO - Completed processing callback for node 69c3401718a3717aa65d303f INFO: 172.21.0.1:33342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57554 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:55:51,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:55:51,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43694 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:56:18,511 - lava_callback - INFO - Received callback for node 69c337ab18a3717aa65d2d00 INFO: 172.21.0.1:43704 - "POST /node/69c337ab18a3717aa65d2d00 HTTP/1.0" 202 Accepted 2026-03-25 01:56:18,519 - lava_callback - INFO - Starting async job submit for node 69c337ab18a3717aa65d2d00 2026-03-25 01:56:18,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:56:18,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337ab18a3717aa65d2d00 HTTP/1.1" 200 None 2026-03-25 01:56:18,578 - lava_callback - DEBUG - Processing node 69c337ab18a3717aa65d2d00: baseline-nfs-arm64 2026-03-25 01:56:19,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:56:19,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:56:19,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337ab18a3717aa65d2d00/log.txt.gz 2026-03-25 01:56:20,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:56:20,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:56:20,892 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337ab18a3717aa65d2d00/lava-logs.yaml 2026-03-25 01:56:20,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:56:21,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:56:21,100 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337ab18a3717aa65d2d00/lava_callback.json.gz DEBUG: Setup failed for 69c337ab18a3717aa65d2d00. Transitting job node result: incomplete -> fail 2026-03-25 01:56:21,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:56:21,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337ab18a3717aa65d2d00 HTTP/1.1" 200 None 2026-03-25 01:56:21,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:56:21,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:56:21,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:56:21,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337ab18a3717aa65d2d00 HTTP/1.1" 200 None 2026-03-25 01:56:21,357 - lava_callback - INFO - Completed processing callback for node 69c337ab18a3717aa65d2d00 INFO: 172.21.0.1:51862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52138 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:56:51,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:56:51,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36242 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 01:57:15,355 - lava_callback - INFO - Received callback for node 69c09df3ce0c9bc43bcce4a3 INFO: 172.21.0.1:36246 - "POST /node/69c09df3ce0c9bc43bcce4a3 HTTP/1.0" 202 Accepted 2026-03-25 01:57:15,361 - lava_callback - INFO - Starting async job submit for node 69c09df3ce0c9bc43bcce4a3 2026-03-25 01:57:15,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:57:15,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09df3ce0c9bc43bcce4a3 HTTP/1.1" 200 None 2026-03-25 01:57:15,409 - lava_callback - DEBUG - Processing node 69c09df3ce0c9bc43bcce4a3: baseline-arm64-baylibre 2026-03-25 01:57:15,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09df3ce0c9bc43bcce4a3 2026-03-25 01:57:15,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:57:15,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09df3ce0c9bc43bcce4a3 HTTP/1.1" 200 None 2026-03-25 01:57:15,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:57:15,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 01:57:15,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:57:15,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09df3ce0c9bc43bcce4a3 HTTP/1.1" 200 None 2026-03-25 01:57:15,491 - lava_callback - ERROR - Error processing callback for node 69c09df3ce0c9bc43bcce4a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56428 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:58:28,272 - lava_callback - INFO - Received callback for node 69c33f7c18a3717aa65d3030 INFO: 172.21.0.1:56436 - "POST /node/69c33f7c18a3717aa65d3030 HTTP/1.0" 202 Accepted 2026-03-25 01:58:28,280 - lava_callback - INFO - Starting async job submit for node 69c33f7c18a3717aa65d3030 2026-03-25 01:58:28,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:58:28,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33f7c18a3717aa65d3030 HTTP/1.1" 200 None 2026-03-25 01:58:28,333 - lava_callback - DEBUG - Processing node 69c33f7c18a3717aa65d3030: baseline-arm64 2026-03-25 01:58:28,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:58:28,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:58:28,871 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33f7c18a3717aa65d3030/log.txt.gz 2026-03-25 01:58:29,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:58:29,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:58:29,335 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33f7c18a3717aa65d3030/lava-logs.yaml 2026-03-25 01:58:29,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 01:58:29,499 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 01:58:29,500 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c33f7c18a3717aa65d3030/lava_callback.json.gz DEBUG: Setup failed for 69c33f7c18a3717aa65d3030. Transitting job node result: incomplete -> fail 2026-03-25 01:58:29,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:58:29,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33f7c18a3717aa65d3030 HTTP/1.1" 200 None 2026-03-25 01:58:29,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:58:29,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 01:58:29,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:58:29,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33f7c18a3717aa65d3030 HTTP/1.1" 200 None 2026-03-25 01:58:29,728 - lava_callback - INFO - Completed processing callback for node 69c33f7c18a3717aa65d3030 INFO: 172.21.0.1:50928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49644 - "GET / HTTP/1.0" 200 OK 2026-03-25 01:58:51,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 01:58:51,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35822 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:01:13,408 - lava_callback - INFO - Received callback for node 69c341a918a3717aa65d307a INFO: 172.21.0.1:51160 - "POST /node/69c341a918a3717aa65d307a HTTP/1.0" 202 Accepted 2026-03-25 02:01:13,415 - lava_callback - INFO - Starting async job submit for node 69c341a918a3717aa65d307a 2026-03-25 02:01:13,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:01:13,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c341a918a3717aa65d307a HTTP/1.1" 200 None 2026-03-25 02:01:13,459 - lava_callback - DEBUG - Processing node 69c341a918a3717aa65d307a: baseline-arm64 2026-03-25 02:01:13,591 - lava_callback - WARNING - Failed to upload log 2026-03-25 02:01:13,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:01:13,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:01:13,879 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c341a918a3717aa65d307a/lava-logs.yaml 2026-03-25 02:01:13,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:01:14,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:01:14,023 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c341a918a3717aa65d307a/lava_callback.json.gz 2026-03-25 02:01:14,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:01:14,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c341a918a3717aa65d307a HTTP/1.1" 200 None 2026-03-25 02:01:14,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:01:14,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 02:01:14,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:01:14,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c341a918a3717aa65d307a HTTP/1.1" 200 None 2026-03-25 02:01:14,190 - lava_callback - INFO - Completed processing callback for node 69c341a918a3717aa65d307a INFO: 172.21.0.1:51162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51176 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:01:21,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:01:21,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42774 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 02:02:15,648 - lava_callback - INFO - Received callback for node 69c09f23ce0c9bc43bcce4d0 INFO: 172.21.0.1:42784 - "POST /node/69c09f23ce0c9bc43bcce4d0 HTTP/1.0" 202 Accepted 2026-03-25 02:02:15,656 - lava_callback - INFO - Starting async job submit for node 69c09f23ce0c9bc43bcce4d0 2026-03-25 02:02:15,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:02:15,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09f23ce0c9bc43bcce4d0 HTTP/1.1" 200 None 2026-03-25 02:02:15,675 - lava_callback - DEBUG - Processing node 69c09f23ce0c9bc43bcce4d0: baseline-arm64-baylibre 2026-03-25 02:02:15,675 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c09f23ce0c9bc43bcce4d0 2026-03-25 02:02:15,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:02:15,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c09f23ce0c9bc43bcce4d0 HTTP/1.1" 200 None 2026-03-25 02:02:15,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:02:15,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:02:15,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:02:15,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c09f23ce0c9bc43bcce4d0 HTTP/1.1" 200 None 2026-03-25 02:02:15,733 - lava_callback - ERROR - Error processing callback for node 69c09f23ce0c9bc43bcce4d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42786 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:02:24,602 - lava_callback - INFO - Received callback for node 69c33def18a3717aa65d2fd6 INFO: 172.21.0.1:40436 - "POST /node/69c33def18a3717aa65d2fd6 HTTP/1.0" 202 Accepted 2026-03-25 02:02:24,609 - lava_callback - INFO - Starting async job submit for node 69c33def18a3717aa65d2fd6 2026-03-25 02:02:24,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:02:24,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33def18a3717aa65d2fd6 HTTP/1.1" 200 None 2026-03-25 02:02:24,678 - lava_callback - DEBUG - Processing node 69c33def18a3717aa65d2fd6: baseline-arm 2026-03-25 02:02:25,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:02:25,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:02:25,452 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c33def18a3717aa65d2fd6/log.txt.gz 2026-03-25 02:02:25,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:02:26,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:02:26,060 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c33def18a3717aa65d2fd6/lava-logs.yaml 2026-03-25 02:02:26,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:02:26,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:02:26,252 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c33def18a3717aa65d2fd6/lava_callback.json.gz DEBUG: Setup failed for 69c33def18a3717aa65d2fd6. Transitting job node result: incomplete -> fail 2026-03-25 02:02:26,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:02:26,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33def18a3717aa65d2fd6 HTTP/1.1" 200 None 2026-03-25 02:02:26,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:02:26,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 02:02:26,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:02:26,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33def18a3717aa65d2fd6 HTTP/1.1" 200 None INFO: 172.21.0.1:40438 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:02:26,549 - lava_callback - INFO - Completed processing callback for node 69c33def18a3717aa65d2fd6 INFO: 172.21.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42658 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:02:51,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:02:51,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39198 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:04:28,098 - lava_callback - INFO - Received callback for node 69c3379818a3717aa65d2cba INFO: 172.21.0.1:39210 - "POST /node/69c3379818a3717aa65d2cba HTTP/1.0" 202 Accepted 2026-03-25 02:04:28,105 - lava_callback - INFO - Starting async job submit for node 69c3379818a3717aa65d2cba 2026-03-25 02:04:28,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:04:28,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379818a3717aa65d2cba HTTP/1.1" 200 None 2026-03-25 02:04:28,142 - lava_callback - DEBUG - Processing node 69c3379818a3717aa65d2cba: baseline-arm64 2026-03-25 02:04:28,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:04:28,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:04:28,798 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379818a3717aa65d2cba/log.txt.gz 2026-03-25 02:04:29,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:04:29,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:04:29,276 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379818a3717aa65d2cba/lava-logs.yaml 2026-03-25 02:04:29,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:04:29,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:04:29,439 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3379818a3717aa65d2cba/lava_callback.json.gz 2026-03-25 02:04:29,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:04:29,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3379818a3717aa65d2cba HTTP/1.1" 200 None 2026-03-25 02:04:29,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:04:29,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 02:04:29,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:04:29,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3379818a3717aa65d2cba HTTP/1.1" 200 None 2026-03-25 02:04:29,533 - lava_callback - INFO - Completed processing callback for node 69c3379818a3717aa65d2cba INFO: 172.21.0.1:56272 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:04:32,664 - lava_callback - INFO - Received callback for node 69c337a818a3717aa65d2cfa INFO: 172.21.0.1:56274 - "POST /node/69c337a818a3717aa65d2cfa HTTP/1.0" 202 Accepted 2026-03-25 02:04:32,671 - lava_callback - INFO - Starting async job submit for node 69c337a818a3717aa65d2cfa 2026-03-25 02:04:32,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:04:32,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a818a3717aa65d2cfa HTTP/1.1" 200 None 2026-03-25 02:04:32,705 - lava_callback - DEBUG - Processing node 69c337a818a3717aa65d2cfa: baseline-nfs-arm64 2026-03-25 02:04:33,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:04:33,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:04:33,277 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337a818a3717aa65d2cfa/log.txt.gz 2026-03-25 02:04:33,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:04:33,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:04:33,756 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337a818a3717aa65d2cfa/lava-logs.yaml 2026-03-25 02:04:33,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:04:33,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:04:33,920 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c337a818a3717aa65d2cfa/lava_callback.json.gz 2026-03-25 02:04:33,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:04:33,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337a818a3717aa65d2cfa HTTP/1.1" 200 None 2026-03-25 02:04:33,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:04:33,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 02:04:33,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:04:34,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337a818a3717aa65d2cfa HTTP/1.1" 200 None 2026-03-25 02:04:34,010 - lava_callback - INFO - Completed processing callback for node 69c337a818a3717aa65d2cfa INFO: 172.21.0.1:56282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56340 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:04:51,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:04:51,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37296 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:07:35,885 - lava_callback - INFO - Received callback for node 69c0a052ce0c9bc43bcce51a INFO: 172.21.0.1:37304 - "POST /node/69c0a052ce0c9bc43bcce51a HTTP/1.0" 202 Accepted 2026-03-25 02:07:35,890 - lava_callback - INFO - Starting async job submit for node 69c0a052ce0c9bc43bcce51a 2026-03-25 02:07:35,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:07:35,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a052ce0c9bc43bcce51a HTTP/1.1" 200 None 2026-03-25 02:07:35,906 - lava_callback - DEBUG - Processing node 69c0a052ce0c9bc43bcce51a: baseline-arm64-baylibre 2026-03-25 02:07:35,907 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0a052ce0c9bc43bcce51a 2026-03-25 02:07:35,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:07:35,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a052ce0c9bc43bcce51a HTTP/1.1" 200 None 2026-03-25 02:07:35,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:07:35,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:07:35,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:07:35,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0a052ce0c9bc43bcce51a HTTP/1.1" 200 None 2026-03-25 02:07:35,967 - lava_callback - ERROR - Error processing callback for node 69c0a052ce0c9bc43bcce51a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56612 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:08:06,654 - lava_callback - INFO - Received callback for node 69c3434718a3717aa65d30bb INFO: 172.21.0.1:56626 - "POST /node/69c3434718a3717aa65d30bb HTTP/1.0" 202 Accepted 2026-03-25 02:08:06,661 - lava_callback - INFO - Starting async job submit for node 69c3434718a3717aa65d30bb 2026-03-25 02:08:06,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:08:06,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3434718a3717aa65d30bb HTTP/1.1" 200 None 2026-03-25 02:08:06,701 - lava_callback - DEBUG - Processing node 69c3434718a3717aa65d30bb: baseline-arm64 2026-03-25 02:08:07,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:08:07,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:08:07,149 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3434718a3717aa65d30bb/log.txt.gz 2026-03-25 02:08:07,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:08:07,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:08:07,531 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3434718a3717aa65d30bb/lava-logs.yaml 2026-03-25 02:08:07,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:08:07,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:08:07,744 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3434718a3717aa65d30bb/lava_callback.json.gz 2026-03-25 02:08:07,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:08:07,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3434718a3717aa65d30bb HTTP/1.1" 200 None 2026-03-25 02:08:07,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:08:07,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 02:08:07,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:08:07,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3434718a3717aa65d30bb HTTP/1.1" 200 None 2026-03-25 02:08:07,832 - lava_callback - INFO - Completed processing callback for node 69c3434718a3717aa65d30bb INFO: 172.21.0.1:48136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48140 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:08:21,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:08:21,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-25 02:08:24,238 - lava_callback - INFO - Received callback for node 69c33f0c18a3717aa65d301e INFO: 172.21.0.1:55210 - "POST /node/69c33f0c18a3717aa65d301e HTTP/1.0" 202 Accepted 2026-03-25 02:08:24,285 - lava_callback - INFO - Starting async job submit for node 69c33f0c18a3717aa65d301e 2026-03-25 02:08:24,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:08:24,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33f0c18a3717aa65d301e HTTP/1.1" 200 None 2026-03-25 02:08:24,336 - lava_callback - DEBUG - Processing node 69c33f0c18a3717aa65d301e: baseline-arm-pengutronix 2026-03-25 02:08:24,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:08:24,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:08:24,582 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c33f0c18a3717aa65d301e/log.txt.gz 2026-03-25 02:08:24,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:08:24,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:08:24,805 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c33f0c18a3717aa65d301e/lava-logs.yaml 2026-03-25 02:08:24,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:08:24,953 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:08:24,954 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c33f0c18a3717aa65d301e/lava_callback.json.gz DEBUG: Setup failed for 69c33f0c18a3717aa65d301e. Transitting job node result: incomplete -> fail 2026-03-25 02:08:25,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:08:25,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c33f0c18a3717aa65d301e HTTP/1.1" 200 None 2026-03-25 02:08:25,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:08:25,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 02:08:25,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:08:25,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c33f0c18a3717aa65d301e HTTP/1.1" 200 None 2026-03-25 02:08:25,117 - lava_callback - INFO - Completed processing callback for node 69c33f0c18a3717aa65d301e INFO: 172.21.0.1:55220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44716 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:08:51,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:08:51,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44234 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 02:11:16,182 - lava_callback - INFO - Received callback for node 69c0a132ce0c9bc43bcce53d INFO: 172.21.0.1:44248 - "POST /node/69c0a132ce0c9bc43bcce53d HTTP/1.0" 202 Accepted 2026-03-25 02:11:16,188 - lava_callback - INFO - Starting async job submit for node 69c0a132ce0c9bc43bcce53d 2026-03-25 02:11:16,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:11:16,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a132ce0c9bc43bcce53d HTTP/1.1" 200 None 2026-03-25 02:11:16,204 - lava_callback - DEBUG - Processing node 69c0a132ce0c9bc43bcce53d: baseline-arm64-baylibre 2026-03-25 02:11:16,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0a132ce0c9bc43bcce53d 2026-03-25 02:11:16,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:11:16,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a132ce0c9bc43bcce53d HTTP/1.1" 200 None 2026-03-25 02:11:16,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:11:16,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:11:16,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:11:16,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0a132ce0c9bc43bcce53d HTTP/1.1" 200 None 2026-03-25 02:11:16,258 - lava_callback - ERROR - Error processing callback for node 69c0a132ce0c9bc43bcce53d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49640 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:12:36,070 - lava_callback - INFO - Received callback for node 69c0a17ece0c9bc43bcce547 INFO: 172.21.0.1:49656 - "POST /node/69c0a17ece0c9bc43bcce547 HTTP/1.0" 202 Accepted 2026-03-25 02:12:36,076 - lava_callback - INFO - Starting async job submit for node 69c0a17ece0c9bc43bcce547 2026-03-25 02:12:36,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:12:36,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a17ece0c9bc43bcce547 HTTP/1.1" 200 None 2026-03-25 02:12:36,092 - lava_callback - DEBUG - Processing node 69c0a17ece0c9bc43bcce547: baseline-arm64-baylibre 2026-03-25 02:12:36,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0a17ece0c9bc43bcce547 2026-03-25 02:12:36,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:12:36,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a17ece0c9bc43bcce547 HTTP/1.1" 200 None 2026-03-25 02:12:36,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:12:36,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:12:36,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:12:36,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0a17ece0c9bc43bcce547 HTTP/1.1" 200 None 2026-03-25 02:12:36,165 - lava_callback - ERROR - Error processing callback for node 69c0a17ece0c9bc43bcce547: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39832 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:13:36,148 - lava_callback - INFO - Received callback for node 69c0a1cace0c9bc43bcce553 INFO: 172.21.0.1:39838 - "POST /node/69c0a1cace0c9bc43bcce553 HTTP/1.0" 202 Accepted 2026-03-25 02:13:36,154 - lava_callback - INFO - Starting async job submit for node 69c0a1cace0c9bc43bcce553 2026-03-25 02:13:36,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:13:36,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a1cace0c9bc43bcce553 HTTP/1.1" 200 None 2026-03-25 02:13:36,170 - lava_callback - DEBUG - Processing node 69c0a1cace0c9bc43bcce553: baseline-arm64-baylibre 2026-03-25 02:13:36,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0a1cace0c9bc43bcce553 2026-03-25 02:13:36,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:13:36,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a1cace0c9bc43bcce553 HTTP/1.1" 200 None 2026-03-25 02:13:36,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:13:36,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:13:36,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:13:36,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0a1cace0c9bc43bcce553 HTTP/1.1" 200 None 2026-03-25 02:13:36,242 - lava_callback - ERROR - Error processing callback for node 69c0a1cace0c9bc43bcce553: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34006 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:15:07,693 - lava_callback - INFO - Received callback for node 69c3410518a3717aa65d3061 INFO: 172.21.0.1:34018 - "POST /node/69c3410518a3717aa65d3061 HTTP/1.0" 202 Accepted 2026-03-25 02:15:07,700 - lava_callback - INFO - Starting async job submit for node 69c3410518a3717aa65d3061 2026-03-25 02:15:07,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:15:07,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3410518a3717aa65d3061 HTTP/1.1" 200 None 2026-03-25 02:15:07,795 - lava_callback - DEBUG - Processing node 69c3410518a3717aa65d3061: ltp-smoke 2026-03-25 02:15:08,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:15:08,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:15:08,447 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c3410518a3717aa65d3061/log.txt.gz 2026-03-25 02:15:08,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:15:08,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:15:08,978 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c3410518a3717aa65d3061/lava-logs.yaml 2026-03-25 02:15:08,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:15:09,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:15:09,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c3410518a3717aa65d3061/lava_callback.json.gz 2026-03-25 02:15:09,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:15:09,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3410518a3717aa65d3061 HTTP/1.1" 200 None 2026-03-25 02:15:09,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:15:09,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3294718a3717aa65d01b3 HTTP/1.1" 200 None 2026-03-25 02:15:09,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:15:09,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3410518a3717aa65d3061 HTTP/1.1" 200 None 2026-03-25 02:15:09,468 - lava_callback - INFO - Completed processing callback for node 69c3410518a3717aa65d3061 INFO: 172.21.0.1:39980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39988 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:15:21,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:15:21,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49056 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:16:20,218 - lava_callback - INFO - Received callback for node 69c0a267ce0c9bc43bcce567 INFO: 172.21.0.1:48692 - "POST /node/69c0a267ce0c9bc43bcce567 HTTP/1.0" 202 Accepted 2026-03-25 02:16:20,224 - lava_callback - INFO - Starting async job submit for node 69c0a267ce0c9bc43bcce567 2026-03-25 02:16:20,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:16:20,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a267ce0c9bc43bcce567 HTTP/1.1" 200 None 2026-03-25 02:16:20,240 - lava_callback - DEBUG - Processing node 69c0a267ce0c9bc43bcce567: baseline-arm64-clabbe 2026-03-25 02:16:20,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0a267ce0c9bc43bcce567 2026-03-25 02:16:20,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:16:20,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a267ce0c9bc43bcce567 HTTP/1.1" 200 None 2026-03-25 02:16:20,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:16:20,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:16:20,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:16:20,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0a267ce0c9bc43bcce567 HTTP/1.1" 200 None 2026-03-25 02:16:20,302 - lava_callback - ERROR - Error processing callback for node 69c0a267ce0c9bc43bcce567: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57228 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:21:20,304 - lava_callback - INFO - Received callback for node 69c0a39ace0c9bc43bcce589 INFO: 172.21.0.1:35714 - "POST /node/69c0a39ace0c9bc43bcce589 HTTP/1.0" 202 Accepted 2026-03-25 02:21:20,310 - lava_callback - INFO - Starting async job submit for node 69c0a39ace0c9bc43bcce589 2026-03-25 02:21:20,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:21:20,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a39ace0c9bc43bcce589 HTTP/1.1" 200 None 2026-03-25 02:21:20,326 - lava_callback - DEBUG - Processing node 69c0a39ace0c9bc43bcce589: baseline-arm64-clabbe 2026-03-25 02:21:20,326 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0a39ace0c9bc43bcce589 2026-03-25 02:21:20,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:21:20,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a39ace0c9bc43bcce589 HTTP/1.1" 200 None 2026-03-25 02:21:20,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:21:20,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:21:20,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:21:20,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0a39ace0c9bc43bcce589 HTTP/1.1" 200 None 2026-03-25 02:21:20,382 - lava_callback - ERROR - Error processing callback for node 69c0a39ace0c9bc43bcce589: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48640 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:24:00,412 - lava_callback - INFO - Received callback for node 69c0a433ce0c9bc43bcce5a5 INFO: 172.21.0.1:50140 - "POST /node/69c0a433ce0c9bc43bcce5a5 HTTP/1.0" 202 Accepted 2026-03-25 02:24:00,418 - lava_callback - INFO - Starting async job submit for node 69c0a433ce0c9bc43bcce5a5 2026-03-25 02:24:00,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:24:00,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a433ce0c9bc43bcce5a5 HTTP/1.1" 200 None 2026-03-25 02:24:00,461 - lava_callback - DEBUG - Processing node 69c0a433ce0c9bc43bcce5a5: baseline-arm64-clabbe 2026-03-25 02:24:00,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0a433ce0c9bc43bcce5a5 2026-03-25 02:24:00,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:24:00,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a433ce0c9bc43bcce5a5 HTTP/1.1" 200 None 2026-03-25 02:24:00,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:24:00,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:24:00,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:24:00,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0a433ce0c9bc43bcce5a5 HTTP/1.1" 200 None 2026-03-25 02:24:00,529 - lava_callback - ERROR - Error processing callback for node 69c0a433ce0c9bc43bcce5a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39672 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:26:40,498 - lava_callback - INFO - Received callback for node 69c0a4d2ce0c9bc43bcce5b5 INFO: 172.21.0.1:40224 - "POST /node/69c0a4d2ce0c9bc43bcce5b5 HTTP/1.0" 202 Accepted 2026-03-25 02:26:40,505 - lava_callback - INFO - Starting async job submit for node 69c0a4d2ce0c9bc43bcce5b5 2026-03-25 02:26:40,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:26:40,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a4d2ce0c9bc43bcce5b5 HTTP/1.1" 200 None 2026-03-25 02:26:40,520 - lava_callback - DEBUG - Processing node 69c0a4d2ce0c9bc43bcce5b5: baseline-arm64-clabbe 2026-03-25 02:26:40,520 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0a4d2ce0c9bc43bcce5b5 2026-03-25 02:26:40,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:26:40,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a4d2ce0c9bc43bcce5b5 HTTP/1.1" 200 None 2026-03-25 02:26:40,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:26:40,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:26:40,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:26:40,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0a4d2ce0c9bc43bcce5b5 HTTP/1.1" 200 None 2026-03-25 02:26:40,586 - lava_callback - ERROR - Error processing callback for node 69c0a4d2ce0c9bc43bcce5b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54296 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:27:00,239 - lava_callback - INFO - Received callback for node 69c347b418a3717aa65d3146 INFO: 172.21.0.1:44450 - "POST /node/69c347b418a3717aa65d3146 HTTP/1.0" 202 Accepted 2026-03-25 02:27:00,246 - lava_callback - INFO - Starting async job submit for node 69c347b418a3717aa65d3146 2026-03-25 02:27:00,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:27:00,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c347b418a3717aa65d3146 HTTP/1.1" 200 None 2026-03-25 02:27:00,316 - lava_callback - DEBUG - Processing node 69c347b418a3717aa65d3146: baseline-arm64 2026-03-25 02:27:00,517 - lava_callback - WARNING - Failed to upload log 2026-03-25 02:27:00,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:27:00,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:27:00,827 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c347b418a3717aa65d3146/lava-logs.yaml 2026-03-25 02:27:00,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:27:00,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:27:00,982 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c347b418a3717aa65d3146/lava_callback.json.gz 2026-03-25 02:27:01,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:27:01,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c347b418a3717aa65d3146 HTTP/1.1" 200 None 2026-03-25 02:27:01,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:27:01,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 02:27:01,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:27:01,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c347b418a3717aa65d3146 HTTP/1.1" 200 None 2026-03-25 02:27:01,145 - lava_callback - INFO - Completed processing callback for node 69c347b418a3717aa65d3146 INFO: 172.21.0.1:44454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49516 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:27:21,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:27:21,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33322 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:28:00,595 - lava_callback - INFO - Received callback for node 69c0a520ce0c9bc43bcce5ba INFO: 172.21.0.1:53950 - "POST /node/69c0a520ce0c9bc43bcce5ba HTTP/1.0" 202 Accepted 2026-03-25 02:28:00,601 - lava_callback - INFO - Starting async job submit for node 69c0a520ce0c9bc43bcce5ba 2026-03-25 02:28:00,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:28:00,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a520ce0c9bc43bcce5ba HTTP/1.1" 200 None 2026-03-25 02:28:00,620 - lava_callback - DEBUG - Processing node 69c0a520ce0c9bc43bcce5ba: baseline-arm64-clabbe 2026-03-25 02:28:00,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0a520ce0c9bc43bcce5ba 2026-03-25 02:28:00,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:28:00,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a520ce0c9bc43bcce5ba HTTP/1.1" 200 None 2026-03-25 02:28:00,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:28:00,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:28:00,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:28:00,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0a520ce0c9bc43bcce5ba HTTP/1.1" 200 None 2026-03-25 02:28:00,679 - lava_callback - ERROR - Error processing callback for node 69c0a520ce0c9bc43bcce5ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60374 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:29:20,618 - lava_callback - INFO - Received callback for node 69c0a56cce0c9bc43bcce5c4 INFO: 172.21.0.1:51478 - "POST /node/69c0a56cce0c9bc43bcce5c4 HTTP/1.0" 202 Accepted 2026-03-25 02:29:20,624 - lava_callback - INFO - Starting async job submit for node 69c0a56cce0c9bc43bcce5c4 2026-03-25 02:29:20,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:29:20,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a56cce0c9bc43bcce5c4 HTTP/1.1" 200 None 2026-03-25 02:29:20,640 - lava_callback - DEBUG - Processing node 69c0a56cce0c9bc43bcce5c4: baseline-arm64-clabbe 2026-03-25 02:29:20,640 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0a56cce0c9bc43bcce5c4 2026-03-25 02:29:20,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:29:20,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0a56cce0c9bc43bcce5c4 HTTP/1.1" 200 None 2026-03-25 02:29:20,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:29:20,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b89db8fbe29c53ae85ccf4 HTTP/1.1" 200 None 2026-03-25 02:29:20,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:29:20,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0a56cce0c9bc43bcce5c4 HTTP/1.1" 200 None 2026-03-25 02:29:20,710 - lava_callback - ERROR - Error processing callback for node 69c0a56cce0c9bc43bcce5c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45448 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:32:08,312 - lava_callback - INFO - Received callback for node 69c337c418a3717aa65d2d51 INFO: 172.21.0.1:45452 - "POST /node/69c337c418a3717aa65d2d51 HTTP/1.0" 202 Accepted 2026-03-25 02:32:08,318 - lava_callback - INFO - Starting async job submit for node 69c337c418a3717aa65d2d51 2026-03-25 02:32:08,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:32:08,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337c418a3717aa65d2d51 HTTP/1.1" 200 None 2026-03-25 02:32:08,356 - lava_callback - DEBUG - Processing node 69c337c418a3717aa65d2d51: baseline-arm64-broonie 2026-03-25 02:32:08,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:32:08,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:32:08,878 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c418a3717aa65d2d51/log.txt.gz 2026-03-25 02:32:09,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:32:09,256 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:32:09,257 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c418a3717aa65d2d51/lava-logs.yaml 2026-03-25 02:32:09,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:32:09,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:32:09,422 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c337c418a3717aa65d2d51/lava_callback.json.gz 2026-03-25 02:32:09,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:32:09,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c337c418a3717aa65d2d51 HTTP/1.1" 200 None 2026-03-25 02:32:09,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:32:09,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 02:32:09,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:32:09,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c337c418a3717aa65d2d51 HTTP/1.1" 200 None 2026-03-25 02:32:09,511 - lava_callback - INFO - Completed processing callback for node 69c337c418a3717aa65d2d51 INFO: 172.21.0.1:47082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47094 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:32:21,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:32:21,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39262 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:35:51,328 - lava_callback - INFO - Received callback for node 69c3485618a3717aa65d3159 INFO: 172.21.0.1:48910 - "POST /node/69c3485618a3717aa65d3159 HTTP/1.0" 202 Accepted 2026-03-25 02:35:51,336 - lava_callback - INFO - Starting async job submit for node 69c3485618a3717aa65d3159 2026-03-25 02:35:51,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:35:51,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3485618a3717aa65d3159 HTTP/1.1" 200 None 2026-03-25 02:35:51,388 - lava_callback - DEBUG - Processing node 69c3485618a3717aa65d3159: baseline-arm64 2026-03-25 02:35:51,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:35:51,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:35:51,928 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3485618a3717aa65d3159/log.txt.gz 2026-03-25 02:35:52,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:35:52,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:35:52,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3485618a3717aa65d3159/lava-logs.yaml 2026-03-25 02:35:52,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:35:52,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:35:52,589 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c3485618a3717aa65d3159/lava_callback.json.gz DEBUG: Setup failed for 69c3485618a3717aa65d3159. Transitting job node result: incomplete -> fail 2026-03-25 02:35:52,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:35:52,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3485618a3717aa65d3159 HTTP/1.1" 200 None 2026-03-25 02:35:52,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:35:52,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 02:35:52,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:35:52,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3485618a3717aa65d3159 HTTP/1.1" 200 None 2026-03-25 02:35:52,821 - lava_callback - INFO - Completed processing callback for node 69c3485618a3717aa65d3159 INFO: 172.21.0.1:48922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49358 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:36:21,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:36:21,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36682 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:41:10,515 - lava_callback - INFO - Received callback for node 69c34a1018a3717aa65d3177 INFO: 172.21.0.1:45454 - "POST /node/69c34a1018a3717aa65d3177 HTTP/1.0" 202 Accepted 2026-03-25 02:41:10,522 - lava_callback - INFO - Starting async job submit for node 69c34a1018a3717aa65d3177 2026-03-25 02:41:10,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:41:10,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34a1018a3717aa65d3177 HTTP/1.1" 200 None 2026-03-25 02:41:10,573 - lava_callback - DEBUG - Processing node 69c34a1018a3717aa65d3177: baseline-arm64 2026-03-25 02:41:10,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:41:11,134 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:41:11,136 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c34a1018a3717aa65d3177/log.txt.gz 2026-03-25 02:41:11,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:41:11,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:41:11,719 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c34a1018a3717aa65d3177/lava-logs.yaml 2026-03-25 02:41:11,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:41:11,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:41:11,906 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c34a1018a3717aa65d3177/lava_callback.json.gz DEBUG: Setup failed for 69c34a1018a3717aa65d3177. Transitting job node result: incomplete -> fail 2026-03-25 02:41:11,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:41:11,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34a1018a3717aa65d3177 HTTP/1.1" 200 None 2026-03-25 02:41:11,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:41:12,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 02:41:12,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:41:12,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34a1018a3717aa65d3177 HTTP/1.1" 200 None 2026-03-25 02:41:12,124 - lava_callback - INFO - Completed processing callback for node 69c34a1018a3717aa65d3177 INFO: 172.21.0.1:45460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45476 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:41:21,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:41:21,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35120 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:47:14,691 - lava_callback - INFO - Received callback for node 69c3480a18a3717aa65d3153 INFO: 172.21.0.1:48426 - "POST /node/69c3480a18a3717aa65d3153 HTTP/1.0" 202 Accepted 2026-03-25 02:47:14,699 - lava_callback - INFO - Starting async job submit for node 69c3480a18a3717aa65d3153 2026-03-25 02:47:14,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:47:14,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3480a18a3717aa65d3153 HTTP/1.1" 200 None 2026-03-25 02:47:14,768 - lava_callback - DEBUG - Processing node 69c3480a18a3717aa65d3153: baseline-nfs-arm64 INFO: 172.21.0.1:48442 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 02:47:15,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:47:16,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:47:16,082 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3480a18a3717aa65d3153/log.txt.gz INFO: 172.21.0.1:48452 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:47:16,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:47:17,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:47:17,007 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3480a18a3717aa65d3153/lava-logs.yaml 2026-03-25 02:47:17,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:47:17,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:47:17,246 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3480a18a3717aa65d3153/lava_callback.json.gz DEBUG: Setup failed for 69c3480a18a3717aa65d3153. Transitting job node result: incomplete -> fail 2026-03-25 02:47:17,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:47:17,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3480a18a3717aa65d3153 HTTP/1.1" 200 None 2026-03-25 02:47:17,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:47:17,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 02:47:17,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:47:17,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3480a18a3717aa65d3153 HTTP/1.1" 200 None 2026-03-25 02:47:17,510 - lava_callback - INFO - Completed processing callback for node 69c3480a18a3717aa65d3153 2026-03-25 02:47:21,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:47:21,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39372 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:49:25,163 - lava_callback - INFO - Received callback for node 69c348a818a3717aa65d3163 INFO: 172.21.0.1:53314 - "POST /node/69c348a818a3717aa65d3163 HTTP/1.0" 202 Accepted 2026-03-25 02:49:25,209 - lava_callback - INFO - Starting async job submit for node 69c348a818a3717aa65d3163 2026-03-25 02:49:25,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:49:25,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c348a818a3717aa65d3163 HTTP/1.1" 200 None 2026-03-25 02:49:25,252 - lava_callback - DEBUG - Processing node 69c348a818a3717aa65d3163: baseline-arm-pengutronix 2026-03-25 02:49:25,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:49:25,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:49:25,514 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c348a818a3717aa65d3163/log.txt.gz 2026-03-25 02:49:25,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:49:25,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:49:25,774 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c348a818a3717aa65d3163/lava-logs.yaml 2026-03-25 02:49:25,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 02:49:25,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 02:49:25,921 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c348a818a3717aa65d3163/lava_callback.json.gz DEBUG: Setup failed for 69c348a818a3717aa65d3163. Transitting job node result: incomplete -> fail 2026-03-25 02:49:25,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:49:25,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c348a818a3717aa65d3163 HTTP/1.1" 200 None 2026-03-25 02:49:25,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:49:25,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-25 02:49:26,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:49:26,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c348a818a3717aa65d3163 HTTP/1.1" 200 None 2026-03-25 02:49:26,080 - lava_callback - INFO - Completed processing callback for node 69c348a818a3717aa65d3163 INFO: 172.21.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40142 - "GET / HTTP/1.0" 200 OK 2026-03-25 02:49:51,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 02:49:51,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45578 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:05:59,258 - lava_callback - INFO - Received callback for node 69c0ae0ace0c9bc43bcce6ab INFO: 172.21.0.1:45592 - "POST /node/69c0ae0ace0c9bc43bcce6ab HTTP/1.0" 202 Accepted 2026-03-25 03:05:59,263 - lava_callback - INFO - Starting async job submit for node 69c0ae0ace0c9bc43bcce6ab 2026-03-25 03:05:59,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:05:59,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ae0ace0c9bc43bcce6ab HTTP/1.1" 200 None 2026-03-25 03:05:59,302 - lava_callback - DEBUG - Processing node 69c0ae0ace0c9bc43bcce6ab: baseline-arm64-baylibre 2026-03-25 03:05:59,302 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ae0ace0c9bc43bcce6ab 2026-03-25 03:05:59,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:05:59,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ae0ace0c9bc43bcce6ab HTTP/1.1" 200 None 2026-03-25 03:05:59,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:05:59,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-25 03:05:59,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:05:59,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ae0ace0c9bc43bcce6ab HTTP/1.1" 200 None 2026-03-25 03:05:59,409 - lava_callback - ERROR - Error processing callback for node 69c0ae0ace0c9bc43bcce6ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42640 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:07:22,353 - lava_callback - INFO - Received callback for node 69c0ae59ce0c9bc43bcce6b1 INFO: 172.21.0.1:56884 - "POST /node/69c0ae59ce0c9bc43bcce6b1 HTTP/1.0" 202 Accepted 2026-03-25 03:07:22,359 - lava_callback - INFO - Starting async job submit for node 69c0ae59ce0c9bc43bcce6b1 2026-03-25 03:07:22,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:07:22,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ae59ce0c9bc43bcce6b1 HTTP/1.1" 200 None 2026-03-25 03:07:22,376 - lava_callback - DEBUG - Processing node 69c0ae59ce0c9bc43bcce6b1: baseline-arm64-clabbe 2026-03-25 03:07:22,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ae59ce0c9bc43bcce6b1 2026-03-25 03:07:22,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:07:22,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ae59ce0c9bc43bcce6b1 HTTP/1.1" 200 None 2026-03-25 03:07:22,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:07:22,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-25 03:07:22,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:07:22,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ae59ce0c9bc43bcce6b1 HTTP/1.1" 200 None 2026-03-25 03:07:22,436 - lava_callback - ERROR - Error processing callback for node 69c0ae59ce0c9bc43bcce6b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54424 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:07:34,823 - lava_callback - INFO - Received callback for node 69c34cd018a3717aa65d31c9 INFO: 172.21.0.1:54432 - "POST /node/69c34cd018a3717aa65d31c9 HTTP/1.0" 202 Accepted 2026-03-25 03:07:34,831 - lava_callback - INFO - Starting async job submit for node 69c34cd018a3717aa65d31c9 2026-03-25 03:07:34,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:07:34,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34cd018a3717aa65d31c9 HTTP/1.1" 200 None 2026-03-25 03:07:34,890 - lava_callback - DEBUG - Processing node 69c34cd018a3717aa65d31c9: baseline-nfs-arm64 2026-03-25 03:07:36,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 03:07:36,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 03:07:36,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c34cd018a3717aa65d31c9/log.txt.gz INFO: 172.21.0.1:54444 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:07:36,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 03:07:37,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 03:07:37,155 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c34cd018a3717aa65d31c9/lava-logs.yaml 2026-03-25 03:07:37,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 03:07:37,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 03:07:37,389 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c34cd018a3717aa65d31c9/lava_callback.json.gz DEBUG: Setup failed for 69c34cd018a3717aa65d31c9. Transitting job node result: incomplete -> fail 2026-03-25 03:07:37,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:07:37,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c34cd018a3717aa65d31c9 HTTP/1.1" 200 None 2026-03-25 03:07:37,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:07:37,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 03:07:37,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:07:37,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c34cd018a3717aa65d31c9 HTTP/1.1" 200 None 2026-03-25 03:07:37,642 - lava_callback - INFO - Completed processing callback for node 69c34cd018a3717aa65d31c9 INFO: 172.21.0.1:49728 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:07:51,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:07:51,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43368 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:08:42,391 - lava_callback - INFO - Received callback for node 69c0aea4ce0c9bc43bcce6c5 INFO: 172.21.0.1:53418 - "POST /node/69c0aea4ce0c9bc43bcce6c5 HTTP/1.0" 202 Accepted 2026-03-25 03:08:42,396 - lava_callback - INFO - Starting async job submit for node 69c0aea4ce0c9bc43bcce6c5 2026-03-25 03:08:42,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:08:42,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0aea4ce0c9bc43bcce6c5 HTTP/1.1" 200 None 2026-03-25 03:08:42,422 - lava_callback - DEBUG - Processing node 69c0aea4ce0c9bc43bcce6c5: baseline-arm64-clabbe 2026-03-25 03:08:42,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0aea4ce0c9bc43bcce6c5 2026-03-25 03:08:42,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:08:42,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0aea4ce0c9bc43bcce6c5 HTTP/1.1" 200 None 2026-03-25 03:08:42,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:08:42,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-25 03:08:42,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:08:42,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0aea4ce0c9bc43bcce6c5 HTTP/1.1" 200 None 2026-03-25 03:08:42,482 - lava_callback - ERROR - Error processing callback for node 69c0aea4ce0c9bc43bcce6c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38816 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:09:42,387 - lava_callback - INFO - Received callback for node 69c0aeeece0c9bc43bcce6ce INFO: 172.21.0.1:42606 - "POST /node/69c0aeeece0c9bc43bcce6ce HTTP/1.0" 202 Accepted 2026-03-25 03:09:42,393 - lava_callback - INFO - Starting async job submit for node 69c0aeeece0c9bc43bcce6ce 2026-03-25 03:09:42,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:09:42,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0aeeece0c9bc43bcce6ce HTTP/1.1" 200 None 2026-03-25 03:09:42,409 - lava_callback - DEBUG - Processing node 69c0aeeece0c9bc43bcce6ce: baseline-arm64-clabbe 2026-03-25 03:09:42,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0aeeece0c9bc43bcce6ce 2026-03-25 03:09:42,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:09:42,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0aeeece0c9bc43bcce6ce HTTP/1.1" 200 None 2026-03-25 03:09:42,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:09:42,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-25 03:09:42,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:09:42,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0aeeece0c9bc43bcce6ce HTTP/1.1" 200 None 2026-03-25 03:09:42,483 - lava_callback - ERROR - Error processing callback for node 69c0aeeece0c9bc43bcce6ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45524 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:11:02,469 - lava_callback - INFO - Received callback for node 69c0af3ace0c9bc43bcce6db INFO: 172.21.0.1:35976 - "POST /node/69c0af3ace0c9bc43bcce6db HTTP/1.0" 202 Accepted 2026-03-25 03:11:02,474 - lava_callback - INFO - Starting async job submit for node 69c0af3ace0c9bc43bcce6db 2026-03-25 03:11:02,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:11:02,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0af3ace0c9bc43bcce6db HTTP/1.1" 200 None 2026-03-25 03:11:02,490 - lava_callback - DEBUG - Processing node 69c0af3ace0c9bc43bcce6db: baseline-arm64-clabbe 2026-03-25 03:11:02,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0af3ace0c9bc43bcce6db 2026-03-25 03:11:02,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:11:02,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0af3ace0c9bc43bcce6db HTTP/1.1" 200 None 2026-03-25 03:11:02,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:11:02,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-25 03:11:02,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:11:02,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0af3ace0c9bc43bcce6db HTTP/1.1" 200 None 2026-03-25 03:11:02,544 - lava_callback - ERROR - Error processing callback for node 69c0af3ace0c9bc43bcce6db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:12:22,506 - lava_callback - INFO - Received callback for node 69c0af84ce0c9bc43bcce6e4 INFO: 172.21.0.1:54470 - "POST /node/69c0af84ce0c9bc43bcce6e4 HTTP/1.0" 202 Accepted 2026-03-25 03:12:22,511 - lava_callback - INFO - Starting async job submit for node 69c0af84ce0c9bc43bcce6e4 2026-03-25 03:12:22,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:12:22,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0af84ce0c9bc43bcce6e4 HTTP/1.1" 200 None 2026-03-25 03:12:22,528 - lava_callback - DEBUG - Processing node 69c0af84ce0c9bc43bcce6e4: baseline-arm64-clabbe 2026-03-25 03:12:22,528 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0af84ce0c9bc43bcce6e4 2026-03-25 03:12:22,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:12:22,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0af84ce0c9bc43bcce6e4 HTTP/1.1" 200 None 2026-03-25 03:12:22,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:12:22,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-25 03:12:22,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:12:22,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0af84ce0c9bc43bcce6e4 HTTP/1.1" 200 None 2026-03-25 03:12:22,616 - lava_callback - ERROR - Error processing callback for node 69c0af84ce0c9bc43bcce6e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33346 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:13:42,544 - lava_callback - INFO - Received callback for node 69c0afd9ce0c9bc43bcce6f2 INFO: 172.21.0.1:40244 - "POST /node/69c0afd9ce0c9bc43bcce6f2 HTTP/1.0" 202 Accepted 2026-03-25 03:13:42,550 - lava_callback - INFO - Starting async job submit for node 69c0afd9ce0c9bc43bcce6f2 2026-03-25 03:13:42,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:13:42,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0afd9ce0c9bc43bcce6f2 HTTP/1.1" 200 None 2026-03-25 03:13:42,577 - lava_callback - DEBUG - Processing node 69c0afd9ce0c9bc43bcce6f2: baseline-arm64-clabbe 2026-03-25 03:13:42,577 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0afd9ce0c9bc43bcce6f2 2026-03-25 03:13:42,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:13:42,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0afd9ce0c9bc43bcce6f2 HTTP/1.1" 200 None 2026-03-25 03:13:42,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:13:42,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-25 03:13:42,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:13:42,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0afd9ce0c9bc43bcce6f2 HTTP/1.1" 200 None 2026-03-25 03:13:42,636 - lava_callback - ERROR - Error processing callback for node 69c0afd9ce0c9bc43bcce6f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47616 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:23:39,912 - lava_callback - INFO - Received callback for node 69c0b230ce0c9bc43bcce72a INFO: 172.21.0.1:46150 - "POST /node/69c0b230ce0c9bc43bcce72a HTTP/1.0" 202 Accepted 2026-03-25 03:23:39,917 - lava_callback - INFO - Starting async job submit for node 69c0b230ce0c9bc43bcce72a 2026-03-25 03:23:39,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:23:39,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b230ce0c9bc43bcce72a HTTP/1.1" 200 None 2026-03-25 03:23:39,933 - lava_callback - DEBUG - Processing node 69c0b230ce0c9bc43bcce72a: baseline-arm-baylibre 2026-03-25 03:23:39,933 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0b230ce0c9bc43bcce72a 2026-03-25 03:23:39,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:23:39,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b230ce0c9bc43bcce72a HTTP/1.1" 200 None 2026-03-25 03:23:39,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:23:39,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-25 03:23:39,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:23:39,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0b230ce0c9bc43bcce72a HTTP/1.1" 200 None 2026-03-25 03:23:40,000 - lava_callback - ERROR - Error processing callback for node 69c0b230ce0c9bc43bcce72a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37422 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:24:19,835 - lava_callback - INFO - Received callback for node 69c0b257ce0c9bc43bcce736 INFO: 172.21.0.1:49106 - "POST /node/69c0b257ce0c9bc43bcce736 HTTP/1.0" 202 Accepted 2026-03-25 03:24:19,841 - lava_callback - INFO - Starting async job submit for node 69c0b257ce0c9bc43bcce736 2026-03-25 03:24:19,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:24:19,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b257ce0c9bc43bcce736 HTTP/1.1" 200 None 2026-03-25 03:24:19,857 - lava_callback - DEBUG - Processing node 69c0b257ce0c9bc43bcce736: baseline-arm-baylibre 2026-03-25 03:24:19,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0b257ce0c9bc43bcce736 2026-03-25 03:24:19,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:24:19,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b257ce0c9bc43bcce736 HTTP/1.1" 200 None 2026-03-25 03:24:19,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:24:19,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-25 03:24:19,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:24:19,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0b257ce0c9bc43bcce736 HTTP/1.1" 200 None 2026-03-25 03:24:19,917 - lava_callback - ERROR - Error processing callback for node 69c0b257ce0c9bc43bcce736: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50868 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:24:59,898 - lava_callback - INFO - Received callback for node 69c0b27dce0c9bc43bcce742 INFO: 172.21.0.1:46084 - "POST /node/69c0b27dce0c9bc43bcce742 HTTP/1.0" 202 Accepted 2026-03-25 03:24:59,903 - lava_callback - INFO - Starting async job submit for node 69c0b27dce0c9bc43bcce742 2026-03-25 03:24:59,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:24:59,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b27dce0c9bc43bcce742 HTTP/1.1" 200 None 2026-03-25 03:24:59,920 - lava_callback - DEBUG - Processing node 69c0b27dce0c9bc43bcce742: baseline-arm-baylibre 2026-03-25 03:24:59,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0b27dce0c9bc43bcce742 2026-03-25 03:24:59,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:24:59,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b27dce0c9bc43bcce742 HTTP/1.1" 200 None 2026-03-25 03:24:59,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:24:59,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-25 03:24:59,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:24:59,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0b27dce0c9bc43bcce742 HTTP/1.1" 200 None 2026-03-25 03:24:59,993 - lava_callback - ERROR - Error processing callback for node 69c0b27dce0c9bc43bcce742: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41774 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:28:00,418 - lava_callback - INFO - Received callback for node 69c3519418a3717aa65d329c INFO: 172.21.0.1:51400 - "POST /node/69c3519418a3717aa65d329c HTTP/1.0" 202 Accepted 2026-03-25 03:28:00,428 - lava_callback - INFO - Starting async job submit for node 69c3519418a3717aa65d329c 2026-03-25 03:28:00,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:28:00,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3519418a3717aa65d329c HTTP/1.1" 200 None 2026-03-25 03:28:00,492 - lava_callback - DEBUG - Processing node 69c3519418a3717aa65d329c: baseline-nfs-arm64 2026-03-25 03:28:01,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 03:28:01,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 03:28:01,884 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3519418a3717aa65d329c/log.txt.gz 2026-03-25 03:28:02,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 03:28:03,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 03:28:03,787 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3519418a3717aa65d329c/lava-logs.yaml 2026-03-25 03:28:03,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-25 03:28:03,996 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-25 03:28:03,998 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c3519418a3717aa65d329c/lava_callback.json.gz DEBUG: Setup failed for 69c3519418a3717aa65d329c. Transitting job node result: incomplete -> fail 2026-03-25 03:28:04,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:28:04,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3519418a3717aa65d329c HTTP/1.1" 200 None 2026-03-25 03:28:04,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:28:04,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-25 03:28:04,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:28:04,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3519418a3717aa65d329c HTTP/1.1" 200 None 2026-03-25 03:28:04,239 - lava_callback - INFO - Completed processing callback for node 69c3519418a3717aa65d329c INFO: 172.21.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:28:21,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:28:21,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48336 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:29:23,263 - lava_callback - INFO - Received callback for node 69c0b383ce0c9bc43bcce76b INFO: 172.21.0.1:46286 - "POST /node/69c0b383ce0c9bc43bcce76b HTTP/1.0" 202 Accepted 2026-03-25 03:29:23,269 - lava_callback - INFO - Starting async job submit for node 69c0b383ce0c9bc43bcce76b 2026-03-25 03:29:23,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:29:23,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b383ce0c9bc43bcce76b HTTP/1.1" 200 None 2026-03-25 03:29:23,304 - lava_callback - DEBUG - Processing node 69c0b383ce0c9bc43bcce76b: baseline-arm-clabbe 2026-03-25 03:29:23,305 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0b383ce0c9bc43bcce76b 2026-03-25 03:29:23,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:29:23,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b383ce0c9bc43bcce76b HTTP/1.1" 200 None 2026-03-25 03:29:23,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:29:23,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-25 03:29:23,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:29:23,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0b383ce0c9bc43bcce76b HTTP/1.1" 200 None 2026-03-25 03:29:23,373 - lava_callback - ERROR - Error processing callback for node 69c0b383ce0c9bc43bcce76b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52086 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:30:03,441 - lava_callback - INFO - Received callback for node 69c0b3a7ce0c9bc43bcce777 INFO: 172.21.0.1:54438 - "POST /node/69c0b3a7ce0c9bc43bcce777 HTTP/1.0" 202 Accepted 2026-03-25 03:30:03,447 - lava_callback - INFO - Starting async job submit for node 69c0b3a7ce0c9bc43bcce777 2026-03-25 03:30:03,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:30:03,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b3a7ce0c9bc43bcce777 HTTP/1.1" 200 None 2026-03-25 03:30:03,467 - lava_callback - DEBUG - Processing node 69c0b3a7ce0c9bc43bcce777: baseline-arm-clabbe 2026-03-25 03:30:03,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0b3a7ce0c9bc43bcce777 2026-03-25 03:30:03,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:30:03,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b3a7ce0c9bc43bcce777 HTTP/1.1" 200 None 2026-03-25 03:30:03,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:30:03,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-25 03:30:03,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:30:03,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0b3a7ce0c9bc43bcce777 HTTP/1.1" 200 None 2026-03-25 03:30:03,526 - lava_callback - ERROR - Error processing callback for node 69c0b3a7ce0c9bc43bcce777: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:30:43,374 - lava_callback - INFO - Received callback for node 69c0b3d9ce0c9bc43bcce783 INFO: 172.21.0.1:58330 - "POST /node/69c0b3d9ce0c9bc43bcce783 HTTP/1.0" 202 Accepted 2026-03-25 03:30:43,380 - lava_callback - INFO - Starting async job submit for node 69c0b3d9ce0c9bc43bcce783 2026-03-25 03:30:43,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:30:43,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b3d9ce0c9bc43bcce783 HTTP/1.1" 200 None 2026-03-25 03:30:43,404 - lava_callback - DEBUG - Processing node 69c0b3d9ce0c9bc43bcce783: baseline-arm-clabbe 2026-03-25 03:30:43,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0b3d9ce0c9bc43bcce783 2026-03-25 03:30:43,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:30:43,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b3d9ce0c9bc43bcce783 HTTP/1.1" 200 None 2026-03-25 03:30:43,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:30:43,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-25 03:30:43,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:30:43,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0b3d9ce0c9bc43bcce783 HTTP/1.1" 200 None 2026-03-25 03:30:43,464 - lava_callback - ERROR - Error processing callback for node 69c0b3d9ce0c9bc43bcce783: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46632 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:31:43,409 - lava_callback - INFO - Received callback for node 69c0b414ce0c9bc43bcce791 INFO: 172.21.0.1:48216 - "POST /node/69c0b414ce0c9bc43bcce791 HTTP/1.0" 202 Accepted 2026-03-25 03:31:43,414 - lava_callback - INFO - Starting async job submit for node 69c0b414ce0c9bc43bcce791 2026-03-25 03:31:43,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:31:43,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b414ce0c9bc43bcce791 HTTP/1.1" 200 None 2026-03-25 03:31:43,431 - lava_callback - DEBUG - Processing node 69c0b414ce0c9bc43bcce791: baseline-arm-clabbe 2026-03-25 03:31:43,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0b414ce0c9bc43bcce791 2026-03-25 03:31:43,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:31:43,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b414ce0c9bc43bcce791 HTTP/1.1" 200 None 2026-03-25 03:31:43,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:31:43,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-25 03:31:43,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:31:43,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0b414ce0c9bc43bcce791 HTTP/1.1" 200 None 2026-03-25 03:31:43,491 - lava_callback - ERROR - Error processing callback for node 69c0b414ce0c9bc43bcce791: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48712 - "GET / HTTP/1.0" 200 OK 2026-03-25 03:32:43,427 - lava_callback - INFO - Received callback for node 69c0b451ce0c9bc43bcce79e INFO: 172.21.0.1:37338 - "POST /node/69c0b451ce0c9bc43bcce79e HTTP/1.0" 202 Accepted 2026-03-25 03:32:43,432 - lava_callback - INFO - Starting async job submit for node 69c0b451ce0c9bc43bcce79e 2026-03-25 03:32:43,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:32:43,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b451ce0c9bc43bcce79e HTTP/1.1" 200 None 2026-03-25 03:32:43,449 - lava_callback - DEBUG - Processing node 69c0b451ce0c9bc43bcce79e: baseline-arm-clabbe 2026-03-25 03:32:43,449 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0b451ce0c9bc43bcce79e 2026-03-25 03:32:43,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:32:43,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0b451ce0c9bc43bcce79e HTTP/1.1" 200 None 2026-03-25 03:32:43,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:32:43,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-25 03:32:43,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 03:32:43,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0b451ce0c9bc43bcce79e HTTP/1.1" 200 None 2026-03-25 03:32:43,505 - lava_callback - ERROR - Error processing callback for node 69c0b451ce0c9bc43bcce79e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48120 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:49:44,679 - lava_callback - INFO - Received callback for node 69c0c65fce0c9bc43bcce819 INFO: 172.21.0.1:39930 - "POST /node/69c0c65fce0c9bc43bcce819 HTTP/1.0" 202 Accepted 2026-03-25 04:49:44,685 - lava_callback - INFO - Starting async job submit for node 69c0c65fce0c9bc43bcce819 2026-03-25 04:49:44,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:49:44,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c65fce0c9bc43bcce819 HTTP/1.1" 200 None 2026-03-25 04:49:44,706 - lava_callback - DEBUG - Processing node 69c0c65fce0c9bc43bcce819: baseline-arm-baylibre 2026-03-25 04:49:44,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c65fce0c9bc43bcce819 2026-03-25 04:49:44,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:49:44,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c65fce0c9bc43bcce819 HTTP/1.1" 200 None 2026-03-25 04:49:44,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:49:44,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:49:44,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:49:44,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c65fce0c9bc43bcce819 HTTP/1.1" 200 None 2026-03-25 04:49:44,760 - lava_callback - ERROR - Error processing callback for node 69c0c65fce0c9bc43bcce819: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56536 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:50:24,615 - lava_callback - INFO - Received callback for node 69c0c684ce0c9bc43bcce825 INFO: 172.21.0.1:48190 - "POST /node/69c0c684ce0c9bc43bcce825 HTTP/1.0" 202 Accepted 2026-03-25 04:50:24,621 - lava_callback - INFO - Starting async job submit for node 69c0c684ce0c9bc43bcce825 2026-03-25 04:50:24,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:50:24,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c684ce0c9bc43bcce825 HTTP/1.1" 200 None 2026-03-25 04:50:24,639 - lava_callback - DEBUG - Processing node 69c0c684ce0c9bc43bcce825: baseline-arm-baylibre 2026-03-25 04:50:24,639 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c684ce0c9bc43bcce825 2026-03-25 04:50:24,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:50:24,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c684ce0c9bc43bcce825 HTTP/1.1" 200 None 2026-03-25 04:50:24,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:50:24,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:50:24,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:50:24,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c684ce0c9bc43bcce825 HTTP/1.1" 200 None 2026-03-25 04:50:24,706 - lava_callback - ERROR - Error processing callback for node 69c0c684ce0c9bc43bcce825: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60468 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:51:05,024 - lava_callback - INFO - Received callback for node 69c0c6a9ce0c9bc43bcce835 INFO: 172.21.0.1:43296 - "POST /node/69c0c6a9ce0c9bc43bcce835 HTTP/1.0" 202 Accepted 2026-03-25 04:51:05,030 - lava_callback - INFO - Starting async job submit for node 69c0c6a9ce0c9bc43bcce835 2026-03-25 04:51:05,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:51:05,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c6a9ce0c9bc43bcce835 HTTP/1.1" 200 None 2026-03-25 04:51:05,045 - lava_callback - DEBUG - Processing node 69c0c6a9ce0c9bc43bcce835: baseline-arm-baylibre 2026-03-25 04:51:05,046 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c6a9ce0c9bc43bcce835 2026-03-25 04:51:05,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:51:05,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c6a9ce0c9bc43bcce835 HTTP/1.1" 200 None 2026-03-25 04:51:05,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:51:05,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:51:05,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:51:05,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c6a9ce0c9bc43bcce835 HTTP/1.1" 200 None 2026-03-25 04:51:05,104 - lava_callback - ERROR - Error processing callback for node 69c0c6a9ce0c9bc43bcce835: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36500 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:51:44,701 - lava_callback - INFO - Received callback for node 69c0c6cdce0c9bc43bcce83f INFO: 172.21.0.1:45512 - "POST /node/69c0c6cdce0c9bc43bcce83f HTTP/1.0" 202 Accepted 2026-03-25 04:51:44,706 - lava_callback - INFO - Starting async job submit for node 69c0c6cdce0c9bc43bcce83f 2026-03-25 04:51:44,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:51:44,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c6cdce0c9bc43bcce83f HTTP/1.1" 200 None 2026-03-25 04:51:44,722 - lava_callback - DEBUG - Processing node 69c0c6cdce0c9bc43bcce83f: baseline-arm-baylibre 2026-03-25 04:51:44,723 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c6cdce0c9bc43bcce83f 2026-03-25 04:51:44,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:51:44,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c6cdce0c9bc43bcce83f HTTP/1.1" 200 None 2026-03-25 04:51:44,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:51:44,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:51:44,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:51:44,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c6cdce0c9bc43bcce83f HTTP/1.1" 200 None 2026-03-25 04:51:44,797 - lava_callback - ERROR - Error processing callback for node 69c0c6cdce0c9bc43bcce83f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38830 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:52:24,862 - lava_callback - INFO - Received callback for node 69c0c6f3ce0c9bc43bcce84b INFO: 172.21.0.1:59252 - "POST /node/69c0c6f3ce0c9bc43bcce84b HTTP/1.0" 202 Accepted 2026-03-25 04:52:24,867 - lava_callback - INFO - Starting async job submit for node 69c0c6f3ce0c9bc43bcce84b 2026-03-25 04:52:24,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:52:24,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c6f3ce0c9bc43bcce84b HTTP/1.1" 200 None 2026-03-25 04:52:24,883 - lava_callback - DEBUG - Processing node 69c0c6f3ce0c9bc43bcce84b: baseline-arm-baylibre 2026-03-25 04:52:24,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c6f3ce0c9bc43bcce84b 2026-03-25 04:52:24,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:52:24,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c6f3ce0c9bc43bcce84b HTTP/1.1" 200 None 2026-03-25 04:52:24,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:52:24,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:52:24,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:52:24,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c6f3ce0c9bc43bcce84b HTTP/1.1" 200 None 2026-03-25 04:52:24,955 - lava_callback - ERROR - Error processing callback for node 69c0c6f3ce0c9bc43bcce84b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49444 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:52:44,749 - lava_callback - INFO - Received callback for node 69c0c717ce0c9bc43bcce859 INFO: 172.21.0.1:58814 - "POST /node/69c0c717ce0c9bc43bcce859 HTTP/1.0" 202 Accepted 2026-03-25 04:52:44,755 - lava_callback - INFO - Starting async job submit for node 69c0c717ce0c9bc43bcce859 2026-03-25 04:52:44,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:52:44,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c717ce0c9bc43bcce859 HTTP/1.1" 200 None 2026-03-25 04:52:44,771 - lava_callback - DEBUG - Processing node 69c0c717ce0c9bc43bcce859: baseline-arm-baylibre 2026-03-25 04:52:44,772 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c717ce0c9bc43bcce859 2026-03-25 04:52:44,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:52:44,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c717ce0c9bc43bcce859 HTTP/1.1" 200 None 2026-03-25 04:52:44,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:52:44,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:52:44,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:52:44,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c717ce0c9bc43bcce859 HTTP/1.1" 200 None 2026-03-25 04:52:44,834 - lava_callback - ERROR - Error processing callback for node 69c0c717ce0c9bc43bcce859: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51150 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:55:07,086 - lava_callback - INFO - Received callback for node 69c0c79dce0c9bc43bcce884 INFO: 172.21.0.1:51164 - "POST /node/69c0c79dce0c9bc43bcce884 HTTP/1.0" 202 Accepted 2026-03-25 04:55:07,092 - lava_callback - INFO - Starting async job submit for node 69c0c79dce0c9bc43bcce884 2026-03-25 04:55:07,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:55:07,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c79dce0c9bc43bcce884 HTTP/1.1" 200 None 2026-03-25 04:55:07,113 - lava_callback - DEBUG - Processing node 69c0c79dce0c9bc43bcce884: baseline-arm-clabbe 2026-03-25 04:55:07,113 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c79dce0c9bc43bcce884 2026-03-25 04:55:07,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:55:07,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c79dce0c9bc43bcce884 HTTP/1.1" 200 None 2026-03-25 04:55:07,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:55:07,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:55:07,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:55:07,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c79dce0c9bc43bcce884 HTTP/1.1" 200 None 2026-03-25 04:55:07,167 - lava_callback - ERROR - Error processing callback for node 69c0c79dce0c9bc43bcce884: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58570 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:55:46,963 - lava_callback - INFO - Received callback for node 69c0c7ccce0c9bc43bcce890 INFO: 172.21.0.1:58582 - "POST /node/69c0c7ccce0c9bc43bcce890 HTTP/1.0" 202 Accepted 2026-03-25 04:55:46,968 - lava_callback - INFO - Starting async job submit for node 69c0c7ccce0c9bc43bcce890 2026-03-25 04:55:46,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:55:46,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c7ccce0c9bc43bcce890 HTTP/1.1" 200 None 2026-03-25 04:55:46,986 - lava_callback - DEBUG - Processing node 69c0c7ccce0c9bc43bcce890: baseline-arm-clabbe 2026-03-25 04:55:46,986 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c7ccce0c9bc43bcce890 2026-03-25 04:55:46,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:55:47,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c7ccce0c9bc43bcce890 HTTP/1.1" 200 None 2026-03-25 04:55:47,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:55:47,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:55:47,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:55:47,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c7ccce0c9bc43bcce890 HTTP/1.1" 200 None 2026-03-25 04:55:47,059 - lava_callback - ERROR - Error processing callback for node 69c0c7ccce0c9bc43bcce890: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34872 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:56:27,039 - lava_callback - INFO - Received callback for node 69c0c7f1ce0c9bc43bcce89d INFO: 172.21.0.1:34888 - "POST /node/69c0c7f1ce0c9bc43bcce89d HTTP/1.0" 202 Accepted 2026-03-25 04:56:27,045 - lava_callback - INFO - Starting async job submit for node 69c0c7f1ce0c9bc43bcce89d 2026-03-25 04:56:27,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:56:27,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c7f1ce0c9bc43bcce89d HTTP/1.1" 200 None 2026-03-25 04:56:27,063 - lava_callback - DEBUG - Processing node 69c0c7f1ce0c9bc43bcce89d: baseline-arm-clabbe 2026-03-25 04:56:27,064 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c7f1ce0c9bc43bcce89d 2026-03-25 04:56:27,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:56:27,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c7f1ce0c9bc43bcce89d HTTP/1.1" 200 None 2026-03-25 04:56:27,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:56:27,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:56:27,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:56:27,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c7f1ce0c9bc43bcce89d HTTP/1.1" 200 None 2026-03-25 04:56:27,135 - lava_callback - ERROR - Error processing callback for node 69c0c7f1ce0c9bc43bcce89d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54070 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:57:07,047 - lava_callback - INFO - Received callback for node 69c0c817ce0c9bc43bcce8ab INFO: 172.21.0.1:54084 - "POST /node/69c0c817ce0c9bc43bcce8ab HTTP/1.0" 202 Accepted 2026-03-25 04:57:07,053 - lava_callback - INFO - Starting async job submit for node 69c0c817ce0c9bc43bcce8ab 2026-03-25 04:57:07,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:57:07,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c817ce0c9bc43bcce8ab HTTP/1.1" 200 None 2026-03-25 04:57:07,070 - lava_callback - DEBUG - Processing node 69c0c817ce0c9bc43bcce8ab: baseline-arm-clabbe 2026-03-25 04:57:07,070 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c817ce0c9bc43bcce8ab 2026-03-25 04:57:07,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:57:07,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c817ce0c9bc43bcce8ab HTTP/1.1" 200 None 2026-03-25 04:57:07,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:57:07,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:57:07,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:57:07,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c817ce0c9bc43bcce8ab HTTP/1.1" 200 None 2026-03-25 04:57:07,149 - lava_callback - ERROR - Error processing callback for node 69c0c817ce0c9bc43bcce8ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46966 - "GET / HTTP/1.0" 200 OK 2026-03-25 04:57:47,118 - lava_callback - INFO - Received callback for node 69c0c83bce0c9bc43bcce8b5 INFO: 172.21.0.1:46968 - "POST /node/69c0c83bce0c9bc43bcce8b5 HTTP/1.0" 202 Accepted 2026-03-25 04:57:47,123 - lava_callback - INFO - Starting async job submit for node 69c0c83bce0c9bc43bcce8b5 2026-03-25 04:57:47,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:57:47,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c83bce0c9bc43bcce8b5 HTTP/1.1" 200 None 2026-03-25 04:57:47,140 - lava_callback - DEBUG - Processing node 69c0c83bce0c9bc43bcce8b5: baseline-arm-clabbe 2026-03-25 04:57:47,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0c83bce0c9bc43bcce8b5 2026-03-25 04:57:47,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:57:47,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0c83bce0c9bc43bcce8b5 HTTP/1.1" 200 None 2026-03-25 04:57:47,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:57:47,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-25 04:57:47,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 04:57:47,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0c83bce0c9bc43bcce8b5 HTTP/1.1" 200 None 2026-03-25 04:57:47,220 - lava_callback - ERROR - Error processing callback for node 69c0c83bce0c9bc43bcce8b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60772 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:10:05,861 - lava_callback - INFO - Received callback for node 69c0cb1fce0c9bc43bcce902 INFO: 172.21.0.1:44200 - "POST /node/69c0cb1fce0c9bc43bcce902 HTTP/1.0" 202 Accepted 2026-03-25 05:10:05,866 - lava_callback - INFO - Starting async job submit for node 69c0cb1fce0c9bc43bcce902 2026-03-25 05:10:05,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:10:05,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cb1fce0c9bc43bcce902 HTTP/1.1" 200 None 2026-03-25 05:10:05,883 - lava_callback - DEBUG - Processing node 69c0cb1fce0c9bc43bcce902: baseline-arm64-baylibre 2026-03-25 05:10:05,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cb1fce0c9bc43bcce902 2026-03-25 05:10:05,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:10:05,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cb1fce0c9bc43bcce902 HTTP/1.1" 200 None 2026-03-25 05:10:05,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:10:05,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:10:05,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:10:05,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cb1fce0c9bc43bcce902 HTTP/1.1" 200 None 2026-03-25 05:10:05,961 - lava_callback - ERROR - Error processing callback for node 69c0cb1fce0c9bc43bcce902: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51016 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:12:07,749 - lava_callback - INFO - Received callback for node 69c0cb9cce0c9bc43bcce911 INFO: 172.21.0.1:51030 - "POST /node/69c0cb9cce0c9bc43bcce911 HTTP/1.0" 202 Accepted 2026-03-25 05:12:07,755 - lava_callback - INFO - Starting async job submit for node 69c0cb9cce0c9bc43bcce911 2026-03-25 05:12:07,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:12:07,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cb9cce0c9bc43bcce911 HTTP/1.1" 200 None 2026-03-25 05:12:07,771 - lava_callback - DEBUG - Processing node 69c0cb9cce0c9bc43bcce911: baseline-arm64-clabbe 2026-03-25 05:12:07,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cb9cce0c9bc43bcce911 2026-03-25 05:12:07,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:12:07,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cb9cce0c9bc43bcce911 HTTP/1.1" 200 None 2026-03-25 05:12:07,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:12:07,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:12:07,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:12:07,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cb9cce0c9bc43bcce911 HTTP/1.1" 200 None 2026-03-25 05:12:07,836 - lava_callback - ERROR - Error processing callback for node 69c0cb9cce0c9bc43bcce911: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50222 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:14:07,810 - lava_callback - INFO - Received callback for node 69c0cc14ce0c9bc43bcce91c INFO: 172.21.0.1:50226 - "POST /node/69c0cc14ce0c9bc43bcce91c HTTP/1.0" 202 Accepted 2026-03-25 05:14:07,815 - lava_callback - INFO - Starting async job submit for node 69c0cc14ce0c9bc43bcce91c 2026-03-25 05:14:07,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:14:07,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cc14ce0c9bc43bcce91c HTTP/1.1" 200 None 2026-03-25 05:14:07,831 - lava_callback - DEBUG - Processing node 69c0cc14ce0c9bc43bcce91c: baseline-arm64-clabbe 2026-03-25 05:14:07,832 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cc14ce0c9bc43bcce91c 2026-03-25 05:14:07,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:14:07,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cc14ce0c9bc43bcce91c HTTP/1.1" 200 None 2026-03-25 05:14:07,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:14:07,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:14:07,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:14:07,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cc14ce0c9bc43bcce91c HTTP/1.1" 200 None 2026-03-25 05:14:07,908 - lava_callback - ERROR - Error processing callback for node 69c0cc14ce0c9bc43bcce91c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35540 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:15:27,829 - lava_callback - INFO - Received callback for node 69c0cc63ce0c9bc43bcce924 INFO: 172.21.0.1:35548 - "POST /node/69c0cc63ce0c9bc43bcce924 HTTP/1.0" 202 Accepted 2026-03-25 05:15:27,835 - lava_callback - INFO - Starting async job submit for node 69c0cc63ce0c9bc43bcce924 2026-03-25 05:15:27,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:15:27,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cc63ce0c9bc43bcce924 HTTP/1.1" 200 None 2026-03-25 05:15:27,851 - lava_callback - DEBUG - Processing node 69c0cc63ce0c9bc43bcce924: baseline-arm64-clabbe 2026-03-25 05:15:27,851 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cc63ce0c9bc43bcce924 2026-03-25 05:15:27,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:15:27,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cc63ce0c9bc43bcce924 HTTP/1.1" 200 None 2026-03-25 05:15:27,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:15:27,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:15:27,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:15:27,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cc63ce0c9bc43bcce924 HTTP/1.1" 200 None 2026-03-25 05:15:27,910 - lava_callback - ERROR - Error processing callback for node 69c0cc63ce0c9bc43bcce924: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42696 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:16:47,899 - lava_callback - INFO - Received callback for node 69c0ccb0ce0c9bc43bcce931 INFO: 172.21.0.1:42708 - "POST /node/69c0ccb0ce0c9bc43bcce931 HTTP/1.0" 202 Accepted 2026-03-25 05:16:47,905 - lava_callback - INFO - Starting async job submit for node 69c0ccb0ce0c9bc43bcce931 2026-03-25 05:16:47,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:16:47,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ccb0ce0c9bc43bcce931 HTTP/1.1" 200 None 2026-03-25 05:16:47,928 - lava_callback - DEBUG - Processing node 69c0ccb0ce0c9bc43bcce931: baseline-arm64-clabbe 2026-03-25 05:16:47,928 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ccb0ce0c9bc43bcce931 2026-03-25 05:16:47,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:16:47,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ccb0ce0c9bc43bcce931 HTTP/1.1" 200 None 2026-03-25 05:16:47,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:16:47,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:16:47,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:16:47,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ccb0ce0c9bc43bcce931 HTTP/1.1" 200 None 2026-03-25 05:16:47,984 - lava_callback - ERROR - Error processing callback for node 69c0ccb0ce0c9bc43bcce931: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43402 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:18:08,029 - lava_callback - INFO - Received callback for node 69c0cd06ce0c9bc43bcce93d INFO: 172.21.0.1:43414 - "POST /node/69c0cd06ce0c9bc43bcce93d HTTP/1.0" 202 Accepted 2026-03-25 05:18:08,034 - lava_callback - INFO - Starting async job submit for node 69c0cd06ce0c9bc43bcce93d 2026-03-25 05:18:08,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:18:08,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cd06ce0c9bc43bcce93d HTTP/1.1" 200 None 2026-03-25 05:18:08,051 - lava_callback - DEBUG - Processing node 69c0cd06ce0c9bc43bcce93d: baseline-arm64-clabbe 2026-03-25 05:18:08,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cd06ce0c9bc43bcce93d 2026-03-25 05:18:08,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:18:08,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cd06ce0c9bc43bcce93d HTTP/1.1" 200 None 2026-03-25 05:18:08,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:18:08,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:18:08,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:18:08,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cd06ce0c9bc43bcce93d HTTP/1.1" 200 None 2026-03-25 05:18:08,109 - lava_callback - ERROR - Error processing callback for node 69c0cd06ce0c9bc43bcce93d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37522 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:19:26,322 - lava_callback - INFO - Received callback for node 69c0cd4fce0c9bc43bcce947 INFO: 172.21.0.1:46640 - "POST /node/69c0cd4fce0c9bc43bcce947 HTTP/1.0" 202 Accepted 2026-03-25 05:19:26,327 - lava_callback - INFO - Starting async job submit for node 69c0cd4fce0c9bc43bcce947 2026-03-25 05:19:26,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:19:26,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cd4fce0c9bc43bcce947 HTTP/1.1" 200 None 2026-03-25 05:19:26,344 - lava_callback - DEBUG - Processing node 69c0cd4fce0c9bc43bcce947: baseline-arm64-baylibre 2026-03-25 05:19:26,345 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cd4fce0c9bc43bcce947 2026-03-25 05:19:26,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:19:26,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cd4fce0c9bc43bcce947 HTTP/1.1" 200 None 2026-03-25 05:19:26,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:19:26,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:19:26,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:19:26,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cd4fce0c9bc43bcce947 HTTP/1.1" 200 None 2026-03-25 05:19:26,407 - lava_callback - ERROR - Error processing callback for node 69c0cd4fce0c9bc43bcce947: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55350 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:20:48,094 - lava_callback - INFO - Received callback for node 69c0cda3ce0c9bc43bcce953 INFO: 172.21.0.1:55364 - "POST /node/69c0cda3ce0c9bc43bcce953 HTTP/1.0" 202 Accepted 2026-03-25 05:20:48,100 - lava_callback - INFO - Starting async job submit for node 69c0cda3ce0c9bc43bcce953 2026-03-25 05:20:48,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:20:48,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cda3ce0c9bc43bcce953 HTTP/1.1" 200 None 2026-03-25 05:20:48,117 - lava_callback - DEBUG - Processing node 69c0cda3ce0c9bc43bcce953: baseline-arm64-clabbe 2026-03-25 05:20:48,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cda3ce0c9bc43bcce953 2026-03-25 05:20:48,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:20:48,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cda3ce0c9bc43bcce953 HTTP/1.1" 200 None 2026-03-25 05:20:48,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:20:48,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:20:48,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:20:48,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cda3ce0c9bc43bcce953 HTTP/1.1" 200 None 2026-03-25 05:20:48,209 - lava_callback - ERROR - Error processing callback for node 69c0cda3ce0c9bc43bcce953: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57512 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:22:06,489 - lava_callback - INFO - Received callback for node 69c0cdeace0c9bc43bcce95f INFO: 172.21.0.1:57090 - "POST /node/69c0cdeace0c9bc43bcce95f HTTP/1.0" 202 Accepted 2026-03-25 05:22:06,495 - lava_callback - INFO - Starting async job submit for node 69c0cdeace0c9bc43bcce95f 2026-03-25 05:22:06,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:22:06,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cdeace0c9bc43bcce95f HTTP/1.1" 200 None INFO: 172.21.0.1:57096 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:22:06,541 - lava_callback - DEBUG - Processing node 69c0cdeace0c9bc43bcce95f: baseline-arm64-baylibre 2026-03-25 05:22:06,541 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cdeace0c9bc43bcce95f 2026-03-25 05:22:06,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:22:06,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cdeace0c9bc43bcce95f HTTP/1.1" 200 None 2026-03-25 05:22:06,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:22:06,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:22:06,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:22:06,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cdeace0c9bc43bcce95f HTTP/1.1" 200 None 2026-03-25 05:22:06,600 - lava_callback - ERROR - Error processing callback for node 69c0cdeace0c9bc43bcce95f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47340 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:23:08,215 - lava_callback - INFO - Received callback for node 69c0ce2dce0c9bc43bcce96d INFO: 172.21.0.1:47354 - "POST /node/69c0ce2dce0c9bc43bcce96d HTTP/1.0" 202 Accepted 2026-03-25 05:23:08,220 - lava_callback - INFO - Starting async job submit for node 69c0ce2dce0c9bc43bcce96d 2026-03-25 05:23:08,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:23:08,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ce2dce0c9bc43bcce96d HTTP/1.1" 200 None 2026-03-25 05:23:08,240 - lava_callback - DEBUG - Processing node 69c0ce2dce0c9bc43bcce96d: baseline-arm-clabbe 2026-03-25 05:23:08,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ce2dce0c9bc43bcce96d 2026-03-25 05:23:08,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:23:08,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ce2dce0c9bc43bcce96d HTTP/1.1" 200 None 2026-03-25 05:23:08,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:23:08,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 05:23:08,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:23:08,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ce2dce0c9bc43bcce96d HTTP/1.1" 200 None 2026-03-25 05:23:08,326 - lava_callback - ERROR - Error processing callback for node 69c0ce2dce0c9bc43bcce96d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43464 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:23:48,185 - lava_callback - INFO - Received callback for node 69c0ce52ce0c9bc43bcce974 INFO: 172.21.0.1:43478 - "POST /node/69c0ce52ce0c9bc43bcce974 HTTP/1.0" 202 Accepted 2026-03-25 05:23:48,190 - lava_callback - INFO - Starting async job submit for node 69c0ce52ce0c9bc43bcce974 2026-03-25 05:23:48,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:23:48,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ce52ce0c9bc43bcce974 HTTP/1.1" 200 None 2026-03-25 05:23:48,233 - lava_callback - DEBUG - Processing node 69c0ce52ce0c9bc43bcce974: baseline-arm-clabbe 2026-03-25 05:23:48,233 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ce52ce0c9bc43bcce974 2026-03-25 05:23:48,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:23:48,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ce52ce0c9bc43bcce974 HTTP/1.1" 200 None 2026-03-25 05:23:48,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:23:48,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 05:23:48,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:23:48,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ce52ce0c9bc43bcce974 HTTP/1.1" 200 None 2026-03-25 05:23:48,313 - lava_callback - ERROR - Error processing callback for node 69c0ce52ce0c9bc43bcce974: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45158 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:24:28,303 - lava_callback - INFO - Received callback for node 69c0ce76ce0c9bc43bcce983 INFO: 172.21.0.1:45172 - "POST /node/69c0ce76ce0c9bc43bcce983 HTTP/1.0" 202 Accepted 2026-03-25 05:24:28,308 - lava_callback - INFO - Starting async job submit for node 69c0ce76ce0c9bc43bcce983 2026-03-25 05:24:28,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:24:28,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ce76ce0c9bc43bcce983 HTTP/1.1" 200 None 2026-03-25 05:24:28,325 - lava_callback - DEBUG - Processing node 69c0ce76ce0c9bc43bcce983: baseline-arm-clabbe 2026-03-25 05:24:28,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ce76ce0c9bc43bcce983 2026-03-25 05:24:28,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:24:28,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ce76ce0c9bc43bcce983 HTTP/1.1" 200 None 2026-03-25 05:24:28,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:24:28,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 05:24:28,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:24:28,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ce76ce0c9bc43bcce983 HTTP/1.1" 200 None 2026-03-25 05:24:28,385 - lava_callback - ERROR - Error processing callback for node 69c0ce76ce0c9bc43bcce983: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38686 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:24:48,235 - lava_callback - INFO - Received callback for node 69c0ce9dce0c9bc43bcce98f INFO: 172.21.0.1:38692 - "POST /node/69c0ce9dce0c9bc43bcce98f HTTP/1.0" 202 Accepted 2026-03-25 05:24:48,241 - lava_callback - INFO - Starting async job submit for node 69c0ce9dce0c9bc43bcce98f 2026-03-25 05:24:48,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:24:48,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ce9dce0c9bc43bcce98f HTTP/1.1" 200 None 2026-03-25 05:24:48,257 - lava_callback - DEBUG - Processing node 69c0ce9dce0c9bc43bcce98f: baseline-arm-clabbe 2026-03-25 05:24:48,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ce9dce0c9bc43bcce98f 2026-03-25 05:24:48,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:24:48,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ce9dce0c9bc43bcce98f HTTP/1.1" 200 None 2026-03-25 05:24:48,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:24:48,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 05:24:48,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:24:48,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ce9dce0c9bc43bcce98f HTTP/1.1" 200 None 2026-03-25 05:24:48,310 - lava_callback - ERROR - Error processing callback for node 69c0ce9dce0c9bc43bcce98f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35954 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:25:28,413 - lava_callback - INFO - Received callback for node 69c0cec2ce0c9bc43bcce99c INFO: 172.21.0.1:35964 - "POST /node/69c0cec2ce0c9bc43bcce99c HTTP/1.0" 202 Accepted 2026-03-25 05:25:28,419 - lava_callback - INFO - Starting async job submit for node 69c0cec2ce0c9bc43bcce99c 2026-03-25 05:25:28,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:25:28,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cec2ce0c9bc43bcce99c HTTP/1.1" 200 None 2026-03-25 05:25:28,434 - lava_callback - DEBUG - Processing node 69c0cec2ce0c9bc43bcce99c: baseline-arm-clabbe 2026-03-25 05:25:28,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cec2ce0c9bc43bcce99c 2026-03-25 05:25:28,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:25:28,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cec2ce0c9bc43bcce99c HTTP/1.1" 200 None 2026-03-25 05:25:28,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:25:28,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 05:25:28,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:25:28,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cec2ce0c9bc43bcce99c HTTP/1.1" 200 None 2026-03-25 05:25:28,489 - lava_callback - ERROR - Error processing callback for node 69c0cec2ce0c9bc43bcce99c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60120 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:26:28,355 - lava_callback - INFO - Received callback for node 69c0cef5ce0c9bc43bcce9a3 INFO: 172.21.0.1:60122 - "POST /node/69c0cef5ce0c9bc43bcce9a3 HTTP/1.0" 202 Accepted 2026-03-25 05:26:28,361 - lava_callback - INFO - Starting async job submit for node 69c0cef5ce0c9bc43bcce9a3 2026-03-25 05:26:28,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:26:28,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cef5ce0c9bc43bcce9a3 HTTP/1.1" 200 None 2026-03-25 05:26:28,377 - lava_callback - DEBUG - Processing node 69c0cef5ce0c9bc43bcce9a3: baseline-arm64-clabbe 2026-03-25 05:26:28,377 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cef5ce0c9bc43bcce9a3 2026-03-25 05:26:28,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:26:28,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cef5ce0c9bc43bcce9a3 HTTP/1.1" 200 None 2026-03-25 05:26:28,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:26:28,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:26:28,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:26:28,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cef5ce0c9bc43bcce9a3 HTTP/1.1" 200 None 2026-03-25 05:26:28,437 - lava_callback - ERROR - Error processing callback for node 69c0cef5ce0c9bc43bcce9a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51688 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:27:48,423 - lava_callback - INFO - Received callback for node 69c0cf41ce0c9bc43bcce9b3 INFO: 172.21.0.1:51698 - "POST /node/69c0cf41ce0c9bc43bcce9b3 HTTP/1.0" 202 Accepted 2026-03-25 05:27:48,429 - lava_callback - INFO - Starting async job submit for node 69c0cf41ce0c9bc43bcce9b3 2026-03-25 05:27:48,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:27:48,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cf41ce0c9bc43bcce9b3 HTTP/1.1" 200 None 2026-03-25 05:27:48,445 - lava_callback - DEBUG - Processing node 69c0cf41ce0c9bc43bcce9b3: baseline-arm64-clabbe 2026-03-25 05:27:48,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cf41ce0c9bc43bcce9b3 2026-03-25 05:27:48,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:27:48,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cf41ce0c9bc43bcce9b3 HTTP/1.1" 200 None 2026-03-25 05:27:48,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:27:48,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:27:48,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:27:48,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cf41ce0c9bc43bcce9b3 HTTP/1.1" 200 None 2026-03-25 05:27:48,503 - lava_callback - ERROR - Error processing callback for node 69c0cf41ce0c9bc43bcce9b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49488 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:28:48,483 - lava_callback - INFO - Received callback for node 69c0cf8ece0c9bc43bcce9c0 INFO: 172.21.0.1:49504 - "POST /node/69c0cf8ece0c9bc43bcce9c0 HTTP/1.0" 202 Accepted 2026-03-25 05:28:48,489 - lava_callback - INFO - Starting async job submit for node 69c0cf8ece0c9bc43bcce9c0 2026-03-25 05:28:48,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:28:48,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cf8ece0c9bc43bcce9c0 HTTP/1.1" 200 None 2026-03-25 05:28:48,505 - lava_callback - DEBUG - Processing node 69c0cf8ece0c9bc43bcce9c0: baseline-arm64-clabbe 2026-03-25 05:28:48,505 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cf8ece0c9bc43bcce9c0 2026-03-25 05:28:48,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:28:48,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cf8ece0c9bc43bcce9c0 HTTP/1.1" 200 None 2026-03-25 05:28:48,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:28:48,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:28:48,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:28:48,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cf8ece0c9bc43bcce9c0 HTTP/1.1" 200 None 2026-03-25 05:28:48,580 - lava_callback - ERROR - Error processing callback for node 69c0cf8ece0c9bc43bcce9c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50674 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:30:08,477 - lava_callback - INFO - Received callback for node 69c0cfd9ce0c9bc43bcce9cc INFO: 172.21.0.1:50684 - "POST /node/69c0cfd9ce0c9bc43bcce9cc HTTP/1.0" 202 Accepted 2026-03-25 05:30:08,483 - lava_callback - INFO - Starting async job submit for node 69c0cfd9ce0c9bc43bcce9cc 2026-03-25 05:30:08,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:30:08,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cfd9ce0c9bc43bcce9cc HTTP/1.1" 200 None 2026-03-25 05:30:08,525 - lava_callback - DEBUG - Processing node 69c0cfd9ce0c9bc43bcce9cc: baseline-arm64-clabbe 2026-03-25 05:30:08,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0cfd9ce0c9bc43bcce9cc 2026-03-25 05:30:08,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:30:08,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0cfd9ce0c9bc43bcce9cc HTTP/1.1" 200 None 2026-03-25 05:30:08,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:30:08,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:30:08,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:30:08,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0cfd9ce0c9bc43bcce9cc HTTP/1.1" 200 None 2026-03-25 05:30:08,601 - lava_callback - ERROR - Error processing callback for node 69c0cfd9ce0c9bc43bcce9cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56968 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:31:28,604 - lava_callback - INFO - Received callback for node 69c0d027ce0c9bc43bcce9d8 INFO: 172.21.0.1:56980 - "POST /node/69c0d027ce0c9bc43bcce9d8 HTTP/1.0" 202 Accepted 2026-03-25 05:31:28,609 - lava_callback - INFO - Starting async job submit for node 69c0d027ce0c9bc43bcce9d8 2026-03-25 05:31:28,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:31:28,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d027ce0c9bc43bcce9d8 HTTP/1.1" 200 None 2026-03-25 05:31:28,625 - lava_callback - DEBUG - Processing node 69c0d027ce0c9bc43bcce9d8: baseline-arm64-clabbe 2026-03-25 05:31:28,626 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d027ce0c9bc43bcce9d8 2026-03-25 05:31:28,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:31:28,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d027ce0c9bc43bcce9d8 HTTP/1.1" 200 None 2026-03-25 05:31:28,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:31:28,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:31:28,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:31:28,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d027ce0c9bc43bcce9d8 HTTP/1.1" 200 None 2026-03-25 05:31:28,681 - lava_callback - ERROR - Error processing callback for node 69c0d027ce0c9bc43bcce9d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51988 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:32:48,720 - lava_callback - INFO - Received callback for node 69c0d076ce0c9bc43bcce9e2 INFO: 172.21.0.1:51990 - "POST /node/69c0d076ce0c9bc43bcce9e2 HTTP/1.0" 202 Accepted 2026-03-25 05:32:48,725 - lava_callback - INFO - Starting async job submit for node 69c0d076ce0c9bc43bcce9e2 2026-03-25 05:32:48,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:32:48,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d076ce0c9bc43bcce9e2 HTTP/1.1" 200 None 2026-03-25 05:32:48,746 - lava_callback - DEBUG - Processing node 69c0d076ce0c9bc43bcce9e2: baseline-arm64-clabbe 2026-03-25 05:32:48,746 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d076ce0c9bc43bcce9e2 2026-03-25 05:32:48,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:32:48,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d076ce0c9bc43bcce9e2 HTTP/1.1" 200 None 2026-03-25 05:32:48,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:32:48,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:32:48,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:32:48,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d076ce0c9bc43bcce9e2 HTTP/1.1" 200 None 2026-03-25 05:32:48,810 - lava_callback - ERROR - Error processing callback for node 69c0d076ce0c9bc43bcce9e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56796 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:34:08,705 - lava_callback - INFO - Received callback for node 69c0d0c4ce0c9bc43bcce9f1 INFO: 172.21.0.1:56808 - "POST /node/69c0d0c4ce0c9bc43bcce9f1 HTTP/1.0" 202 Accepted 2026-03-25 05:34:08,711 - lava_callback - INFO - Starting async job submit for node 69c0d0c4ce0c9bc43bcce9f1 2026-03-25 05:34:08,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:34:08,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d0c4ce0c9bc43bcce9f1 HTTP/1.1" 200 None 2026-03-25 05:34:08,728 - lava_callback - DEBUG - Processing node 69c0d0c4ce0c9bc43bcce9f1: baseline-arm64-clabbe 2026-03-25 05:34:08,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d0c4ce0c9bc43bcce9f1 2026-03-25 05:34:08,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:34:08,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d0c4ce0c9bc43bcce9f1 HTTP/1.1" 200 None 2026-03-25 05:34:08,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:34:08,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:34:08,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:34:08,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d0c4ce0c9bc43bcce9f1 HTTP/1.1" 200 None 2026-03-25 05:34:08,789 - lava_callback - ERROR - Error processing callback for node 69c0d0c4ce0c9bc43bcce9f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37228 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:35:28,769 - lava_callback - INFO - Received callback for node 69c0d111ce0c9bc43bcce9f8 INFO: 172.21.0.1:37238 - "POST /node/69c0d111ce0c9bc43bcce9f8 HTTP/1.0" 202 Accepted 2026-03-25 05:35:28,775 - lava_callback - INFO - Starting async job submit for node 69c0d111ce0c9bc43bcce9f8 2026-03-25 05:35:28,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:35:28,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d111ce0c9bc43bcce9f8 HTTP/1.1" 200 None 2026-03-25 05:35:28,793 - lava_callback - DEBUG - Processing node 69c0d111ce0c9bc43bcce9f8: baseline-arm64-clabbe 2026-03-25 05:35:28,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d111ce0c9bc43bcce9f8 2026-03-25 05:35:28,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:35:28,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d111ce0c9bc43bcce9f8 HTTP/1.1" 200 None 2026-03-25 05:35:28,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:35:28,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:35:28,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:35:28,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d111ce0c9bc43bcce9f8 HTTP/1.1" 200 None 2026-03-25 05:35:28,892 - lava_callback - ERROR - Error processing callback for node 69c0d111ce0c9bc43bcce9f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47794 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:36:48,867 - lava_callback - INFO - Received callback for node 69c0d15ece0c9bc43bccea02 INFO: 172.21.0.1:47802 - "POST /node/69c0d15ece0c9bc43bccea02 HTTP/1.0" 202 Accepted 2026-03-25 05:36:48,873 - lava_callback - INFO - Starting async job submit for node 69c0d15ece0c9bc43bccea02 2026-03-25 05:36:48,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:36:48,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d15ece0c9bc43bccea02 HTTP/1.1" 200 None 2026-03-25 05:36:48,905 - lava_callback - DEBUG - Processing node 69c0d15ece0c9bc43bccea02: baseline-arm64-clabbe 2026-03-25 05:36:48,905 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d15ece0c9bc43bccea02 2026-03-25 05:36:48,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:36:48,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d15ece0c9bc43bccea02 HTTP/1.1" 200 None 2026-03-25 05:36:48,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:36:48,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:36:48,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:36:48,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d15ece0c9bc43bccea02 HTTP/1.1" 200 None 2026-03-25 05:36:48,978 - lava_callback - ERROR - Error processing callback for node 69c0d15ece0c9bc43bccea02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39400 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:38:08,899 - lava_callback - INFO - Received callback for node 69c0d1adce0c9bc43bccea10 INFO: 172.21.0.1:39412 - "POST /node/69c0d1adce0c9bc43bccea10 HTTP/1.0" 202 Accepted 2026-03-25 05:38:08,904 - lava_callback - INFO - Starting async job submit for node 69c0d1adce0c9bc43bccea10 2026-03-25 05:38:08,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:38:08,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d1adce0c9bc43bccea10 HTTP/1.1" 200 None 2026-03-25 05:38:08,928 - lava_callback - DEBUG - Processing node 69c0d1adce0c9bc43bccea10: baseline-arm64-clabbe 2026-03-25 05:38:08,928 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d1adce0c9bc43bccea10 2026-03-25 05:38:08,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:38:08,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d1adce0c9bc43bccea10 HTTP/1.1" 200 None 2026-03-25 05:38:08,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:38:08,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:38:08,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:38:08,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d1adce0c9bc43bccea10 HTTP/1.1" 200 None 2026-03-25 05:38:08,991 - lava_callback - ERROR - Error processing callback for node 69c0d1adce0c9bc43bccea10: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49920 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:39:09,027 - lava_callback - INFO - Received callback for node 69c0d1f9ce0c9bc43bccea1a INFO: 172.21.0.1:49930 - "POST /node/69c0d1f9ce0c9bc43bccea1a HTTP/1.0" 202 Accepted 2026-03-25 05:39:09,032 - lava_callback - INFO - Starting async job submit for node 69c0d1f9ce0c9bc43bccea1a 2026-03-25 05:39:09,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:39:09,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d1f9ce0c9bc43bccea1a HTTP/1.1" 200 None 2026-03-25 05:39:09,072 - lava_callback - DEBUG - Processing node 69c0d1f9ce0c9bc43bccea1a: baseline-arm64-clabbe 2026-03-25 05:39:09,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d1f9ce0c9bc43bccea1a 2026-03-25 05:39:09,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:39:09,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d1f9ce0c9bc43bccea1a HTTP/1.1" 200 None 2026-03-25 05:39:09,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:39:09,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:39:09,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:39:09,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d1f9ce0c9bc43bccea1a HTTP/1.1" 200 None 2026-03-25 05:39:09,181 - lava_callback - ERROR - Error processing callback for node 69c0d1f9ce0c9bc43bccea1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49356 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:40:28,947 - lava_callback - INFO - Received callback for node 69c0d249ce0c9bc43bccea27 INFO: 172.21.0.1:49360 - "POST /node/69c0d249ce0c9bc43bccea27 HTTP/1.0" 202 Accepted 2026-03-25 05:40:28,953 - lava_callback - INFO - Starting async job submit for node 69c0d249ce0c9bc43bccea27 2026-03-25 05:40:28,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:40:28,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d249ce0c9bc43bccea27 HTTP/1.1" 200 None 2026-03-25 05:40:28,969 - lava_callback - DEBUG - Processing node 69c0d249ce0c9bc43bccea27: baseline-arm64-clabbe 2026-03-25 05:40:28,969 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d249ce0c9bc43bccea27 2026-03-25 05:40:28,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:40:28,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d249ce0c9bc43bccea27 HTTP/1.1" 200 None 2026-03-25 05:40:28,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:40:29,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-25 05:40:29,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:40:29,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d249ce0c9bc43bccea27 HTTP/1.1" 200 None 2026-03-25 05:40:29,069 - lava_callback - ERROR - Error processing callback for node 69c0d249ce0c9bc43bccea27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49754 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:48:08,013 - lava_callback - INFO - Received callback for node 69c0d407ce0c9bc43bccea6e INFO: 172.21.0.1:49760 - "POST /node/69c0d407ce0c9bc43bccea6e HTTP/1.0" 202 Accepted 2026-03-25 05:48:08,019 - lava_callback - INFO - Starting async job submit for node 69c0d407ce0c9bc43bccea6e 2026-03-25 05:48:08,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:48:08,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d407ce0c9bc43bccea6e HTTP/1.1" 200 None 2026-03-25 05:48:08,059 - lava_callback - DEBUG - Processing node 69c0d407ce0c9bc43bccea6e: baseline-arm64-baylibre 2026-03-25 05:48:08,060 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d407ce0c9bc43bccea6e 2026-03-25 05:48:08,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:48:08,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d407ce0c9bc43bccea6e HTTP/1.1" 200 None 2026-03-25 05:48:08,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:48:08,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 05:48:08,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:48:08,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d407ce0c9bc43bccea6e HTTP/1.1" 200 None 2026-03-25 05:48:08,116 - lava_callback - ERROR - Error processing callback for node 69c0d407ce0c9bc43bccea6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59748 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:49:09,304 - lava_callback - INFO - Received callback for node 69c0d44fce0c9bc43bccea77 INFO: 172.21.0.1:59762 - "POST /node/69c0d44fce0c9bc43bccea77 HTTP/1.0" 202 Accepted 2026-03-25 05:49:09,309 - lava_callback - INFO - Starting async job submit for node 69c0d44fce0c9bc43bccea77 2026-03-25 05:49:09,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:49:09,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d44fce0c9bc43bccea77 HTTP/1.1" 200 None 2026-03-25 05:49:09,350 - lava_callback - DEBUG - Processing node 69c0d44fce0c9bc43bccea77: baseline-arm-clabbe 2026-03-25 05:49:09,350 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d44fce0c9bc43bccea77 2026-03-25 05:49:09,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:49:09,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d44fce0c9bc43bccea77 HTTP/1.1" 200 None 2026-03-25 05:49:09,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:49:09,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 05:49:09,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:49:09,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d44fce0c9bc43bccea77 HTTP/1.1" 200 None 2026-03-25 05:49:09,434 - lava_callback - ERROR - Error processing callback for node 69c0d44fce0c9bc43bccea77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35940 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:49:49,294 - lava_callback - INFO - Received callback for node 69c0d474ce0c9bc43bccea82 INFO: 172.21.0.1:35956 - "POST /node/69c0d474ce0c9bc43bccea82 HTTP/1.0" 202 Accepted 2026-03-25 05:49:49,299 - lava_callback - INFO - Starting async job submit for node 69c0d474ce0c9bc43bccea82 2026-03-25 05:49:49,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:49:49,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d474ce0c9bc43bccea82 HTTP/1.1" 200 None 2026-03-25 05:49:49,316 - lava_callback - DEBUG - Processing node 69c0d474ce0c9bc43bccea82: baseline-arm-clabbe 2026-03-25 05:49:49,316 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d474ce0c9bc43bccea82 2026-03-25 05:49:49,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:49:49,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d474ce0c9bc43bccea82 HTTP/1.1" 200 None 2026-03-25 05:49:49,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:49:49,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 05:49:49,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:49:49,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d474ce0c9bc43bccea82 HTTP/1.1" 200 None 2026-03-25 05:49:49,405 - lava_callback - ERROR - Error processing callback for node 69c0d474ce0c9bc43bccea82: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41052 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:50:28,040 - lava_callback - INFO - Received callback for node 69c0d4a0ce0c9bc43bccea8d INFO: 172.21.0.1:41054 - "POST /node/69c0d4a0ce0c9bc43bccea8d HTTP/1.0" 202 Accepted 2026-03-25 05:50:28,046 - lava_callback - INFO - Starting async job submit for node 69c0d4a0ce0c9bc43bccea8d 2026-03-25 05:50:28,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:50:28,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d4a0ce0c9bc43bccea8d HTTP/1.1" 200 None 2026-03-25 05:50:28,081 - lava_callback - DEBUG - Processing node 69c0d4a0ce0c9bc43bccea8d: baseline-arm64-baylibre 2026-03-25 05:50:28,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d4a0ce0c9bc43bccea8d 2026-03-25 05:50:28,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:50:28,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d4a0ce0c9bc43bccea8d HTTP/1.1" 200 None 2026-03-25 05:50:28,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:50:28,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 05:50:28,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:50:28,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d4a0ce0c9bc43bccea8d HTTP/1.1" 200 None 2026-03-25 05:50:28,169 - lava_callback - ERROR - Error processing callback for node 69c0d4a0ce0c9bc43bccea8d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47940 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:51:48,094 - lava_callback - INFO - Received callback for node 69c0d4ebce0c9bc43bccea99 INFO: 172.21.0.1:47946 - "POST /node/69c0d4ebce0c9bc43bccea99 HTTP/1.0" 202 Accepted 2026-03-25 05:51:48,100 - lava_callback - INFO - Starting async job submit for node 69c0d4ebce0c9bc43bccea99 2026-03-25 05:51:48,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:51:48,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d4ebce0c9bc43bccea99 HTTP/1.1" 200 None 2026-03-25 05:51:48,127 - lava_callback - DEBUG - Processing node 69c0d4ebce0c9bc43bccea99: baseline-arm64-baylibre 2026-03-25 05:51:48,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d4ebce0c9bc43bccea99 2026-03-25 05:51:48,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:51:48,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d4ebce0c9bc43bccea99 HTTP/1.1" 200 None 2026-03-25 05:51:48,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:51:48,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 05:51:48,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:51:48,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d4ebce0c9bc43bccea99 HTTP/1.1" 200 None 2026-03-25 05:51:48,182 - lava_callback - ERROR - Error processing callback for node 69c0d4ebce0c9bc43bccea99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:53:08,171 - lava_callback - INFO - Received callback for node 69c0d538ce0c9bc43bcceaab INFO: 172.21.0.1:35012 - "POST /node/69c0d538ce0c9bc43bcceaab HTTP/1.0" 202 Accepted 2026-03-25 05:53:08,176 - lava_callback - INFO - Starting async job submit for node 69c0d538ce0c9bc43bcceaab 2026-03-25 05:53:08,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:53:08,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d538ce0c9bc43bcceaab HTTP/1.1" 200 None 2026-03-25 05:53:08,201 - lava_callback - DEBUG - Processing node 69c0d538ce0c9bc43bcceaab: baseline-arm64-baylibre 2026-03-25 05:53:08,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d538ce0c9bc43bcceaab 2026-03-25 05:53:08,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:53:08,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d538ce0c9bc43bcceaab HTTP/1.1" 200 None 2026-03-25 05:53:08,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:53:08,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 05:53:08,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:53:08,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d538ce0c9bc43bcceaab HTTP/1.1" 200 None 2026-03-25 05:53:08,256 - lava_callback - ERROR - Error processing callback for node 69c0d538ce0c9bc43bcceaab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45314 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:54:29,486 - lava_callback - INFO - Received callback for node 69c0d57fce0c9bc43bcceab4 INFO: 172.21.0.1:45316 - "POST /node/69c0d57fce0c9bc43bcceab4 HTTP/1.0" 202 Accepted 2026-03-25 05:54:29,492 - lava_callback - INFO - Starting async job submit for node 69c0d57fce0c9bc43bcceab4 2026-03-25 05:54:29,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:54:29,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d57fce0c9bc43bcceab4 HTTP/1.1" 200 None 2026-03-25 05:54:29,509 - lava_callback - DEBUG - Processing node 69c0d57fce0c9bc43bcceab4: baseline-arm-clabbe 2026-03-25 05:54:29,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d57fce0c9bc43bcceab4 2026-03-25 05:54:29,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:54:29,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d57fce0c9bc43bcceab4 HTTP/1.1" 200 None 2026-03-25 05:54:29,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:54:29,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 05:54:29,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:54:29,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d57fce0c9bc43bcceab4 HTTP/1.1" 200 None 2026-03-25 05:54:29,601 - lava_callback - ERROR - Error processing callback for node 69c0d57fce0c9bc43bcceab4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45534 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:55:08,291 - lava_callback - INFO - Received callback for node 69c0d5abce0c9bc43bcceac4 INFO: 172.21.0.1:45546 - "POST /node/69c0d5abce0c9bc43bcceac4 HTTP/1.0" 202 Accepted 2026-03-25 05:55:08,297 - lava_callback - INFO - Starting async job submit for node 69c0d5abce0c9bc43bcceac4 2026-03-25 05:55:08,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:55:08,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d5abce0c9bc43bcceac4 HTTP/1.1" 200 None 2026-03-25 05:55:08,323 - lava_callback - DEBUG - Processing node 69c0d5abce0c9bc43bcceac4: baseline-arm64-baylibre 2026-03-25 05:55:08,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d5abce0c9bc43bcceac4 2026-03-25 05:55:08,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:55:08,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d5abce0c9bc43bcceac4 HTTP/1.1" 200 None 2026-03-25 05:55:08,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:55:08,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 05:55:08,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:55:08,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d5abce0c9bc43bcceac4 HTTP/1.1" 200 None 2026-03-25 05:55:08,375 - lava_callback - ERROR - Error processing callback for node 69c0d5abce0c9bc43bcceac4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51942 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:56:28,373 - lava_callback - INFO - Received callback for node 69c0d5f7ce0c9bc43bcceaca INFO: 172.21.0.1:51946 - "POST /node/69c0d5f7ce0c9bc43bcceaca HTTP/1.0" 202 Accepted 2026-03-25 05:56:28,379 - lava_callback - INFO - Starting async job submit for node 69c0d5f7ce0c9bc43bcceaca 2026-03-25 05:56:28,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:56:28,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d5f7ce0c9bc43bcceaca HTTP/1.1" 200 None 2026-03-25 05:56:28,401 - lava_callback - DEBUG - Processing node 69c0d5f7ce0c9bc43bcceaca: baseline-arm64-baylibre 2026-03-25 05:56:28,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d5f7ce0c9bc43bcceaca 2026-03-25 05:56:28,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:56:28,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d5f7ce0c9bc43bcceaca HTTP/1.1" 200 None 2026-03-25 05:56:28,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:56:28,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 05:56:28,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:56:28,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d5f7ce0c9bc43bcceaca HTTP/1.1" 200 None 2026-03-25 05:56:28,471 - lava_callback - ERROR - Error processing callback for node 69c0d5f7ce0c9bc43bcceaca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33654 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:57:29,598 - lava_callback - INFO - Received callback for node 69c0d63bce0c9bc43bccead5 INFO: 172.21.0.1:33656 - "POST /node/69c0d63bce0c9bc43bccead5 HTTP/1.0" 202 Accepted 2026-03-25 05:57:29,606 - lava_callback - INFO - Starting async job submit for node 69c0d63bce0c9bc43bccead5 2026-03-25 05:57:29,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:57:29,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d63bce0c9bc43bccead5 HTTP/1.1" 200 None 2026-03-25 05:57:29,622 - lava_callback - DEBUG - Processing node 69c0d63bce0c9bc43bccead5: baseline-arm-clabbe 2026-03-25 05:57:29,623 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d63bce0c9bc43bccead5 2026-03-25 05:57:29,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:57:29,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d63bce0c9bc43bccead5 HTTP/1.1" 200 None 2026-03-25 05:57:29,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:57:29,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 05:57:29,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:57:29,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d63bce0c9bc43bccead5 HTTP/1.1" 200 None 2026-03-25 05:57:29,678 - lava_callback - ERROR - Error processing callback for node 69c0d63bce0c9bc43bccead5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55664 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:58:29,618 - lava_callback - INFO - Received callback for node 69c0d66fce0c9bc43bcceae4 INFO: 172.21.0.1:55670 - "POST /node/69c0d66fce0c9bc43bcceae4 HTTP/1.0" 202 Accepted 2026-03-25 05:58:29,625 - lava_callback - INFO - Starting async job submit for node 69c0d66fce0c9bc43bcceae4 2026-03-25 05:58:29,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:58:29,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d66fce0c9bc43bcceae4 HTTP/1.1" 200 None 2026-03-25 05:58:29,665 - lava_callback - DEBUG - Processing node 69c0d66fce0c9bc43bcceae4: baseline-arm64-clabbe 2026-03-25 05:58:29,666 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d66fce0c9bc43bcceae4 2026-03-25 05:58:29,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:58:29,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d66fce0c9bc43bcceae4 HTTP/1.1" 200 None 2026-03-25 05:58:29,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:58:29,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 05:58:29,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:58:29,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d66fce0c9bc43bcceae4 HTTP/1.1" 200 None 2026-03-25 05:58:29,750 - lava_callback - ERROR - Error processing callback for node 69c0d66fce0c9bc43bcceae4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52630 - "GET / HTTP/1.0" 200 OK 2026-03-25 05:59:29,701 - lava_callback - INFO - Received callback for node 69c0d6acce0c9bc43bcceaed INFO: 172.21.0.1:36350 - "POST /node/69c0d6acce0c9bc43bcceaed HTTP/1.0" 202 Accepted 2026-03-25 05:59:29,708 - lava_callback - INFO - Starting async job submit for node 69c0d6acce0c9bc43bcceaed 2026-03-25 05:59:29,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:59:29,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d6acce0c9bc43bcceaed HTTP/1.1" 200 None 2026-03-25 05:59:29,723 - lava_callback - DEBUG - Processing node 69c0d6acce0c9bc43bcceaed: baseline-arm-clabbe 2026-03-25 05:59:29,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d6acce0c9bc43bcceaed 2026-03-25 05:59:29,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:59:29,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d6acce0c9bc43bcceaed HTTP/1.1" 200 None 2026-03-25 05:59:29,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:59:29,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 05:59:29,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 05:59:29,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d6acce0c9bc43bcceaed HTTP/1.1" 200 None 2026-03-25 05:59:29,792 - lava_callback - ERROR - Error processing callback for node 69c0d6acce0c9bc43bcceaed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54704 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:00:09,817 - lava_callback - INFO - Received callback for node 69c0d6dfce0c9bc43bcceafd INFO: 172.21.0.1:34170 - "POST /node/69c0d6dfce0c9bc43bcceafd HTTP/1.0" 202 Accepted 2026-03-25 06:00:09,822 - lava_callback - INFO - Starting async job submit for node 69c0d6dfce0c9bc43bcceafd 2026-03-25 06:00:09,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:00:09,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d6dfce0c9bc43bcceafd HTTP/1.1" 200 None 2026-03-25 06:00:09,846 - lava_callback - DEBUG - Processing node 69c0d6dfce0c9bc43bcceafd: baseline-arm64-clabbe 2026-03-25 06:00:09,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d6dfce0c9bc43bcceafd 2026-03-25 06:00:09,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:00:09,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d6dfce0c9bc43bcceafd HTTP/1.1" 200 None 2026-03-25 06:00:09,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:00:09,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 06:00:09,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:00:09,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d6dfce0c9bc43bcceafd HTTP/1.1" 200 None 2026-03-25 06:00:09,907 - lava_callback - ERROR - Error processing callback for node 69c0d6dfce0c9bc43bcceafd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45236 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:01:29,851 - lava_callback - INFO - Received callback for node 69c0d72cce0c9bc43bcceb03 INFO: 172.21.0.1:40102 - "POST /node/69c0d72cce0c9bc43bcceb03 HTTP/1.0" 202 Accepted 2026-03-25 06:01:29,857 - lava_callback - INFO - Starting async job submit for node 69c0d72cce0c9bc43bcceb03 2026-03-25 06:01:29,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:01:29,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d72cce0c9bc43bcceb03 HTTP/1.1" 200 None 2026-03-25 06:01:29,873 - lava_callback - DEBUG - Processing node 69c0d72cce0c9bc43bcceb03: baseline-arm64-clabbe 2026-03-25 06:01:29,873 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d72cce0c9bc43bcceb03 2026-03-25 06:01:29,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:01:29,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d72cce0c9bc43bcceb03 HTTP/1.1" 200 None 2026-03-25 06:01:29,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:01:29,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 06:01:29,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:01:29,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d72cce0c9bc43bcceb03 HTTP/1.1" 200 None 2026-03-25 06:01:29,934 - lava_callback - ERROR - Error processing callback for node 69c0d72cce0c9bc43bcceb03: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48280 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:02:49,908 - lava_callback - INFO - Received callback for node 69c0d778ce0c9bc43bcceb0f INFO: 172.21.0.1:54426 - "POST /node/69c0d778ce0c9bc43bcceb0f HTTP/1.0" 202 Accepted 2026-03-25 06:02:49,913 - lava_callback - INFO - Starting async job submit for node 69c0d778ce0c9bc43bcceb0f 2026-03-25 06:02:49,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:02:49,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d778ce0c9bc43bcceb0f HTTP/1.1" 200 None 2026-03-25 06:02:49,929 - lava_callback - DEBUG - Processing node 69c0d778ce0c9bc43bcceb0f: baseline-arm64-clabbe 2026-03-25 06:02:49,929 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d778ce0c9bc43bcceb0f 2026-03-25 06:02:49,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:02:49,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d778ce0c9bc43bcceb0f HTTP/1.1" 200 None 2026-03-25 06:02:49,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:02:49,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 06:02:49,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:02:49,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d778ce0c9bc43bcceb0f HTTP/1.1" 200 None 2026-03-25 06:02:49,981 - lava_callback - ERROR - Error processing callback for node 69c0d778ce0c9bc43bcceb0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52264 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:04:09,990 - lava_callback - INFO - Received callback for node 69c0d7c4ce0c9bc43bcceb1c INFO: 172.21.0.1:60338 - "POST /node/69c0d7c4ce0c9bc43bcceb1c HTTP/1.0" 202 Accepted 2026-03-25 06:04:09,996 - lava_callback - INFO - Starting async job submit for node 69c0d7c4ce0c9bc43bcceb1c 2026-03-25 06:04:09,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:04:10,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d7c4ce0c9bc43bcceb1c HTTP/1.1" 200 None 2026-03-25 06:04:10,011 - lava_callback - DEBUG - Processing node 69c0d7c4ce0c9bc43bcceb1c: baseline-arm64-clabbe 2026-03-25 06:04:10,012 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d7c4ce0c9bc43bcceb1c 2026-03-25 06:04:10,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:04:10,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d7c4ce0c9bc43bcceb1c HTTP/1.1" 200 None 2026-03-25 06:04:10,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:04:10,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 06:04:10,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:04:10,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d7c4ce0c9bc43bcceb1c HTTP/1.1" 200 None 2026-03-25 06:04:10,109 - lava_callback - ERROR - Error processing callback for node 69c0d7c4ce0c9bc43bcceb1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48806 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:05:10,121 - lava_callback - INFO - Received callback for node 69c0d80fce0c9bc43bcceb28 INFO: 172.21.0.1:49576 - "POST /node/69c0d80fce0c9bc43bcceb28 HTTP/1.0" 202 Accepted 2026-03-25 06:05:10,127 - lava_callback - INFO - Starting async job submit for node 69c0d80fce0c9bc43bcceb28 2026-03-25 06:05:10,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:05:10,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d80fce0c9bc43bcceb28 HTTP/1.1" 200 None 2026-03-25 06:05:10,154 - lava_callback - DEBUG - Processing node 69c0d80fce0c9bc43bcceb28: baseline-arm64-clabbe 2026-03-25 06:05:10,155 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d80fce0c9bc43bcceb28 2026-03-25 06:05:10,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:05:10,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d80fce0c9bc43bcceb28 HTTP/1.1" 200 None 2026-03-25 06:05:10,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:05:10,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-25 06:05:10,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:05:10,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d80fce0c9bc43bcceb28 HTTP/1.1" 200 None 2026-03-25 06:05:10,221 - lava_callback - ERROR - Error processing callback for node 69c0d80fce0c9bc43bcceb28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49756 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:06:09,976 - lava_callback - INFO - Received callback for node 69c0d84dce0c9bc43bcceb30 INFO: 172.21.0.1:44032 - "POST /node/69c0d84dce0c9bc43bcceb30 HTTP/1.0" 202 Accepted 2026-03-25 06:06:09,982 - lava_callback - INFO - Starting async job submit for node 69c0d84dce0c9bc43bcceb30 2026-03-25 06:06:09,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:06:09,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d84dce0c9bc43bcceb30 HTTP/1.1" 200 None 2026-03-25 06:06:09,997 - lava_callback - DEBUG - Processing node 69c0d84dce0c9bc43bcceb30: baseline-arm-clabbe 2026-03-25 06:06:09,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d84dce0c9bc43bcceb30 2026-03-25 06:06:09,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:06:10,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d84dce0c9bc43bcceb30 HTTP/1.1" 200 None 2026-03-25 06:06:10,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:06:10,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 06:06:10,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:06:10,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d84dce0c9bc43bcceb30 HTTP/1.1" 200 None 2026-03-25 06:06:10,065 - lava_callback - ERROR - Error processing callback for node 69c0d84dce0c9bc43bcceb30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56664 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:06:49,907 - lava_callback - INFO - Received callback for node 69c0d872ce0c9bc43bcceb41 INFO: 172.21.0.1:54490 - "POST /node/69c0d872ce0c9bc43bcceb41 HTTP/1.0" 202 Accepted 2026-03-25 06:06:49,913 - lava_callback - INFO - Starting async job submit for node 69c0d872ce0c9bc43bcceb41 2026-03-25 06:06:49,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:06:49,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d872ce0c9bc43bcceb41 HTTP/1.1" 200 None 2026-03-25 06:06:49,928 - lava_callback - DEBUG - Processing node 69c0d872ce0c9bc43bcceb41: baseline-arm-clabbe 2026-03-25 06:06:49,928 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d872ce0c9bc43bcceb41 2026-03-25 06:06:49,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:06:49,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d872ce0c9bc43bcceb41 HTTP/1.1" 200 None 2026-03-25 06:06:49,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:06:49,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 06:06:49,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:06:49,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d872ce0c9bc43bcceb41 HTTP/1.1" 200 None 2026-03-25 06:06:49,992 - lava_callback - ERROR - Error processing callback for node 69c0d872ce0c9bc43bcceb41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38132 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:07:30,044 - lava_callback - INFO - Received callback for node 69c0d896ce0c9bc43bcceb4d INFO: 172.21.0.1:56678 - "POST /node/69c0d896ce0c9bc43bcceb4d HTTP/1.0" 202 Accepted 2026-03-25 06:07:30,050 - lava_callback - INFO - Starting async job submit for node 69c0d896ce0c9bc43bcceb4d 2026-03-25 06:07:30,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:07:30,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d896ce0c9bc43bcceb4d HTTP/1.1" 200 None 2026-03-25 06:07:30,066 - lava_callback - DEBUG - Processing node 69c0d896ce0c9bc43bcceb4d: baseline-arm-clabbe 2026-03-25 06:07:30,066 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d896ce0c9bc43bcceb4d 2026-03-25 06:07:30,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:07:30,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d896ce0c9bc43bcceb4d HTTP/1.1" 200 None 2026-03-25 06:07:30,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:07:30,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 06:07:30,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:07:30,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d896ce0c9bc43bcceb4d HTTP/1.1" 200 None 2026-03-25 06:07:30,132 - lava_callback - ERROR - Error processing callback for node 69c0d896ce0c9bc43bcceb4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35538 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:08:10,095 - lava_callback - INFO - Received callback for node 69c0d8bace0c9bc43bcceb58 INFO: 172.21.0.1:50042 - "POST /node/69c0d8bace0c9bc43bcceb58 HTTP/1.0" 202 Accepted 2026-03-25 06:08:10,101 - lava_callback - INFO - Starting async job submit for node 69c0d8bace0c9bc43bcceb58 2026-03-25 06:08:10,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:08:10,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d8bace0c9bc43bcceb58 HTTP/1.1" 200 None 2026-03-25 06:08:10,117 - lava_callback - DEBUG - Processing node 69c0d8bace0c9bc43bcceb58: baseline-arm-clabbe 2026-03-25 06:08:10,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d8bace0c9bc43bcceb58 2026-03-25 06:08:10,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:08:10,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d8bace0c9bc43bcceb58 HTTP/1.1" 200 None 2026-03-25 06:08:10,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:08:10,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 06:08:10,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:08:10,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d8bace0c9bc43bcceb58 HTTP/1.1" 200 None 2026-03-25 06:08:10,184 - lava_callback - ERROR - Error processing callback for node 69c0d8bace0c9bc43bcceb58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45376 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:08:50,145 - lava_callback - INFO - Received callback for node 69c0d8dfce0c9bc43bcceb60 INFO: 172.21.0.1:53240 - "POST /node/69c0d8dfce0c9bc43bcceb60 HTTP/1.0" 202 Accepted 2026-03-25 06:08:50,150 - lava_callback - INFO - Starting async job submit for node 69c0d8dfce0c9bc43bcceb60 2026-03-25 06:08:50,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:08:50,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d8dfce0c9bc43bcceb60 HTTP/1.1" 200 None 2026-03-25 06:08:50,166 - lava_callback - DEBUG - Processing node 69c0d8dfce0c9bc43bcceb60: baseline-arm-clabbe 2026-03-25 06:08:50,167 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0d8dfce0c9bc43bcceb60 2026-03-25 06:08:50,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:08:50,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0d8dfce0c9bc43bcceb60 HTTP/1.1" 200 None 2026-03-25 06:08:50,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:08:50,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-25 06:08:50,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:08:50,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0d8dfce0c9bc43bcceb60 HTTP/1.1" 200 None 2026-03-25 06:08:50,238 - lava_callback - ERROR - Error processing callback for node 69c0d8dfce0c9bc43bcceb60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48550 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:40:11,609 - lava_callback - INFO - Received callback for node 69c0e037ce0c9bc43bcceb8e INFO: 172.21.0.1:50462 - "POST /node/69c0e037ce0c9bc43bcceb8e HTTP/1.0" 202 Accepted 2026-03-25 06:40:11,614 - lava_callback - INFO - Starting async job submit for node 69c0e037ce0c9bc43bcceb8e 2026-03-25 06:40:11,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:40:11,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e037ce0c9bc43bcceb8e HTTP/1.1" 200 None 2026-03-25 06:40:11,630 - lava_callback - DEBUG - Processing node 69c0e037ce0c9bc43bcceb8e: baseline-arm-clabbe 2026-03-25 06:40:11,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e037ce0c9bc43bcceb8e 2026-03-25 06:40:11,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:40:11,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e037ce0c9bc43bcceb8e HTTP/1.1" 200 None 2026-03-25 06:40:11,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:40:11,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 06:40:11,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:40:11,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e037ce0c9bc43bcceb8e HTTP/1.1" 200 None 2026-03-25 06:40:11,696 - lava_callback - ERROR - Error processing callback for node 69c0e037ce0c9bc43bcceb8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58768 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:41:11,587 - lava_callback - INFO - Received callback for node 69c0e072ce0c9bc43bcceb9a INFO: 172.21.0.1:47742 - "POST /node/69c0e072ce0c9bc43bcceb9a HTTP/1.0" 202 Accepted 2026-03-25 06:41:11,592 - lava_callback - INFO - Starting async job submit for node 69c0e072ce0c9bc43bcceb9a 2026-03-25 06:41:11,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:41:11,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e072ce0c9bc43bcceb9a HTTP/1.1" 200 None 2026-03-25 06:41:11,608 - lava_callback - DEBUG - Processing node 69c0e072ce0c9bc43bcceb9a: baseline-arm-clabbe 2026-03-25 06:41:11,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e072ce0c9bc43bcceb9a 2026-03-25 06:41:11,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:41:11,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e072ce0c9bc43bcceb9a HTTP/1.1" 200 None 2026-03-25 06:41:11,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:41:11,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 06:41:11,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:41:11,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e072ce0c9bc43bcceb9a HTTP/1.1" 200 None 2026-03-25 06:41:11,662 - lava_callback - ERROR - Error processing callback for node 69c0e072ce0c9bc43bcceb9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50950 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:42:11,893 - lava_callback - INFO - Received callback for node 69c0e0afce0c9bc43bccebc6 INFO: 172.21.0.1:55456 - "POST /node/69c0e0afce0c9bc43bccebc6 HTTP/1.0" 202 Accepted 2026-03-25 06:42:11,898 - lava_callback - INFO - Starting async job submit for node 69c0e0afce0c9bc43bccebc6 2026-03-25 06:42:11,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:42:11,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e0afce0c9bc43bccebc6 HTTP/1.1" 200 None 2026-03-25 06:42:11,920 - lava_callback - DEBUG - Processing node 69c0e0afce0c9bc43bccebc6: baseline-arm-clabbe 2026-03-25 06:42:11,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e0afce0c9bc43bccebc6 2026-03-25 06:42:11,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:42:11,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e0afce0c9bc43bccebc6 HTTP/1.1" 200 None 2026-03-25 06:42:11,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:42:11,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 06:42:11,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:42:11,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e0afce0c9bc43bccebc6 HTTP/1.1" 200 None 2026-03-25 06:42:11,973 - lava_callback - ERROR - Error processing callback for node 69c0e0afce0c9bc43bccebc6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55894 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:43:51,128 - lava_callback - INFO - Received callback for node 69c0e116ce0c9bc43bccebf9 INFO: 172.21.0.1:34916 - "POST /node/69c0e116ce0c9bc43bccebf9 HTTP/1.0" 202 Accepted 2026-03-25 06:43:51,133 - lava_callback - INFO - Starting async job submit for node 69c0e116ce0c9bc43bccebf9 2026-03-25 06:43:51,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:43:51,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e116ce0c9bc43bccebf9 HTTP/1.1" 200 None 2026-03-25 06:43:51,157 - lava_callback - DEBUG - Processing node 69c0e116ce0c9bc43bccebf9: baseline-arm-baylibre 2026-03-25 06:43:51,157 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e116ce0c9bc43bccebf9 2026-03-25 06:43:51,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:43:51,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e116ce0c9bc43bccebf9 HTTP/1.1" 200 None 2026-03-25 06:43:51,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:43:51,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:43:51,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:43:51,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e116ce0c9bc43bccebf9 HTTP/1.1" 200 None 2026-03-25 06:43:51,225 - lava_callback - ERROR - Error processing callback for node 69c0e116ce0c9bc43bccebf9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41406 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:44:30,992 - lava_callback - INFO - Received callback for node 69c0e13bce0c9bc43bccec04 INFO: 172.21.0.1:50384 - "POST /node/69c0e13bce0c9bc43bccec04 HTTP/1.0" 202 Accepted 2026-03-25 06:44:31,001 - lava_callback - INFO - Starting async job submit for node 69c0e13bce0c9bc43bccec04 2026-03-25 06:44:31,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:44:31,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e13bce0c9bc43bccec04 HTTP/1.1" 200 None 2026-03-25 06:44:31,022 - lava_callback - DEBUG - Processing node 69c0e13bce0c9bc43bccec04: baseline-arm-baylibre 2026-03-25 06:44:31,022 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e13bce0c9bc43bccec04 2026-03-25 06:44:31,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:44:31,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e13bce0c9bc43bccec04 HTTP/1.1" 200 None 2026-03-25 06:44:31,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:44:31,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:44:31,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:44:31,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e13bce0c9bc43bccec04 HTTP/1.1" 200 None 2026-03-25 06:44:31,086 - lava_callback - ERROR - Error processing callback for node 69c0e13bce0c9bc43bccec04: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45400 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:44:51,012 - lava_callback - INFO - Received callback for node 69c0e15fce0c9bc43bccec0e INFO: 172.21.0.1:37748 - "POST /node/69c0e15fce0c9bc43bccec0e HTTP/1.0" 202 Accepted 2026-03-25 06:44:51,017 - lava_callback - INFO - Starting async job submit for node 69c0e15fce0c9bc43bccec0e 2026-03-25 06:44:51,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:44:51,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e15fce0c9bc43bccec0e HTTP/1.1" 200 None 2026-03-25 06:44:51,066 - lava_callback - DEBUG - Processing node 69c0e15fce0c9bc43bccec0e: baseline-arm-baylibre 2026-03-25 06:44:51,066 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e15fce0c9bc43bccec0e 2026-03-25 06:44:51,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:44:51,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e15fce0c9bc43bccec0e HTTP/1.1" 200 None 2026-03-25 06:44:51,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:44:51,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:44:51,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:44:51,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e15fce0c9bc43bccec0e HTTP/1.1" 200 None 2026-03-25 06:44:51,128 - lava_callback - ERROR - Error processing callback for node 69c0e15fce0c9bc43bccec0e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40932 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:45:31,091 - lava_callback - INFO - Received callback for node 69c0e184ce0c9bc43bccec18 INFO: 172.21.0.1:58550 - "POST /node/69c0e184ce0c9bc43bccec18 HTTP/1.0" 202 Accepted 2026-03-25 06:45:31,096 - lava_callback - INFO - Starting async job submit for node 69c0e184ce0c9bc43bccec18 2026-03-25 06:45:31,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:45:31,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e184ce0c9bc43bccec18 HTTP/1.1" 200 None 2026-03-25 06:45:31,114 - lava_callback - DEBUG - Processing node 69c0e184ce0c9bc43bccec18: baseline-arm-baylibre 2026-03-25 06:45:31,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e184ce0c9bc43bccec18 2026-03-25 06:45:31,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:45:31,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e184ce0c9bc43bccec18 HTTP/1.1" 200 None 2026-03-25 06:45:31,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:45:31,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:45:31,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:45:31,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e184ce0c9bc43bccec18 HTTP/1.1" 200 None 2026-03-25 06:45:31,167 - lava_callback - ERROR - Error processing callback for node 69c0e184ce0c9bc43bccec18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41116 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:46:11,095 - lava_callback - INFO - Received callback for node 69c0e1abce0c9bc43bccec21 INFO: 172.21.0.1:41624 - "POST /node/69c0e1abce0c9bc43bccec21 HTTP/1.0" 202 Accepted 2026-03-25 06:46:11,100 - lava_callback - INFO - Starting async job submit for node 69c0e1abce0c9bc43bccec21 2026-03-25 06:46:11,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:46:11,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e1abce0c9bc43bccec21 HTTP/1.1" 200 None 2026-03-25 06:46:11,118 - lava_callback - DEBUG - Processing node 69c0e1abce0c9bc43bccec21: baseline-arm-baylibre 2026-03-25 06:46:11,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e1abce0c9bc43bccec21 2026-03-25 06:46:11,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:46:11,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e1abce0c9bc43bccec21 HTTP/1.1" 200 None 2026-03-25 06:46:11,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:46:11,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:46:11,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:46:11,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e1abce0c9bc43bccec21 HTTP/1.1" 200 None 2026-03-25 06:46:11,173 - lava_callback - ERROR - Error processing callback for node 69c0e1abce0c9bc43bccec21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35678 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:46:51,177 - lava_callback - INFO - Received callback for node 69c0e1cece0c9bc43bccec2a INFO: 172.21.0.1:57458 - "POST /node/69c0e1cece0c9bc43bccec2a HTTP/1.0" 202 Accepted 2026-03-25 06:46:51,183 - lava_callback - INFO - Starting async job submit for node 69c0e1cece0c9bc43bccec2a 2026-03-25 06:46:51,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:46:51,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e1cece0c9bc43bccec2a HTTP/1.1" 200 None 2026-03-25 06:46:51,207 - lava_callback - DEBUG - Processing node 69c0e1cece0c9bc43bccec2a: baseline-arm-baylibre 2026-03-25 06:46:51,207 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e1cece0c9bc43bccec2a 2026-03-25 06:46:51,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:46:51,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e1cece0c9bc43bccec2a HTTP/1.1" 200 None 2026-03-25 06:46:51,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:46:51,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:46:51,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:46:51,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e1cece0c9bc43bccec2a HTTP/1.1" 200 None 2026-03-25 06:46:51,269 - lava_callback - ERROR - Error processing callback for node 69c0e1cece0c9bc43bccec2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:48:31,923 - lava_callback - INFO - Received callback for node 69c0e23dce0c9bc43bccec47 INFO: 172.21.0.1:39320 - "POST /node/69c0e23dce0c9bc43bccec47 HTTP/1.0" 202 Accepted 2026-03-25 06:48:31,929 - lava_callback - INFO - Starting async job submit for node 69c0e23dce0c9bc43bccec47 2026-03-25 06:48:31,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:48:31,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e23dce0c9bc43bccec47 HTTP/1.1" 200 None 2026-03-25 06:48:31,949 - lava_callback - DEBUG - Processing node 69c0e23dce0c9bc43bccec47: baseline-arm-clabbe 2026-03-25 06:48:31,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e23dce0c9bc43bccec47 2026-03-25 06:48:31,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:48:31,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e23dce0c9bc43bccec47 HTTP/1.1" 200 None 2026-03-25 06:48:31,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:48:31,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:48:31,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:48:32,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e23dce0c9bc43bccec47 HTTP/1.1" 200 None 2026-03-25 06:48:32,013 - lava_callback - ERROR - Error processing callback for node 69c0e23dce0c9bc43bccec47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33510 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:49:11,926 - lava_callback - INFO - Received callback for node 69c0e264ce0c9bc43bccec52 INFO: 172.21.0.1:40636 - "POST /node/69c0e264ce0c9bc43bccec52 HTTP/1.0" 202 Accepted 2026-03-25 06:49:11,931 - lava_callback - INFO - Starting async job submit for node 69c0e264ce0c9bc43bccec52 2026-03-25 06:49:11,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:49:11,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e264ce0c9bc43bccec52 HTTP/1.1" 200 None 2026-03-25 06:49:11,948 - lava_callback - DEBUG - Processing node 69c0e264ce0c9bc43bccec52: baseline-arm-clabbe 2026-03-25 06:49:11,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e264ce0c9bc43bccec52 2026-03-25 06:49:11,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:49:11,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e264ce0c9bc43bccec52 HTTP/1.1" 200 None 2026-03-25 06:49:11,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:49:11,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:49:11,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:49:12,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e264ce0c9bc43bccec52 HTTP/1.1" 200 None 2026-03-25 06:49:12,011 - lava_callback - ERROR - Error processing callback for node 69c0e264ce0c9bc43bccec52: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41660 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:49:51,991 - lava_callback - INFO - Received callback for node 69c0e289ce0c9bc43bccec5d INFO: 172.21.0.1:40218 - "POST /node/69c0e289ce0c9bc43bccec5d HTTP/1.0" 202 Accepted 2026-03-25 06:49:51,996 - lava_callback - INFO - Starting async job submit for node 69c0e289ce0c9bc43bccec5d 2026-03-25 06:49:51,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:49:52,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e289ce0c9bc43bccec5d HTTP/1.1" 200 None 2026-03-25 06:49:52,013 - lava_callback - DEBUG - Processing node 69c0e289ce0c9bc43bccec5d: baseline-arm-clabbe 2026-03-25 06:49:52,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e289ce0c9bc43bccec5d 2026-03-25 06:49:52,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:49:52,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e289ce0c9bc43bccec5d HTTP/1.1" 200 None 2026-03-25 06:49:52,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:49:52,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:49:52,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:49:52,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e289ce0c9bc43bccec5d HTTP/1.1" 200 None 2026-03-25 06:49:52,088 - lava_callback - ERROR - Error processing callback for node 69c0e289ce0c9bc43bccec5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44662 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:50:31,993 - lava_callback - INFO - Received callback for node 69c0e2aece0c9bc43bccec6c INFO: 172.21.0.1:58280 - "POST /node/69c0e2aece0c9bc43bccec6c HTTP/1.0" 202 Accepted 2026-03-25 06:50:31,999 - lava_callback - INFO - Starting async job submit for node 69c0e2aece0c9bc43bccec6c 2026-03-25 06:50:32,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:50:32,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e2aece0c9bc43bccec6c HTTP/1.1" 200 None 2026-03-25 06:50:32,015 - lava_callback - DEBUG - Processing node 69c0e2aece0c9bc43bccec6c: baseline-arm-clabbe 2026-03-25 06:50:32,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e2aece0c9bc43bccec6c 2026-03-25 06:50:32,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:50:32,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e2aece0c9bc43bccec6c HTTP/1.1" 200 None 2026-03-25 06:50:32,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:50:32,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:50:32,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:50:32,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e2aece0c9bc43bccec6c HTTP/1.1" 200 None 2026-03-25 06:50:32,075 - lava_callback - ERROR - Error processing callback for node 69c0e2aece0c9bc43bccec6c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54984 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:51:12,076 - lava_callback - INFO - Received callback for node 69c0e2d3ce0c9bc43bccec73 INFO: 172.21.0.1:55716 - "POST /node/69c0e2d3ce0c9bc43bccec73 HTTP/1.0" 202 Accepted 2026-03-25 06:51:12,082 - lava_callback - INFO - Starting async job submit for node 69c0e2d3ce0c9bc43bccec73 2026-03-25 06:51:12,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:51:12,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e2d3ce0c9bc43bccec73 HTTP/1.1" 200 None 2026-03-25 06:51:12,124 - lava_callback - DEBUG - Processing node 69c0e2d3ce0c9bc43bccec73: baseline-arm-clabbe 2026-03-25 06:51:12,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e2d3ce0c9bc43bccec73 2026-03-25 06:51:12,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:51:12,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e2d3ce0c9bc43bccec73 HTTP/1.1" 200 None 2026-03-25 06:51:12,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:51:12,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-25 06:51:12,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:51:12,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e2d3ce0c9bc43bccec73 HTTP/1.1" 200 None 2026-03-25 06:51:12,208 - lava_callback - ERROR - Error processing callback for node 69c0e2d3ce0c9bc43bccec73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58052 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:51:52,051 - lava_callback - INFO - Received callback for node 69c0e2f8ce0c9bc43bccec7c INFO: 172.21.0.1:57650 - "POST /node/69c0e2f8ce0c9bc43bccec7c HTTP/1.0" 202 Accepted 2026-03-25 06:51:52,056 - lava_callback - INFO - Starting async job submit for node 69c0e2f8ce0c9bc43bccec7c 2026-03-25 06:51:52,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:51:52,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e2f8ce0c9bc43bccec7c HTTP/1.1" 200 None 2026-03-25 06:51:52,079 - lava_callback - DEBUG - Processing node 69c0e2f8ce0c9bc43bccec7c: baseline-arm-clabbe 2026-03-25 06:51:52,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e2f8ce0c9bc43bccec7c 2026-03-25 06:51:52,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:51:52,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e2f8ce0c9bc43bccec7c HTTP/1.1" 200 None 2026-03-25 06:51:52,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:51:52,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 06:51:52,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:51:52,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e2f8ce0c9bc43bccec7c HTTP/1.1" 200 None 2026-03-25 06:51:52,145 - lava_callback - ERROR - Error processing callback for node 69c0e2f8ce0c9bc43bccec7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47670 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:52:32,113 - lava_callback - INFO - Received callback for node 69c0e31dce0c9bc43bccec87 INFO: 172.21.0.1:33138 - "POST /node/69c0e31dce0c9bc43bccec87 HTTP/1.0" 202 Accepted 2026-03-25 06:52:32,119 - lava_callback - INFO - Starting async job submit for node 69c0e31dce0c9bc43bccec87 2026-03-25 06:52:32,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:52:32,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e31dce0c9bc43bccec87 HTTP/1.1" 200 None 2026-03-25 06:52:32,135 - lava_callback - DEBUG - Processing node 69c0e31dce0c9bc43bccec87: baseline-arm-clabbe 2026-03-25 06:52:32,136 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e31dce0c9bc43bccec87 2026-03-25 06:52:32,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:52:32,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e31dce0c9bc43bccec87 HTTP/1.1" 200 None 2026-03-25 06:52:32,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:52:32,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 06:52:32,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:52:32,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e31dce0c9bc43bccec87 HTTP/1.1" 200 None 2026-03-25 06:52:32,206 - lava_callback - ERROR - Error processing callback for node 69c0e31dce0c9bc43bccec87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58970 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:53:12,144 - lava_callback - INFO - Received callback for node 69c0e343ce0c9bc43bccec94 INFO: 172.21.0.1:54038 - "POST /node/69c0e343ce0c9bc43bccec94 HTTP/1.0" 202 Accepted 2026-03-25 06:53:12,149 - lava_callback - INFO - Starting async job submit for node 69c0e343ce0c9bc43bccec94 2026-03-25 06:53:12,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:53:12,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e343ce0c9bc43bccec94 HTTP/1.1" 200 None 2026-03-25 06:53:12,165 - lava_callback - DEBUG - Processing node 69c0e343ce0c9bc43bccec94: baseline-arm-clabbe 2026-03-25 06:53:12,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e343ce0c9bc43bccec94 2026-03-25 06:53:12,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:53:12,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e343ce0c9bc43bccec94 HTTP/1.1" 200 None 2026-03-25 06:53:12,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:53:12,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 06:53:12,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:53:12,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e343ce0c9bc43bccec94 HTTP/1.1" 200 None 2026-03-25 06:53:12,217 - lava_callback - ERROR - Error processing callback for node 69c0e343ce0c9bc43bccec94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52590 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:53:52,181 - lava_callback - INFO - Received callback for node 69c0e36bce0c9bc43bccec9d INFO: 172.21.0.1:55796 - "POST /node/69c0e36bce0c9bc43bccec9d HTTP/1.0" 202 Accepted 2026-03-25 06:53:52,187 - lava_callback - INFO - Starting async job submit for node 69c0e36bce0c9bc43bccec9d 2026-03-25 06:53:52,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:53:52,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e36bce0c9bc43bccec9d HTTP/1.1" 200 None 2026-03-25 06:53:52,204 - lava_callback - DEBUG - Processing node 69c0e36bce0c9bc43bccec9d: baseline-arm-clabbe 2026-03-25 06:53:52,205 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e36bce0c9bc43bccec9d 2026-03-25 06:53:52,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:53:52,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e36bce0c9bc43bccec9d HTTP/1.1" 200 None 2026-03-25 06:53:52,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:53:52,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 06:53:52,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:53:52,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e36bce0c9bc43bccec9d HTTP/1.1" 200 None 2026-03-25 06:53:52,260 - lava_callback - ERROR - Error processing callback for node 69c0e36bce0c9bc43bccec9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59712 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:54:12,175 - lava_callback - INFO - Received callback for node 69c0e390ce0c9bc43bcceca8 INFO: 172.21.0.1:50370 - "POST /node/69c0e390ce0c9bc43bcceca8 HTTP/1.0" 202 Accepted 2026-03-25 06:54:12,181 - lava_callback - INFO - Starting async job submit for node 69c0e390ce0c9bc43bcceca8 2026-03-25 06:54:12,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:54:12,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e390ce0c9bc43bcceca8 HTTP/1.1" 200 None 2026-03-25 06:54:12,196 - lava_callback - DEBUG - Processing node 69c0e390ce0c9bc43bcceca8: baseline-arm-clabbe 2026-03-25 06:54:12,196 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e390ce0c9bc43bcceca8 2026-03-25 06:54:12,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:54:12,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e390ce0c9bc43bcceca8 HTTP/1.1" 200 None 2026-03-25 06:54:12,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:54:12,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 06:54:12,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:54:12,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e390ce0c9bc43bcceca8 HTTP/1.1" 200 None 2026-03-25 06:54:12,253 - lava_callback - ERROR - Error processing callback for node 69c0e390ce0c9bc43bcceca8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51674 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:54:52,256 - lava_callback - INFO - Received callback for node 69c0e3b6ce0c9bc43bccecb5 INFO: 172.21.0.1:55382 - "POST /node/69c0e3b6ce0c9bc43bccecb5 HTTP/1.0" 202 Accepted 2026-03-25 06:54:52,262 - lava_callback - INFO - Starting async job submit for node 69c0e3b6ce0c9bc43bccecb5 2026-03-25 06:54:52,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:54:52,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e3b6ce0c9bc43bccecb5 HTTP/1.1" 200 None 2026-03-25 06:54:52,278 - lava_callback - DEBUG - Processing node 69c0e3b6ce0c9bc43bccecb5: baseline-arm-clabbe 2026-03-25 06:54:52,279 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e3b6ce0c9bc43bccecb5 2026-03-25 06:54:52,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:54:52,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e3b6ce0c9bc43bccecb5 HTTP/1.1" 200 None 2026-03-25 06:54:52,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:54:52,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 06:54:52,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:54:52,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e3b6ce0c9bc43bccecb5 HTTP/1.1" 200 None 2026-03-25 06:54:52,375 - lava_callback - ERROR - Error processing callback for node 69c0e3b6ce0c9bc43bccecb5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52574 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:55:32,226 - lava_callback - INFO - Received callback for node 69c0e3dbce0c9bc43bccecc0 INFO: 172.21.0.1:42466 - "POST /node/69c0e3dbce0c9bc43bccecc0 HTTP/1.0" 202 Accepted 2026-03-25 06:55:32,232 - lava_callback - INFO - Starting async job submit for node 69c0e3dbce0c9bc43bccecc0 2026-03-25 06:55:32,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:55:32,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e3dbce0c9bc43bccecc0 HTTP/1.1" 200 None 2026-03-25 06:55:32,248 - lava_callback - DEBUG - Processing node 69c0e3dbce0c9bc43bccecc0: baseline-arm-clabbe 2026-03-25 06:55:32,248 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e3dbce0c9bc43bccecc0 2026-03-25 06:55:32,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:55:32,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e3dbce0c9bc43bccecc0 HTTP/1.1" 200 None 2026-03-25 06:55:32,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:55:32,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde359d54b5122738c3d93 HTTP/1.1" 200 None 2026-03-25 06:55:32,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:55:32,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e3dbce0c9bc43bccecc0 HTTP/1.1" 200 None 2026-03-25 06:55:32,307 - lava_callback - ERROR - Error processing callback for node 69c0e3dbce0c9bc43bccecc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46426 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:56:11,669 - lava_callback - INFO - Received callback for node 69c0e3face0c9bc43bcceccb INFO: 172.21.0.1:53018 - "POST /node/69c0e3face0c9bc43bcceccb HTTP/1.0" 202 Accepted 2026-03-25 06:56:11,674 - lava_callback - INFO - Starting async job submit for node 69c0e3face0c9bc43bcceccb 2026-03-25 06:56:11,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:56:11,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e3face0c9bc43bcceccb HTTP/1.1" 200 None 2026-03-25 06:56:11,691 - lava_callback - DEBUG - Processing node 69c0e3face0c9bc43bcceccb: baseline-arm-baylibre 2026-03-25 06:56:11,691 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e3face0c9bc43bcceccb 2026-03-25 06:56:11,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:56:11,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e3face0c9bc43bcceccb HTTP/1.1" 200 None 2026-03-25 06:56:11,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:56:11,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 06:56:11,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:56:11,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e3face0c9bc43bcceccb HTTP/1.1" 200 None 2026-03-25 06:56:11,742 - lava_callback - ERROR - Error processing callback for node 69c0e3face0c9bc43bcceccb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57422 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:56:51,642 - lava_callback - INFO - Received callback for node 69c0e420ce0c9bc43bccecd7 INFO: 172.21.0.1:42212 - "POST /node/69c0e420ce0c9bc43bccecd7 HTTP/1.0" 202 Accepted 2026-03-25 06:56:51,647 - lava_callback - INFO - Starting async job submit for node 69c0e420ce0c9bc43bccecd7 2026-03-25 06:56:51,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:56:51,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e420ce0c9bc43bccecd7 HTTP/1.1" 200 None 2026-03-25 06:56:51,664 - lava_callback - DEBUG - Processing node 69c0e420ce0c9bc43bccecd7: baseline-arm-baylibre 2026-03-25 06:56:51,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e420ce0c9bc43bccecd7 2026-03-25 06:56:51,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:56:51,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e420ce0c9bc43bccecd7 HTTP/1.1" 200 None 2026-03-25 06:56:51,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:56:51,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 06:56:51,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:56:51,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e420ce0c9bc43bccecd7 HTTP/1.1" 200 None 2026-03-25 06:56:51,717 - lava_callback - ERROR - Error processing callback for node 69c0e420ce0c9bc43bccecd7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53376 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:57:11,639 - lava_callback - INFO - Received callback for node 69c0e445ce0c9bc43bccece6 INFO: 172.21.0.1:59244 - "POST /node/69c0e445ce0c9bc43bccece6 HTTP/1.0" 202 Accepted 2026-03-25 06:57:11,644 - lava_callback - INFO - Starting async job submit for node 69c0e445ce0c9bc43bccece6 2026-03-25 06:57:11,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:57:11,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e445ce0c9bc43bccece6 HTTP/1.1" 200 None 2026-03-25 06:57:11,661 - lava_callback - DEBUG - Processing node 69c0e445ce0c9bc43bccece6: baseline-arm-baylibre 2026-03-25 06:57:11,661 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e445ce0c9bc43bccece6 2026-03-25 06:57:11,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:57:11,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e445ce0c9bc43bccece6 HTTP/1.1" 200 None 2026-03-25 06:57:11,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:57:11,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 06:57:11,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:57:11,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e445ce0c9bc43bccece6 HTTP/1.1" 200 None 2026-03-25 06:57:11,736 - lava_callback - ERROR - Error processing callback for node 69c0e445ce0c9bc43bccece6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41926 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:57:51,709 - lava_callback - INFO - Received callback for node 69c0e469ce0c9bc43bccecf4 INFO: 172.21.0.1:33806 - "POST /node/69c0e469ce0c9bc43bccecf4 HTTP/1.0" 202 Accepted 2026-03-25 06:57:51,715 - lava_callback - INFO - Starting async job submit for node 69c0e469ce0c9bc43bccecf4 2026-03-25 06:57:51,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:57:51,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e469ce0c9bc43bccecf4 HTTP/1.1" 200 None 2026-03-25 06:57:51,744 - lava_callback - DEBUG - Processing node 69c0e469ce0c9bc43bccecf4: baseline-arm-baylibre 2026-03-25 06:57:51,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e469ce0c9bc43bccecf4 2026-03-25 06:57:51,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:57:51,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e469ce0c9bc43bccecf4 HTTP/1.1" 200 None 2026-03-25 06:57:51,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:57:51,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 06:57:51,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:57:51,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e469ce0c9bc43bccecf4 HTTP/1.1" 200 None 2026-03-25 06:57:51,817 - lava_callback - ERROR - Error processing callback for node 69c0e469ce0c9bc43bccecf4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55964 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:58:31,738 - lava_callback - INFO - Received callback for node 69c0e48fce0c9bc43bcced03 INFO: 172.21.0.1:49744 - "POST /node/69c0e48fce0c9bc43bcced03 HTTP/1.0" 202 Accepted 2026-03-25 06:58:31,744 - lava_callback - INFO - Starting async job submit for node 69c0e48fce0c9bc43bcced03 2026-03-25 06:58:31,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:58:31,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e48fce0c9bc43bcced03 HTTP/1.1" 200 None 2026-03-25 06:58:31,760 - lava_callback - DEBUG - Processing node 69c0e48fce0c9bc43bcced03: baseline-arm-baylibre 2026-03-25 06:58:31,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e48fce0c9bc43bcced03 2026-03-25 06:58:31,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:58:31,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e48fce0c9bc43bcced03 HTTP/1.1" 200 None 2026-03-25 06:58:31,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:58:31,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 06:58:31,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:58:31,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e48fce0c9bc43bcced03 HTTP/1.1" 200 None 2026-03-25 06:58:31,847 - lava_callback - ERROR - Error processing callback for node 69c0e48fce0c9bc43bcced03: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56446 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:59:11,782 - lava_callback - INFO - Received callback for node 69c0e4b1ce0c9bc43bcced11 INFO: 172.21.0.1:44318 - "POST /node/69c0e4b1ce0c9bc43bcced11 HTTP/1.0" 202 Accepted 2026-03-25 06:59:11,787 - lava_callback - INFO - Starting async job submit for node 69c0e4b1ce0c9bc43bcced11 2026-03-25 06:59:11,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:59:11,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e4b1ce0c9bc43bcced11 HTTP/1.1" 200 None 2026-03-25 06:59:11,811 - lava_callback - DEBUG - Processing node 69c0e4b1ce0c9bc43bcced11: baseline-arm-baylibre 2026-03-25 06:59:11,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e4b1ce0c9bc43bcced11 2026-03-25 06:59:11,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:59:11,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e4b1ce0c9bc43bcced11 HTTP/1.1" 200 None 2026-03-25 06:59:11,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:59:11,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 06:59:11,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:59:11,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e4b1ce0c9bc43bcced11 HTTP/1.1" 200 None 2026-03-25 06:59:11,871 - lava_callback - ERROR - Error processing callback for node 69c0e4b1ce0c9bc43bcced11: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34426 - "GET / HTTP/1.0" 200 OK 2026-03-25 06:59:51,792 - lava_callback - INFO - Received callback for node 69c0e4d5ce0c9bc43bcced1c INFO: 172.21.0.1:57082 - "POST /node/69c0e4d5ce0c9bc43bcced1c HTTP/1.0" 202 Accepted 2026-03-25 06:59:51,798 - lava_callback - INFO - Starting async job submit for node 69c0e4d5ce0c9bc43bcced1c 2026-03-25 06:59:51,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:59:51,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e4d5ce0c9bc43bcced1c HTTP/1.1" 200 None 2026-03-25 06:59:51,817 - lava_callback - DEBUG - Processing node 69c0e4d5ce0c9bc43bcced1c: baseline-arm-baylibre 2026-03-25 06:59:51,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e4d5ce0c9bc43bcced1c 2026-03-25 06:59:51,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:59:51,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e4d5ce0c9bc43bcced1c HTTP/1.1" 200 None 2026-03-25 06:59:51,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:59:51,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 06:59:51,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 06:59:51,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e4d5ce0c9bc43bcced1c HTTP/1.1" 200 None 2026-03-25 06:59:51,886 - lava_callback - ERROR - Error processing callback for node 69c0e4d5ce0c9bc43bcced1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38702 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:00:32,469 - lava_callback - INFO - Received callback for node 69c0e4fece0c9bc43bcced2b INFO: 172.21.0.1:57828 - "POST /node/69c0e4fece0c9bc43bcced2b HTTP/1.0" 202 Accepted 2026-03-25 07:00:32,474 - lava_callback - INFO - Starting async job submit for node 69c0e4fece0c9bc43bcced2b 2026-03-25 07:00:32,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:00:32,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e4fece0c9bc43bcced2b HTTP/1.1" 200 None 2026-03-25 07:00:32,491 - lava_callback - DEBUG - Processing node 69c0e4fece0c9bc43bcced2b: baseline-arm-clabbe 2026-03-25 07:00:32,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e4fece0c9bc43bcced2b 2026-03-25 07:00:32,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:00:32,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e4fece0c9bc43bcced2b HTTP/1.1" 200 None 2026-03-25 07:00:32,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:00:32,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:00:32,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:00:32,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e4fece0c9bc43bcced2b HTTP/1.1" 200 None 2026-03-25 07:00:32,544 - lava_callback - ERROR - Error processing callback for node 69c0e4fece0c9bc43bcced2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45104 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:01:11,939 - lava_callback - INFO - Received callback for node 69c0e522ce0c9bc43bcced3c INFO: 172.21.0.1:59116 - "POST /node/69c0e522ce0c9bc43bcced3c HTTP/1.0" 202 Accepted 2026-03-25 07:01:11,944 - lava_callback - INFO - Starting async job submit for node 69c0e522ce0c9bc43bcced3c 2026-03-25 07:01:11,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:01:11,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e522ce0c9bc43bcced3c HTTP/1.1" 200 None 2026-03-25 07:01:11,965 - lava_callback - DEBUG - Processing node 69c0e522ce0c9bc43bcced3c: baseline-arm-baylibre 2026-03-25 07:01:11,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e522ce0c9bc43bcced3c 2026-03-25 07:01:11,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:01:11,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e522ce0c9bc43bcced3c HTTP/1.1" 200 None 2026-03-25 07:01:11,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:01:11,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:01:11,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:01:12,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e522ce0c9bc43bcced3c HTTP/1.1" 200 None 2026-03-25 07:01:12,018 - lava_callback - ERROR - Error processing callback for node 69c0e522ce0c9bc43bcced3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36550 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:01:52,575 - lava_callback - INFO - Received callback for node 69c0e54bce0c9bc43bcced46 INFO: 172.21.0.1:38472 - "POST /node/69c0e54bce0c9bc43bcced46 HTTP/1.0" 202 Accepted 2026-03-25 07:01:52,581 - lava_callback - INFO - Starting async job submit for node 69c0e54bce0c9bc43bcced46 2026-03-25 07:01:52,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:01:52,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e54bce0c9bc43bcced46 HTTP/1.1" 200 None 2026-03-25 07:01:52,598 - lava_callback - DEBUG - Processing node 69c0e54bce0c9bc43bcced46: baseline-arm-clabbe 2026-03-25 07:01:52,598 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e54bce0c9bc43bcced46 2026-03-25 07:01:52,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:01:52,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e54bce0c9bc43bcced46 HTTP/1.1" 200 None 2026-03-25 07:01:52,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:01:52,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:01:52,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:01:52,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e54bce0c9bc43bcced46 HTTP/1.1" 200 None 2026-03-25 07:01:52,658 - lava_callback - ERROR - Error processing callback for node 69c0e54bce0c9bc43bcced46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45358 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:02:11,958 - lava_callback - INFO - Received callback for node 69c0e56dce0c9bc43bcced52 INFO: 172.21.0.1:40890 - "POST /node/69c0e56dce0c9bc43bcced52 HTTP/1.0" 202 Accepted 2026-03-25 07:02:11,963 - lava_callback - INFO - Starting async job submit for node 69c0e56dce0c9bc43bcced52 2026-03-25 07:02:11,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:02:11,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e56dce0c9bc43bcced52 HTTP/1.1" 200 None 2026-03-25 07:02:11,980 - lava_callback - DEBUG - Processing node 69c0e56dce0c9bc43bcced52: baseline-arm-baylibre 2026-03-25 07:02:11,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e56dce0c9bc43bcced52 2026-03-25 07:02:11,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:02:11,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e56dce0c9bc43bcced52 HTTP/1.1" 200 None 2026-03-25 07:02:11,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:02:12,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:02:12,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:02:12,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e56dce0c9bc43bcced52 HTTP/1.1" 200 None 2026-03-25 07:02:12,031 - lava_callback - ERROR - Error processing callback for node 69c0e56dce0c9bc43bcced52: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48436 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:02:52,578 - lava_callback - INFO - Received callback for node 69c0e596ce0c9bc43bcced60 INFO: 172.21.0.1:37234 - "POST /node/69c0e596ce0c9bc43bcced60 HTTP/1.0" 202 Accepted 2026-03-25 07:02:52,583 - lava_callback - INFO - Starting async job submit for node 69c0e596ce0c9bc43bcced60 2026-03-25 07:02:52,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:02:52,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e596ce0c9bc43bcced60 HTTP/1.1" 200 None 2026-03-25 07:02:52,605 - lava_callback - DEBUG - Processing node 69c0e596ce0c9bc43bcced60: baseline-arm-clabbe 2026-03-25 07:02:52,606 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e596ce0c9bc43bcced60 2026-03-25 07:02:52,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:02:52,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e596ce0c9bc43bcced60 HTTP/1.1" 200 None 2026-03-25 07:02:52,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:02:52,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:02:52,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:02:52,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e596ce0c9bc43bcced60 HTTP/1.1" 200 None 2026-03-25 07:02:52,665 - lava_callback - ERROR - Error processing callback for node 69c0e596ce0c9bc43bcced60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42782 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:03:32,051 - lava_callback - INFO - Received callback for node 69c0e5b6ce0c9bc43bcced6a INFO: 172.21.0.1:56864 - "POST /node/69c0e5b6ce0c9bc43bcced6a HTTP/1.0" 202 Accepted 2026-03-25 07:03:32,057 - lava_callback - INFO - Starting async job submit for node 69c0e5b6ce0c9bc43bcced6a 2026-03-25 07:03:32,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:03:32,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e5b6ce0c9bc43bcced6a HTTP/1.1" 200 None 2026-03-25 07:03:32,086 - lava_callback - DEBUG - Processing node 69c0e5b6ce0c9bc43bcced6a: baseline-arm-baylibre 2026-03-25 07:03:32,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e5b6ce0c9bc43bcced6a 2026-03-25 07:03:32,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:03:32,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e5b6ce0c9bc43bcced6a HTTP/1.1" 200 None 2026-03-25 07:03:32,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:03:32,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:03:32,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:03:32,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e5b6ce0c9bc43bcced6a HTTP/1.1" 200 None 2026-03-25 07:03:32,176 - lava_callback - ERROR - Error processing callback for node 69c0e5b6ce0c9bc43bcced6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53414 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:04:12,635 - lava_callback - INFO - Received callback for node 69c0e5e0ce0c9bc43bcced77 INFO: 172.21.0.1:44856 - "POST /node/69c0e5e0ce0c9bc43bcced77 HTTP/1.0" 202 Accepted 2026-03-25 07:04:12,641 - lava_callback - INFO - Starting async job submit for node 69c0e5e0ce0c9bc43bcced77 2026-03-25 07:04:12,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:04:12,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e5e0ce0c9bc43bcced77 HTTP/1.1" 200 None 2026-03-25 07:04:12,657 - lava_callback - DEBUG - Processing node 69c0e5e0ce0c9bc43bcced77: baseline-arm-clabbe 2026-03-25 07:04:12,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e5e0ce0c9bc43bcced77 2026-03-25 07:04:12,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:04:12,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e5e0ce0c9bc43bcced77 HTTP/1.1" 200 None 2026-03-25 07:04:12,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:04:12,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:04:12,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:04:12,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e5e0ce0c9bc43bcced77 HTTP/1.1" 200 None 2026-03-25 07:04:12,718 - lava_callback - ERROR - Error processing callback for node 69c0e5e0ce0c9bc43bcced77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59592 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:04:52,104 - lava_callback - INFO - Received callback for node 69c0e602ce0c9bc43bcced84 INFO: 172.21.0.1:35082 - "POST /node/69c0e602ce0c9bc43bcced84 HTTP/1.0" 202 Accepted 2026-03-25 07:04:52,110 - lava_callback - INFO - Starting async job submit for node 69c0e602ce0c9bc43bcced84 2026-03-25 07:04:52,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:04:52,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e602ce0c9bc43bcced84 HTTP/1.1" 200 None 2026-03-25 07:04:52,126 - lava_callback - DEBUG - Processing node 69c0e602ce0c9bc43bcced84: baseline-arm-baylibre 2026-03-25 07:04:52,126 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e602ce0c9bc43bcced84 2026-03-25 07:04:52,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:04:52,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e602ce0c9bc43bcced84 HTTP/1.1" 200 None 2026-03-25 07:04:52,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:04:52,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:04:52,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:04:52,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e602ce0c9bc43bcced84 HTTP/1.1" 200 None 2026-03-25 07:04:52,215 - lava_callback - ERROR - Error processing callback for node 69c0e602ce0c9bc43bcced84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58506 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:05:12,103 - lava_callback - INFO - Received callback for node 69c0e624ce0c9bc43bcced8f INFO: 172.21.0.1:55176 - "POST /node/69c0e624ce0c9bc43bcced8f HTTP/1.0" 202 Accepted 2026-03-25 07:05:12,108 - lava_callback - INFO - Starting async job submit for node 69c0e624ce0c9bc43bcced8f 2026-03-25 07:05:12,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:05:12,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e624ce0c9bc43bcced8f HTTP/1.1" 200 None 2026-03-25 07:05:12,123 - lava_callback - DEBUG - Processing node 69c0e624ce0c9bc43bcced8f: baseline-arm-baylibre 2026-03-25 07:05:12,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e624ce0c9bc43bcced8f 2026-03-25 07:05:12,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:05:12,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e624ce0c9bc43bcced8f HTTP/1.1" 200 None 2026-03-25 07:05:12,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:05:12,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:05:12,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:05:12,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e624ce0c9bc43bcced8f HTTP/1.1" 200 None 2026-03-25 07:05:12,182 - lava_callback - ERROR - Error processing callback for node 69c0e624ce0c9bc43bcced8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56098 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:05:52,153 - lava_callback - INFO - Received callback for node 69c0e649ce0c9bc43bcced9a INFO: 172.21.0.1:36698 - "POST /node/69c0e649ce0c9bc43bcced9a HTTP/1.0" 202 Accepted 2026-03-25 07:05:52,159 - lava_callback - INFO - Starting async job submit for node 69c0e649ce0c9bc43bcced9a 2026-03-25 07:05:52,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:05:52,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e649ce0c9bc43bcced9a HTTP/1.1" 200 None 2026-03-25 07:05:52,174 - lava_callback - DEBUG - Processing node 69c0e649ce0c9bc43bcced9a: baseline-arm-baylibre 2026-03-25 07:05:52,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e649ce0c9bc43bcced9a 2026-03-25 07:05:52,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:05:52,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e649ce0c9bc43bcced9a HTTP/1.1" 200 None 2026-03-25 07:05:52,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:05:52,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:05:52,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:05:52,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e649ce0c9bc43bcced9a HTTP/1.1" 200 None 2026-03-25 07:05:52,234 - lava_callback - ERROR - Error processing callback for node 69c0e649ce0c9bc43bcced9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43522 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:06:32,758 - lava_callback - INFO - Received callback for node 69c0e671ce0c9bc43bcceda8 INFO: 172.21.0.1:48242 - "POST /node/69c0e671ce0c9bc43bcceda8 HTTP/1.0" 202 Accepted 2026-03-25 07:06:32,763 - lava_callback - INFO - Starting async job submit for node 69c0e671ce0c9bc43bcceda8 2026-03-25 07:06:32,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:06:32,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e671ce0c9bc43bcceda8 HTTP/1.1" 200 None 2026-03-25 07:06:32,779 - lava_callback - DEBUG - Processing node 69c0e671ce0c9bc43bcceda8: baseline-arm-clabbe 2026-03-25 07:06:32,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e671ce0c9bc43bcceda8 2026-03-25 07:06:32,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:06:32,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e671ce0c9bc43bcceda8 HTTP/1.1" 200 None 2026-03-25 07:06:32,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:06:32,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:06:32,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:06:32,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e671ce0c9bc43bcceda8 HTTP/1.1" 200 None 2026-03-25 07:06:32,850 - lava_callback - ERROR - Error processing callback for node 69c0e671ce0c9bc43bcceda8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40962 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:07:12,253 - lava_callback - INFO - Received callback for node 69c0e693ce0c9bc43bccedb3 INFO: 172.21.0.1:34058 - "POST /node/69c0e693ce0c9bc43bccedb3 HTTP/1.0" 202 Accepted 2026-03-25 07:07:12,258 - lava_callback - INFO - Starting async job submit for node 69c0e693ce0c9bc43bccedb3 2026-03-25 07:07:12,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:07:12,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e693ce0c9bc43bccedb3 HTTP/1.1" 200 None 2026-03-25 07:07:12,274 - lava_callback - DEBUG - Processing node 69c0e693ce0c9bc43bccedb3: baseline-arm-baylibre 2026-03-25 07:07:12,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e693ce0c9bc43bccedb3 2026-03-25 07:07:12,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:07:12,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e693ce0c9bc43bccedb3 HTTP/1.1" 200 None 2026-03-25 07:07:12,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:07:12,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:07:12,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:07:12,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e693ce0c9bc43bccedb3 HTTP/1.1" 200 None 2026-03-25 07:07:12,326 - lava_callback - ERROR - Error processing callback for node 69c0e693ce0c9bc43bccedb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46532 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:07:52,827 - lava_callback - INFO - Received callback for node 69c0e6bcce0c9bc43bccedbf INFO: 172.21.0.1:46976 - "POST /node/69c0e6bcce0c9bc43bccedbf HTTP/1.0" 202 Accepted 2026-03-25 07:07:52,833 - lava_callback - INFO - Starting async job submit for node 69c0e6bcce0c9bc43bccedbf 2026-03-25 07:07:52,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:07:52,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e6bcce0c9bc43bccedbf HTTP/1.1" 200 None 2026-03-25 07:07:52,874 - lava_callback - DEBUG - Processing node 69c0e6bcce0c9bc43bccedbf: baseline-arm-clabbe 2026-03-25 07:07:52,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e6bcce0c9bc43bccedbf 2026-03-25 07:07:52,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:07:52,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e6bcce0c9bc43bccedbf HTTP/1.1" 200 None 2026-03-25 07:07:52,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:07:52,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:07:52,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:07:52,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e6bcce0c9bc43bccedbf HTTP/1.1" 200 None 2026-03-25 07:07:52,949 - lava_callback - ERROR - Error processing callback for node 69c0e6bcce0c9bc43bccedbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57754 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:08:32,340 - lava_callback - INFO - Received callback for node 69c0e6e0ce0c9bc43bccedca INFO: 172.21.0.1:48944 - "POST /node/69c0e6e0ce0c9bc43bccedca HTTP/1.0" 202 Accepted 2026-03-25 07:08:32,346 - lava_callback - INFO - Starting async job submit for node 69c0e6e0ce0c9bc43bccedca 2026-03-25 07:08:32,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:08:32,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e6e0ce0c9bc43bccedca HTTP/1.1" 200 None 2026-03-25 07:08:32,362 - lava_callback - DEBUG - Processing node 69c0e6e0ce0c9bc43bccedca: baseline-arm-baylibre 2026-03-25 07:08:32,363 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e6e0ce0c9bc43bccedca 2026-03-25 07:08:32,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:08:32,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e6e0ce0c9bc43bccedca HTTP/1.1" 200 None 2026-03-25 07:08:32,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:08:32,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:08:32,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:08:32,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e6e0ce0c9bc43bccedca HTTP/1.1" 200 None 2026-03-25 07:08:32,421 - lava_callback - ERROR - Error processing callback for node 69c0e6e0ce0c9bc43bccedca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45350 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:09:12,892 - lava_callback - INFO - Received callback for node 69c0e707ce0c9bc43bccedd7 INFO: 172.21.0.1:53706 - "POST /node/69c0e707ce0c9bc43bccedd7 HTTP/1.0" 202 Accepted 2026-03-25 07:09:12,898 - lava_callback - INFO - Starting async job submit for node 69c0e707ce0c9bc43bccedd7 2026-03-25 07:09:12,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:09:12,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e707ce0c9bc43bccedd7 HTTP/1.1" 200 None 2026-03-25 07:09:12,919 - lava_callback - DEBUG - Processing node 69c0e707ce0c9bc43bccedd7: baseline-arm-clabbe 2026-03-25 07:09:12,919 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e707ce0c9bc43bccedd7 2026-03-25 07:09:12,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:09:12,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e707ce0c9bc43bccedd7 HTTP/1.1" 200 None 2026-03-25 07:09:12,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:09:12,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:09:12,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:09:13,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e707ce0c9bc43bccedd7 HTTP/1.1" 200 None 2026-03-25 07:09:13,037 - lava_callback - ERROR - Error processing callback for node 69c0e707ce0c9bc43bccedd7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39032 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:09:53,000 - lava_callback - INFO - Received callback for node 69c0e72dce0c9bc43bccede2 INFO: 172.21.0.1:60496 - "POST /node/69c0e72dce0c9bc43bccede2 HTTP/1.0" 202 Accepted 2026-03-25 07:09:53,005 - lava_callback - INFO - Starting async job submit for node 69c0e72dce0c9bc43bccede2 2026-03-25 07:09:53,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:09:53,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e72dce0c9bc43bccede2 HTTP/1.1" 200 None 2026-03-25 07:09:53,035 - lava_callback - DEBUG - Processing node 69c0e72dce0c9bc43bccede2: baseline-arm-clabbe 2026-03-25 07:09:53,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e72dce0c9bc43bccede2 2026-03-25 07:09:53,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:09:53,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e72dce0c9bc43bccede2 HTTP/1.1" 200 None 2026-03-25 07:09:53,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:09:53,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:09:53,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:09:53,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e72dce0c9bc43bccede2 HTTP/1.1" 200 None 2026-03-25 07:09:53,099 - lava_callback - ERROR - Error processing callback for node 69c0e72dce0c9bc43bccede2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37360 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:10:32,950 - lava_callback - INFO - Received callback for node 69c0e753ce0c9bc43bccedef INFO: 172.21.0.1:58584 - "POST /node/69c0e753ce0c9bc43bccedef HTTP/1.0" 202 Accepted 2026-03-25 07:10:32,956 - lava_callback - INFO - Starting async job submit for node 69c0e753ce0c9bc43bccedef 2026-03-25 07:10:32,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:10:32,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e753ce0c9bc43bccedef HTTP/1.1" 200 None 2026-03-25 07:10:32,995 - lava_callback - DEBUG - Processing node 69c0e753ce0c9bc43bccedef: baseline-arm-clabbe 2026-03-25 07:10:32,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e753ce0c9bc43bccedef 2026-03-25 07:10:32,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:10:33,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e753ce0c9bc43bccedef HTTP/1.1" 200 None 2026-03-25 07:10:33,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:10:33,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:10:33,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:10:33,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e753ce0c9bc43bccedef HTTP/1.1" 200 None 2026-03-25 07:10:33,074 - lava_callback - ERROR - Error processing callback for node 69c0e753ce0c9bc43bccedef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50968 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:11:12,907 - lava_callback - INFO - Received callback for node 69c0e77dce0c9bc43bccee00 INFO: 172.21.0.1:39022 - "POST /node/69c0e77dce0c9bc43bccee00 HTTP/1.0" 202 Accepted 2026-03-25 07:11:12,913 - lava_callback - INFO - Starting async job submit for node 69c0e77dce0c9bc43bccee00 2026-03-25 07:11:12,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:11:12,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e77dce0c9bc43bccee00 HTTP/1.1" 200 None 2026-03-25 07:11:12,939 - lava_callback - DEBUG - Processing node 69c0e77dce0c9bc43bccee00: baseline-arm-clabbe 2026-03-25 07:11:12,939 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e77dce0c9bc43bccee00 2026-03-25 07:11:12,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:11:12,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e77dce0c9bc43bccee00 HTTP/1.1" 200 None 2026-03-25 07:11:12,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:11:12,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:11:12,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:11:12,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e77dce0c9bc43bccee00 HTTP/1.1" 200 None 2026-03-25 07:11:12,995 - lava_callback - ERROR - Error processing callback for node 69c0e77dce0c9bc43bccee00: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51558 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:11:32,463 - lava_callback - INFO - Received callback for node 69c0e79fce0c9bc43bccee0a INFO: 172.21.0.1:44930 - "POST /node/69c0e79fce0c9bc43bccee0a HTTP/1.0" 202 Accepted 2026-03-25 07:11:32,468 - lava_callback - INFO - Starting async job submit for node 69c0e79fce0c9bc43bccee0a 2026-03-25 07:11:32,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:11:32,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e79fce0c9bc43bccee0a HTTP/1.1" 200 None 2026-03-25 07:11:32,486 - lava_callback - DEBUG - Processing node 69c0e79fce0c9bc43bccee0a: baseline-arm-baylibre 2026-03-25 07:11:32,486 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e79fce0c9bc43bccee0a 2026-03-25 07:11:32,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:11:32,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e79fce0c9bc43bccee0a HTTP/1.1" 200 None 2026-03-25 07:11:32,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:11:32,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:11:32,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:11:32,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e79fce0c9bc43bccee0a HTTP/1.1" 200 None 2026-03-25 07:11:32,597 - lava_callback - ERROR - Error processing callback for node 69c0e79fce0c9bc43bccee0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33570 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:12:12,993 - lava_callback - INFO - Received callback for node 69c0e7c6ce0c9bc43bccee14 INFO: 172.21.0.1:36398 - "POST /node/69c0e7c6ce0c9bc43bccee14 HTTP/1.0" 202 Accepted 2026-03-25 07:12:12,998 - lava_callback - INFO - Starting async job submit for node 69c0e7c6ce0c9bc43bccee14 2026-03-25 07:12:12,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:12:13,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e7c6ce0c9bc43bccee14 HTTP/1.1" 200 None 2026-03-25 07:12:13,023 - lava_callback - DEBUG - Processing node 69c0e7c6ce0c9bc43bccee14: baseline-arm-clabbe 2026-03-25 07:12:13,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e7c6ce0c9bc43bccee14 2026-03-25 07:12:13,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:12:13,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e7c6ce0c9bc43bccee14 HTTP/1.1" 200 None 2026-03-25 07:12:13,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:12:13,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:12:13,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:12:13,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e7c6ce0c9bc43bccee14 HTTP/1.1" 200 None 2026-03-25 07:12:13,086 - lava_callback - ERROR - Error processing callback for node 69c0e7c6ce0c9bc43bccee14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:12:52,548 - lava_callback - INFO - Received callback for node 69c0e7eace0c9bc43bccee1f INFO: 172.21.0.1:46842 - "POST /node/69c0e7eace0c9bc43bccee1f HTTP/1.0" 202 Accepted 2026-03-25 07:12:52,553 - lava_callback - INFO - Starting async job submit for node 69c0e7eace0c9bc43bccee1f 2026-03-25 07:12:52,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:12:52,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e7eace0c9bc43bccee1f HTTP/1.1" 200 None 2026-03-25 07:12:52,570 - lava_callback - DEBUG - Processing node 69c0e7eace0c9bc43bccee1f: baseline-arm-baylibre 2026-03-25 07:12:52,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e7eace0c9bc43bccee1f 2026-03-25 07:12:52,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:12:52,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e7eace0c9bc43bccee1f HTTP/1.1" 200 None 2026-03-25 07:12:52,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:12:52,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:12:52,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:12:52,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e7eace0c9bc43bccee1f HTTP/1.1" 200 None 2026-03-25 07:12:52,655 - lava_callback - ERROR - Error processing callback for node 69c0e7eace0c9bc43bccee1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35254 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:13:33,181 - lava_callback - INFO - Received callback for node 69c0e811ce0c9bc43bccee34 INFO: 172.21.0.1:48534 - "POST /node/69c0e811ce0c9bc43bccee34 HTTP/1.0" 202 Accepted 2026-03-25 07:13:33,186 - lava_callback - INFO - Starting async job submit for node 69c0e811ce0c9bc43bccee34 2026-03-25 07:13:33,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:13:33,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e811ce0c9bc43bccee34 HTTP/1.1" 200 None 2026-03-25 07:13:33,226 - lava_callback - DEBUG - Processing node 69c0e811ce0c9bc43bccee34: baseline-arm-clabbe 2026-03-25 07:13:33,226 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e811ce0c9bc43bccee34 2026-03-25 07:13:33,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:13:33,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e811ce0c9bc43bccee34 HTTP/1.1" 200 None 2026-03-25 07:13:33,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:13:33,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:13:33,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:13:33,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e811ce0c9bc43bccee34 HTTP/1.1" 200 None 2026-03-25 07:13:33,305 - lava_callback - ERROR - Error processing callback for node 69c0e811ce0c9bc43bccee34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58104 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:14:13,046 - lava_callback - INFO - Received callback for node 69c0e836ce0c9bc43bccee3e INFO: 172.21.0.1:51446 - "POST /node/69c0e836ce0c9bc43bccee3e HTTP/1.0" 202 Accepted 2026-03-25 07:14:13,051 - lava_callback - INFO - Starting async job submit for node 69c0e836ce0c9bc43bccee3e 2026-03-25 07:14:13,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:14:13,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e836ce0c9bc43bccee3e HTTP/1.1" 200 None 2026-03-25 07:14:13,069 - lava_callback - DEBUG - Processing node 69c0e836ce0c9bc43bccee3e: baseline-arm-clabbe 2026-03-25 07:14:13,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e836ce0c9bc43bccee3e 2026-03-25 07:14:13,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:14:13,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e836ce0c9bc43bccee3e HTTP/1.1" 200 None 2026-03-25 07:14:13,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:14:13,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:14:13,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:14:13,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e836ce0c9bc43bccee3e HTTP/1.1" 200 None 2026-03-25 07:14:13,128 - lava_callback - ERROR - Error processing callback for node 69c0e836ce0c9bc43bccee3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48484 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:14:53,164 - lava_callback - INFO - Received callback for node 69c0e85bce0c9bc43bccee49 INFO: 172.21.0.1:38020 - "POST /node/69c0e85bce0c9bc43bccee49 HTTP/1.0" 202 Accepted 2026-03-25 07:14:53,169 - lava_callback - INFO - Starting async job submit for node 69c0e85bce0c9bc43bccee49 2026-03-25 07:14:53,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:14:53,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e85bce0c9bc43bccee49 HTTP/1.1" 200 None 2026-03-25 07:14:53,185 - lava_callback - DEBUG - Processing node 69c0e85bce0c9bc43bccee49: baseline-arm-clabbe 2026-03-25 07:14:53,186 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e85bce0c9bc43bccee49 2026-03-25 07:14:53,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:14:53,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e85bce0c9bc43bccee49 HTTP/1.1" 200 None 2026-03-25 07:14:53,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:14:53,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:14:53,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:14:53,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e85bce0c9bc43bccee49 HTTP/1.1" 200 None 2026-03-25 07:14:53,262 - lava_callback - ERROR - Error processing callback for node 69c0e85bce0c9bc43bccee49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46710 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:15:33,137 - lava_callback - INFO - Received callback for node 69c0e880ce0c9bc43bccee55 INFO: 172.21.0.1:39276 - "POST /node/69c0e880ce0c9bc43bccee55 HTTP/1.0" 202 Accepted 2026-03-25 07:15:33,142 - lava_callback - INFO - Starting async job submit for node 69c0e880ce0c9bc43bccee55 2026-03-25 07:15:33,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:15:33,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e880ce0c9bc43bccee55 HTTP/1.1" 200 None 2026-03-25 07:15:33,182 - lava_callback - DEBUG - Processing node 69c0e880ce0c9bc43bccee55: baseline-arm-clabbe 2026-03-25 07:15:33,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e880ce0c9bc43bccee55 2026-03-25 07:15:33,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:15:33,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e880ce0c9bc43bccee55 HTTP/1.1" 200 None 2026-03-25 07:15:33,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:15:33,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40bca1e2bfb1b33443eb HTTP/1.1" 200 None 2026-03-25 07:15:33,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:15:33,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e880ce0c9bc43bccee55 HTTP/1.1" 200 None 2026-03-25 07:15:33,239 - lava_callback - ERROR - Error processing callback for node 69c0e880ce0c9bc43bccee55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41786 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:16:12,804 - lava_callback - INFO - Received callback for node 69c0e8acce0c9bc43bccee64 INFO: 172.21.0.1:39174 - "POST /node/69c0e8acce0c9bc43bccee64 HTTP/1.0" 202 Accepted 2026-03-25 07:16:12,809 - lava_callback - INFO - Starting async job submit for node 69c0e8acce0c9bc43bccee64 2026-03-25 07:16:12,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:16:12,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e8acce0c9bc43bccee64 HTTP/1.1" 200 None 2026-03-25 07:16:12,835 - lava_callback - DEBUG - Processing node 69c0e8acce0c9bc43bccee64: baseline-arm64-baylibre 2026-03-25 07:16:12,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e8acce0c9bc43bccee64 2026-03-25 07:16:12,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:16:12,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e8acce0c9bc43bccee64 HTTP/1.1" 200 None 2026-03-25 07:16:12,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:16:12,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:16:12,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:16:12,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e8acce0c9bc43bccee64 HTTP/1.1" 200 None 2026-03-25 07:16:12,904 - lava_callback - ERROR - Error processing callback for node 69c0e8acce0c9bc43bccee64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58400 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:17:32,802 - lava_callback - INFO - Received callback for node 69c0e8f7ce0c9bc43bccee6e INFO: 172.21.0.1:51098 - "POST /node/69c0e8f7ce0c9bc43bccee6e HTTP/1.0" 202 Accepted 2026-03-25 07:17:32,807 - lava_callback - INFO - Starting async job submit for node 69c0e8f7ce0c9bc43bccee6e 2026-03-25 07:17:32,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:17:32,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e8f7ce0c9bc43bccee6e HTTP/1.1" 200 None 2026-03-25 07:17:32,823 - lava_callback - DEBUG - Processing node 69c0e8f7ce0c9bc43bccee6e: baseline-arm64-baylibre 2026-03-25 07:17:32,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e8f7ce0c9bc43bccee6e 2026-03-25 07:17:32,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:17:32,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e8f7ce0c9bc43bccee6e HTTP/1.1" 200 None 2026-03-25 07:17:32,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:17:32,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:17:32,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:17:32,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e8f7ce0c9bc43bccee6e HTTP/1.1" 200 None 2026-03-25 07:17:32,875 - lava_callback - ERROR - Error processing callback for node 69c0e8f7ce0c9bc43bccee6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41112 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:18:32,815 - lava_callback - INFO - Received callback for node 69c0e943ce0c9bc43bccee84 INFO: 172.21.0.1:40412 - "POST /node/69c0e943ce0c9bc43bccee84 HTTP/1.0" 202 Accepted 2026-03-25 07:18:32,821 - lava_callback - INFO - Starting async job submit for node 69c0e943ce0c9bc43bccee84 2026-03-25 07:18:32,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:18:32,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e943ce0c9bc43bccee84 HTTP/1.1" 200 None 2026-03-25 07:18:32,837 - lava_callback - DEBUG - Processing node 69c0e943ce0c9bc43bccee84: baseline-arm64-baylibre 2026-03-25 07:18:32,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e943ce0c9bc43bccee84 2026-03-25 07:18:32,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:18:32,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e943ce0c9bc43bccee84 HTTP/1.1" 200 None 2026-03-25 07:18:32,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:18:32,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:18:32,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:18:32,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e943ce0c9bc43bccee84 HTTP/1.1" 200 None 2026-03-25 07:18:32,894 - lava_callback - ERROR - Error processing callback for node 69c0e943ce0c9bc43bccee84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49964 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:19:52,960 - lava_callback - INFO - Received callback for node 69c0e992ce0c9bc43bccee8e INFO: 172.21.0.1:43008 - "POST /node/69c0e992ce0c9bc43bccee8e HTTP/1.0" 202 Accepted 2026-03-25 07:19:52,966 - lava_callback - INFO - Starting async job submit for node 69c0e992ce0c9bc43bccee8e 2026-03-25 07:19:52,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:19:52,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e992ce0c9bc43bccee8e HTTP/1.1" 200 None 2026-03-25 07:19:52,982 - lava_callback - DEBUG - Processing node 69c0e992ce0c9bc43bccee8e: baseline-arm64-baylibre 2026-03-25 07:19:52,982 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e992ce0c9bc43bccee8e 2026-03-25 07:19:52,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:19:52,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e992ce0c9bc43bccee8e HTTP/1.1" 200 None 2026-03-25 07:19:52,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:19:53,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:19:53,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:19:53,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e992ce0c9bc43bccee8e HTTP/1.1" 200 None 2026-03-25 07:19:53,039 - lava_callback - ERROR - Error processing callback for node 69c0e992ce0c9bc43bccee8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54040 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:21:13,055 - lava_callback - INFO - Received callback for node 69c0e9dfce0c9bc43bccee9d INFO: 172.21.0.1:34826 - "POST /node/69c0e9dfce0c9bc43bccee9d HTTP/1.0" 202 Accepted 2026-03-25 07:21:13,061 - lava_callback - INFO - Starting async job submit for node 69c0e9dfce0c9bc43bccee9d 2026-03-25 07:21:13,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:21:13,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e9dfce0c9bc43bccee9d HTTP/1.1" 200 None 2026-03-25 07:21:13,078 - lava_callback - DEBUG - Processing node 69c0e9dfce0c9bc43bccee9d: baseline-arm64-baylibre 2026-03-25 07:21:13,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0e9dfce0c9bc43bccee9d 2026-03-25 07:21:13,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:21:13,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0e9dfce0c9bc43bccee9d HTTP/1.1" 200 None 2026-03-25 07:21:13,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:21:13,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:21:13,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:21:13,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0e9dfce0c9bc43bccee9d HTTP/1.1" 200 None 2026-03-25 07:21:13,157 - lava_callback - ERROR - Error processing callback for node 69c0e9dfce0c9bc43bccee9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36736 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:22:33,324 - lava_callback - INFO - Received callback for node 69c0ea28ce0c9bc43bcceea8 INFO: 172.21.0.1:53934 - "POST /node/69c0ea28ce0c9bc43bcceea8 HTTP/1.0" 202 Accepted 2026-03-25 07:22:33,332 - lava_callback - INFO - Starting async job submit for node 69c0ea28ce0c9bc43bcceea8 2026-03-25 07:22:33,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:22:33,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ea28ce0c9bc43bcceea8 HTTP/1.1" 200 None 2026-03-25 07:22:33,355 - lava_callback - DEBUG - Processing node 69c0ea28ce0c9bc43bcceea8: baseline-arm64-baylibre 2026-03-25 07:22:33,355 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ea28ce0c9bc43bcceea8 2026-03-25 07:22:33,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:22:33,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ea28ce0c9bc43bcceea8 HTTP/1.1" 200 None 2026-03-25 07:22:33,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:22:33,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:22:33,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:22:33,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ea28ce0c9bc43bcceea8 HTTP/1.1" 200 None 2026-03-25 07:22:33,424 - lava_callback - ERROR - Error processing callback for node 69c0ea28ce0c9bc43bcceea8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53142 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:53156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58570 - "GET /sitemap.xml HTTP/1.0" 404 Not Found INFO: 172.21.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:23:53,533 - lava_callback - INFO - Received callback for node 69c0ea79ce0c9bc43bcceeb4 INFO: 172.21.0.1:60502 - "POST /node/69c0ea79ce0c9bc43bcceeb4 HTTP/1.0" 202 Accepted 2026-03-25 07:23:53,538 - lava_callback - INFO - Starting async job submit for node 69c0ea79ce0c9bc43bcceeb4 2026-03-25 07:23:53,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:23:53,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ea79ce0c9bc43bcceeb4 HTTP/1.1" 200 None 2026-03-25 07:23:53,554 - lava_callback - DEBUG - Processing node 69c0ea79ce0c9bc43bcceeb4: baseline-arm64-clabbe 2026-03-25 07:23:53,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ea79ce0c9bc43bcceeb4 2026-03-25 07:23:53,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:23:53,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ea79ce0c9bc43bcceeb4 HTTP/1.1" 200 None 2026-03-25 07:23:53,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:23:53,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:23:53,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:23:53,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ea79ce0c9bc43bcceeb4 HTTP/1.1" 200 None 2026-03-25 07:23:53,607 - lava_callback - ERROR - Error processing callback for node 69c0ea79ce0c9bc43bcceeb4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37272 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:25:13,824 - lava_callback - INFO - Received callback for node 69c0eac6ce0c9bc43bcceec6 INFO: 172.21.0.1:43084 - "POST /node/69c0eac6ce0c9bc43bcceec6 HTTP/1.0" 202 Accepted 2026-03-25 07:25:13,829 - lava_callback - INFO - Starting async job submit for node 69c0eac6ce0c9bc43bcceec6 2026-03-25 07:25:13,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:25:13,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0eac6ce0c9bc43bcceec6 HTTP/1.1" 200 None 2026-03-25 07:25:13,846 - lava_callback - DEBUG - Processing node 69c0eac6ce0c9bc43bcceec6: baseline-arm64-clabbe 2026-03-25 07:25:13,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0eac6ce0c9bc43bcceec6 2026-03-25 07:25:13,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:25:13,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0eac6ce0c9bc43bcceec6 HTTP/1.1" 200 None 2026-03-25 07:25:13,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:25:13,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:25:13,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:25:13,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0eac6ce0c9bc43bcceec6 HTTP/1.1" 200 None 2026-03-25 07:25:13,906 - lava_callback - ERROR - Error processing callback for node 69c0eac6ce0c9bc43bcceec6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37890 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:26:13,306 - lava_callback - INFO - Received callback for node 69c0eb0bce0c9bc43bcceed1 INFO: 172.21.0.1:38872 - "POST /node/69c0eb0bce0c9bc43bcceed1 HTTP/1.0" 202 Accepted 2026-03-25 07:26:13,311 - lava_callback - INFO - Starting async job submit for node 69c0eb0bce0c9bc43bcceed1 2026-03-25 07:26:13,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:26:13,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0eb0bce0c9bc43bcceed1 HTTP/1.1" 200 None 2026-03-25 07:26:13,329 - lava_callback - DEBUG - Processing node 69c0eb0bce0c9bc43bcceed1: baseline-arm64-baylibre 2026-03-25 07:26:13,329 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0eb0bce0c9bc43bcceed1 2026-03-25 07:26:13,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:26:13,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0eb0bce0c9bc43bcceed1 HTTP/1.1" 200 None 2026-03-25 07:26:13,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:26:13,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:26:13,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:26:13,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0eb0bce0c9bc43bcceed1 HTTP/1.1" 200 None 2026-03-25 07:26:13,401 - lava_callback - ERROR - Error processing callback for node 69c0eb0bce0c9bc43bcceed1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50042 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:27:33,465 - lava_callback - INFO - Received callback for node 69c0eb59ce0c9bc43bcceede INFO: 172.21.0.1:47498 - "POST /node/69c0eb59ce0c9bc43bcceede HTTP/1.0" 202 Accepted 2026-03-25 07:27:33,470 - lava_callback - INFO - Starting async job submit for node 69c0eb59ce0c9bc43bcceede 2026-03-25 07:27:33,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:27:33,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0eb59ce0c9bc43bcceede HTTP/1.1" 200 None 2026-03-25 07:27:33,487 - lava_callback - DEBUG - Processing node 69c0eb59ce0c9bc43bcceede: baseline-arm64-baylibre 2026-03-25 07:27:33,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0eb59ce0c9bc43bcceede 2026-03-25 07:27:33,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:27:33,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0eb59ce0c9bc43bcceede HTTP/1.1" 200 None 2026-03-25 07:27:33,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:27:33,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:27:33,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:27:33,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0eb59ce0c9bc43bcceede HTTP/1.1" 200 None 2026-03-25 07:27:33,541 - lava_callback - ERROR - Error processing callback for node 69c0eb59ce0c9bc43bcceede: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42410 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:28:53,816 - lava_callback - INFO - Received callback for node 69c0ebacce0c9bc43bcceee5 INFO: 172.21.0.1:33948 - "POST /node/69c0ebacce0c9bc43bcceee5 HTTP/1.0" 202 Accepted 2026-03-25 07:28:53,822 - lava_callback - INFO - Starting async job submit for node 69c0ebacce0c9bc43bcceee5 2026-03-25 07:28:53,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:28:53,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ebacce0c9bc43bcceee5 HTTP/1.1" 200 None 2026-03-25 07:28:53,838 - lava_callback - DEBUG - Processing node 69c0ebacce0c9bc43bcceee5: baseline-arm64-clabbe 2026-03-25 07:28:53,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ebacce0c9bc43bcceee5 2026-03-25 07:28:53,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:28:53,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ebacce0c9bc43bcceee5 HTTP/1.1" 200 None 2026-03-25 07:28:53,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:28:53,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:28:53,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:28:53,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ebacce0c9bc43bcceee5 HTTP/1.1" 200 None 2026-03-25 07:28:53,904 - lava_callback - ERROR - Error processing callback for node 69c0ebacce0c9bc43bcceee5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42248 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:30:13,544 - lava_callback - INFO - Received callback for node 69c0ebf2ce0c9bc43bcceef2 INFO: 172.21.0.1:50352 - "POST /node/69c0ebf2ce0c9bc43bcceef2 HTTP/1.0" 202 Accepted 2026-03-25 07:30:13,549 - lava_callback - INFO - Starting async job submit for node 69c0ebf2ce0c9bc43bcceef2 2026-03-25 07:30:13,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:30:13,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ebf2ce0c9bc43bcceef2 HTTP/1.1" 200 None 2026-03-25 07:30:13,565 - lava_callback - DEBUG - Processing node 69c0ebf2ce0c9bc43bcceef2: baseline-arm64-baylibre 2026-03-25 07:30:13,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ebf2ce0c9bc43bcceef2 2026-03-25 07:30:13,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:30:13,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ebf2ce0c9bc43bcceef2 HTTP/1.1" 200 None 2026-03-25 07:30:13,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:30:13,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:30:13,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:30:13,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ebf2ce0c9bc43bcceef2 HTTP/1.1" 200 None 2026-03-25 07:30:13,628 - lava_callback - ERROR - Error processing callback for node 69c0ebf2ce0c9bc43bcceef2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42424 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:31:33,942 - lava_callback - INFO - Received callback for node 69c0ec47ce0c9bc43bcceef9 INFO: 172.21.0.1:36216 - "POST /node/69c0ec47ce0c9bc43bcceef9 HTTP/1.0" 202 Accepted 2026-03-25 07:31:33,947 - lava_callback - INFO - Starting async job submit for node 69c0ec47ce0c9bc43bcceef9 2026-03-25 07:31:33,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:31:33,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ec47ce0c9bc43bcceef9 HTTP/1.1" 200 None 2026-03-25 07:31:33,963 - lava_callback - DEBUG - Processing node 69c0ec47ce0c9bc43bcceef9: baseline-arm64-clabbe 2026-03-25 07:31:33,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ec47ce0c9bc43bcceef9 2026-03-25 07:31:33,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:31:33,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ec47ce0c9bc43bcceef9 HTTP/1.1" 200 None 2026-03-25 07:31:33,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:31:33,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:31:33,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:31:34,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ec47ce0c9bc43bcceef9 HTTP/1.1" 200 None 2026-03-25 07:31:34,022 - lava_callback - ERROR - Error processing callback for node 69c0ec47ce0c9bc43bcceef9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54774 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:32:33,690 - lava_callback - INFO - Received callback for node 69c0ec8fce0c9bc43bccef07 INFO: 172.21.0.1:37558 - "POST /node/69c0ec8fce0c9bc43bccef07 HTTP/1.0" 202 Accepted 2026-03-25 07:32:33,696 - lava_callback - INFO - Starting async job submit for node 69c0ec8fce0c9bc43bccef07 2026-03-25 07:32:33,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:32:33,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ec8fce0c9bc43bccef07 HTTP/1.1" 200 None 2026-03-25 07:32:33,713 - lava_callback - DEBUG - Processing node 69c0ec8fce0c9bc43bccef07: baseline-arm64-baylibre 2026-03-25 07:32:33,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ec8fce0c9bc43bccef07 2026-03-25 07:32:33,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:32:33,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ec8fce0c9bc43bccef07 HTTP/1.1" 200 None 2026-03-25 07:32:33,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:32:33,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:32:33,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:32:33,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ec8fce0c9bc43bccef07 HTTP/1.1" 200 None 2026-03-25 07:32:33,768 - lava_callback - ERROR - Error processing callback for node 69c0ec8fce0c9bc43bccef07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44998 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:34:14,036 - lava_callback - INFO - Received callback for node 69c0ece3ce0c9bc43bccef18 INFO: 172.21.0.1:42808 - "POST /node/69c0ece3ce0c9bc43bccef18 HTTP/1.0" 202 Accepted 2026-03-25 07:34:14,041 - lava_callback - INFO - Starting async job submit for node 69c0ece3ce0c9bc43bccef18 2026-03-25 07:34:14,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:34:14,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ece3ce0c9bc43bccef18 HTTP/1.1" 200 None 2026-03-25 07:34:14,058 - lava_callback - DEBUG - Processing node 69c0ece3ce0c9bc43bccef18: baseline-arm64-clabbe 2026-03-25 07:34:14,058 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ece3ce0c9bc43bccef18 2026-03-25 07:34:14,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:34:14,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ece3ce0c9bc43bccef18 HTTP/1.1" 200 None 2026-03-25 07:34:14,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:34:14,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:34:14,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:34:14,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ece3ce0c9bc43bccef18 HTTP/1.1" 200 None 2026-03-25 07:34:14,127 - lava_callback - ERROR - Error processing callback for node 69c0ece3ce0c9bc43bccef18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57858 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:35:13,987 - lava_callback - INFO - Received callback for node 69c0ed2bce0c9bc43bccef22 INFO: 172.21.0.1:45996 - "POST /node/69c0ed2bce0c9bc43bccef22 HTTP/1.0" 202 Accepted 2026-03-25 07:35:13,992 - lava_callback - INFO - Starting async job submit for node 69c0ed2bce0c9bc43bccef22 2026-03-25 07:35:13,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:35:14,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ed2bce0c9bc43bccef22 HTTP/1.1" 200 None 2026-03-25 07:35:14,010 - lava_callback - DEBUG - Processing node 69c0ed2bce0c9bc43bccef22: baseline-arm64-baylibre 2026-03-25 07:35:14,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ed2bce0c9bc43bccef22 2026-03-25 07:35:14,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:35:14,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ed2bce0c9bc43bccef22 HTTP/1.1" 200 None 2026-03-25 07:35:14,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:35:14,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:35:14,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:35:14,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ed2bce0c9bc43bccef22 HTTP/1.1" 200 None 2026-03-25 07:35:14,062 - lava_callback - ERROR - Error processing callback for node 69c0ed2bce0c9bc43bccef22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:36:34,149 - lava_callback - INFO - Received callback for node 69c0ed7dce0c9bc43bccef30 INFO: 172.21.0.1:39728 - "POST /node/69c0ed7dce0c9bc43bccef30 HTTP/1.0" 202 Accepted 2026-03-25 07:36:34,155 - lava_callback - INFO - Starting async job submit for node 69c0ed7dce0c9bc43bccef30 2026-03-25 07:36:34,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:36:34,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ed7dce0c9bc43bccef30 HTTP/1.1" 200 None 2026-03-25 07:36:34,171 - lava_callback - DEBUG - Processing node 69c0ed7dce0c9bc43bccef30: baseline-arm64-clabbe 2026-03-25 07:36:34,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ed7dce0c9bc43bccef30 2026-03-25 07:36:34,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:36:34,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ed7dce0c9bc43bccef30 HTTP/1.1" 200 None 2026-03-25 07:36:34,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:36:34,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:36:34,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:36:34,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ed7dce0c9bc43bccef30 HTTP/1.1" 200 None 2026-03-25 07:36:34,244 - lava_callback - ERROR - Error processing callback for node 69c0ed7dce0c9bc43bccef30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57586 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:37:54,092 - lava_callback - INFO - Received callback for node 69c0edc4ce0c9bc43bccef3a INFO: 172.21.0.1:60384 - "POST /node/69c0edc4ce0c9bc43bccef3a HTTP/1.0" 202 Accepted 2026-03-25 07:37:54,098 - lava_callback - INFO - Starting async job submit for node 69c0edc4ce0c9bc43bccef3a 2026-03-25 07:37:54,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:37:54,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0edc4ce0c9bc43bccef3a HTTP/1.1" 200 None 2026-03-25 07:37:54,123 - lava_callback - DEBUG - Processing node 69c0edc4ce0c9bc43bccef3a: baseline-arm64-baylibre 2026-03-25 07:37:54,123 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0edc4ce0c9bc43bccef3a 2026-03-25 07:37:54,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:37:54,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0edc4ce0c9bc43bccef3a HTTP/1.1" 200 None 2026-03-25 07:37:54,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:37:54,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:37:54,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:37:54,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0edc4ce0c9bc43bccef3a HTTP/1.1" 200 None 2026-03-25 07:37:54,189 - lava_callback - ERROR - Error processing callback for node 69c0edc4ce0c9bc43bccef3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33190 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:39:14,331 - lava_callback - INFO - Received callback for node 69c0ee14ce0c9bc43bccef4c INFO: 172.21.0.1:44936 - "POST /node/69c0ee14ce0c9bc43bccef4c HTTP/1.0" 202 Accepted 2026-03-25 07:39:14,337 - lava_callback - INFO - Starting async job submit for node 69c0ee14ce0c9bc43bccef4c 2026-03-25 07:39:14,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:39:14,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ee14ce0c9bc43bccef4c HTTP/1.1" 200 None 2026-03-25 07:39:14,356 - lava_callback - DEBUG - Processing node 69c0ee14ce0c9bc43bccef4c: baseline-arm64-clabbe 2026-03-25 07:39:14,356 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ee14ce0c9bc43bccef4c 2026-03-25 07:39:14,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:39:14,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ee14ce0c9bc43bccef4c HTTP/1.1" 200 None 2026-03-25 07:39:14,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:39:14,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:39:14,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:39:14,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ee14ce0c9bc43bccef4c HTTP/1.1" 200 None 2026-03-25 07:39:14,406 - lava_callback - ERROR - Error processing callback for node 69c0ee14ce0c9bc43bccef4c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57510 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:40:14,301 - lava_callback - INFO - Received callback for node 69c0ee58ce0c9bc43bccef56 INFO: 172.21.0.1:41610 - "POST /node/69c0ee58ce0c9bc43bccef56 HTTP/1.0" 202 Accepted 2026-03-25 07:40:14,306 - lava_callback - INFO - Starting async job submit for node 69c0ee58ce0c9bc43bccef56 2026-03-25 07:40:14,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:40:14,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ee58ce0c9bc43bccef56 HTTP/1.1" 200 None 2026-03-25 07:40:14,333 - lava_callback - DEBUG - Processing node 69c0ee58ce0c9bc43bccef56: baseline-arm64-baylibre 2026-03-25 07:40:14,333 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ee58ce0c9bc43bccef56 2026-03-25 07:40:14,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:40:14,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ee58ce0c9bc43bccef56 HTTP/1.1" 200 None 2026-03-25 07:40:14,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:40:14,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:40:14,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:40:14,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ee58ce0c9bc43bccef56 HTTP/1.1" 200 None 2026-03-25 07:40:14,392 - lava_callback - ERROR - Error processing callback for node 69c0ee58ce0c9bc43bccef56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:41:34,439 - lava_callback - INFO - Received callback for node 69c0eeabce0c9bc43bccef64 INFO: 172.21.0.1:32864 - "POST /node/69c0eeabce0c9bc43bccef64 HTTP/1.0" 202 Accepted 2026-03-25 07:41:34,444 - lava_callback - INFO - Starting async job submit for node 69c0eeabce0c9bc43bccef64 2026-03-25 07:41:34,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:41:34,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0eeabce0c9bc43bccef64 HTTP/1.1" 200 None 2026-03-25 07:41:34,460 - lava_callback - DEBUG - Processing node 69c0eeabce0c9bc43bccef64: baseline-arm64-clabbe 2026-03-25 07:41:34,460 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0eeabce0c9bc43bccef64 2026-03-25 07:41:34,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:41:34,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0eeabce0c9bc43bccef64 HTTP/1.1" 200 None 2026-03-25 07:41:34,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:41:34,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:41:34,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:41:34,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0eeabce0c9bc43bccef64 HTTP/1.1" 200 None 2026-03-25 07:41:34,525 - lava_callback - ERROR - Error processing callback for node 69c0eeabce0c9bc43bccef64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44928 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:42:54,237 - lava_callback - INFO - Received callback for node 69c0eeefce0c9bc43bccef6e INFO: 172.21.0.1:46642 - "POST /node/69c0eeefce0c9bc43bccef6e HTTP/1.0" 202 Accepted 2026-03-25 07:42:54,243 - lava_callback - INFO - Starting async job submit for node 69c0eeefce0c9bc43bccef6e 2026-03-25 07:42:54,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:42:54,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0eeefce0c9bc43bccef6e HTTP/1.1" 200 None 2026-03-25 07:42:54,259 - lava_callback - DEBUG - Processing node 69c0eeefce0c9bc43bccef6e: baseline-arm64-baylibre 2026-03-25 07:42:54,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0eeefce0c9bc43bccef6e 2026-03-25 07:42:54,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:42:54,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0eeefce0c9bc43bccef6e HTTP/1.1" 200 None 2026-03-25 07:42:54,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:42:54,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 07:42:54,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:42:54,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0eeefce0c9bc43bccef6e HTTP/1.1" 200 None 2026-03-25 07:42:54,332 - lava_callback - ERROR - Error processing callback for node 69c0eeefce0c9bc43bccef6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42686 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:44:14,454 - lava_callback - INFO - Received callback for node 69c0ef42ce0c9bc43bccef80 INFO: 172.21.0.1:33520 - "POST /node/69c0ef42ce0c9bc43bccef80 HTTP/1.0" 202 Accepted 2026-03-25 07:44:14,459 - lava_callback - INFO - Starting async job submit for node 69c0ef42ce0c9bc43bccef80 2026-03-25 07:44:14,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:44:14,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ef42ce0c9bc43bccef80 HTTP/1.1" 200 None 2026-03-25 07:44:14,475 - lava_callback - DEBUG - Processing node 69c0ef42ce0c9bc43bccef80: baseline-arm64-clabbe 2026-03-25 07:44:14,476 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ef42ce0c9bc43bccef80 2026-03-25 07:44:14,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:44:14,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ef42ce0c9bc43bccef80 HTTP/1.1" 200 None 2026-03-25 07:44:14,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:44:14,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:44:14,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:44:14,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ef42ce0c9bc43bccef80 HTTP/1.1" 200 None 2026-03-25 07:44:14,541 - lava_callback - ERROR - Error processing callback for node 69c0ef42ce0c9bc43bccef80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44312 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:45:14,412 - lava_callback - INFO - Received callback for node 69c0ef87ce0c9bc43bccef94 INFO: 172.21.0.1:44490 - "POST /node/69c0ef87ce0c9bc43bccef94 HTTP/1.0" 202 Accepted 2026-03-25 07:45:14,417 - lava_callback - INFO - Starting async job submit for node 69c0ef87ce0c9bc43bccef94 2026-03-25 07:45:14,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:45:14,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ef87ce0c9bc43bccef94 HTTP/1.1" 200 None 2026-03-25 07:45:14,433 - lava_callback - DEBUG - Processing node 69c0ef87ce0c9bc43bccef94: baseline-arm64-baylibre 2026-03-25 07:45:14,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0ef87ce0c9bc43bccef94 2026-03-25 07:45:14,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:45:14,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0ef87ce0c9bc43bccef94 HTTP/1.1" 200 None 2026-03-25 07:45:14,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:45:14,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 07:45:14,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:45:14,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0ef87ce0c9bc43bccef94 HTTP/1.1" 200 None 2026-03-25 07:45:14,502 - lava_callback - ERROR - Error processing callback for node 69c0ef87ce0c9bc43bccef94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40596 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:46:34,557 - lava_callback - INFO - Received callback for node 69c0efd4ce0c9bc43bccefca INFO: 172.21.0.1:53400 - "POST /node/69c0efd4ce0c9bc43bccefca HTTP/1.0" 202 Accepted 2026-03-25 07:46:34,565 - lava_callback - INFO - Starting async job submit for node 69c0efd4ce0c9bc43bccefca 2026-03-25 07:46:34,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:46:34,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0efd4ce0c9bc43bccefca HTTP/1.1" 200 None 2026-03-25 07:46:34,582 - lava_callback - DEBUG - Processing node 69c0efd4ce0c9bc43bccefca: baseline-arm64-baylibre 2026-03-25 07:46:34,582 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0efd4ce0c9bc43bccefca 2026-03-25 07:46:34,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:46:34,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0efd4ce0c9bc43bccefca HTTP/1.1" 200 None 2026-03-25 07:46:34,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:46:34,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:46:34,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:46:34,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0efd4ce0c9bc43bccefca HTTP/1.1" 200 None 2026-03-25 07:46:34,649 - lava_callback - ERROR - Error processing callback for node 69c0efd4ce0c9bc43bccefca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57886 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:47:54,521 - lava_callback - INFO - Received callback for node 69c0f020ce0c9bc43bccefd0 INFO: 172.21.0.1:52384 - "POST /node/69c0f020ce0c9bc43bccefd0 HTTP/1.0" 202 Accepted 2026-03-25 07:47:54,526 - lava_callback - INFO - Starting async job submit for node 69c0f020ce0c9bc43bccefd0 2026-03-25 07:47:54,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:47:54,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f020ce0c9bc43bccefd0 HTTP/1.1" 200 None 2026-03-25 07:47:54,543 - lava_callback - DEBUG - Processing node 69c0f020ce0c9bc43bccefd0: baseline-arm64-baylibre 2026-03-25 07:47:54,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f020ce0c9bc43bccefd0 2026-03-25 07:47:54,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:47:54,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f020ce0c9bc43bccefd0 HTTP/1.1" 200 None 2026-03-25 07:47:54,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:47:54,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 07:47:54,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:47:54,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f020ce0c9bc43bccefd0 HTTP/1.1" 200 None 2026-03-25 07:47:54,620 - lava_callback - ERROR - Error processing callback for node 69c0f020ce0c9bc43bccefd0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45562 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:49:14,677 - lava_callback - INFO - Received callback for node 69c0f073ce0c9bc43bccefda INFO: 172.21.0.1:39564 - "POST /node/69c0f073ce0c9bc43bccefda HTTP/1.0" 202 Accepted 2026-03-25 07:49:14,685 - lava_callback - INFO - Starting async job submit for node 69c0f073ce0c9bc43bccefda 2026-03-25 07:49:14,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:49:14,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f073ce0c9bc43bccefda HTTP/1.1" 200 None 2026-03-25 07:49:14,701 - lava_callback - DEBUG - Processing node 69c0f073ce0c9bc43bccefda: baseline-arm64-clabbe 2026-03-25 07:49:14,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f073ce0c9bc43bccefda 2026-03-25 07:49:14,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:49:14,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f073ce0c9bc43bccefda HTTP/1.1" 200 None 2026-03-25 07:49:14,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:49:14,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:49:14,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:49:14,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f073ce0c9bc43bccefda HTTP/1.1" 200 None 2026-03-25 07:49:14,759 - lava_callback - ERROR - Error processing callback for node 69c0f073ce0c9bc43bccefda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36840 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:50:34,683 - lava_callback - INFO - Received callback for node 69c0f0bcce0c9bc43bccefdf INFO: 172.21.0.1:56028 - "POST /node/69c0f0bcce0c9bc43bccefdf HTTP/1.0" 202 Accepted 2026-03-25 07:50:34,689 - lava_callback - INFO - Starting async job submit for node 69c0f0bcce0c9bc43bccefdf 2026-03-25 07:50:34,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:50:34,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f0bcce0c9bc43bccefdf HTTP/1.1" 200 None 2026-03-25 07:50:34,706 - lava_callback - DEBUG - Processing node 69c0f0bcce0c9bc43bccefdf: baseline-arm64-baylibre 2026-03-25 07:50:34,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f0bcce0c9bc43bccefdf 2026-03-25 07:50:34,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:50:34,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f0bcce0c9bc43bccefdf HTTP/1.1" 200 None 2026-03-25 07:50:34,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:50:34,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 07:50:34,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:50:34,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f0bcce0c9bc43bccefdf HTTP/1.1" 200 None 2026-03-25 07:50:34,785 - lava_callback - ERROR - Error processing callback for node 69c0f0bcce0c9bc43bccefdf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39610 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:51:54,710 - lava_callback - INFO - Received callback for node 69c0f10cce0c9bc43bccefe8 INFO: 172.21.0.1:60260 - "POST /node/69c0f10cce0c9bc43bccefe8 HTTP/1.0" 202 Accepted 2026-03-25 07:51:54,716 - lava_callback - INFO - Starting async job submit for node 69c0f10cce0c9bc43bccefe8 2026-03-25 07:51:54,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:51:54,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f10cce0c9bc43bccefe8 HTTP/1.1" 200 None 2026-03-25 07:51:54,732 - lava_callback - DEBUG - Processing node 69c0f10cce0c9bc43bccefe8: baseline-arm64-baylibre 2026-03-25 07:51:54,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f10cce0c9bc43bccefe8 2026-03-25 07:51:54,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:51:54,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f10cce0c9bc43bccefe8 HTTP/1.1" 200 None 2026-03-25 07:51:54,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:51:54,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:51:54,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:51:54,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f10cce0c9bc43bccefe8 HTTP/1.1" 200 None 2026-03-25 07:51:54,792 - lava_callback - ERROR - Error processing callback for node 69c0f10cce0c9bc43bccefe8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42552 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:53:14,901 - lava_callback - INFO - Received callback for node 69c0f157ce0c9bc43bcceff3 INFO: 172.21.0.1:45294 - "POST /node/69c0f157ce0c9bc43bcceff3 HTTP/1.0" 202 Accepted 2026-03-25 07:53:14,907 - lava_callback - INFO - Starting async job submit for node 69c0f157ce0c9bc43bcceff3 2026-03-25 07:53:14,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:53:14,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f157ce0c9bc43bcceff3 HTTP/1.1" 200 None 2026-03-25 07:53:14,929 - lava_callback - DEBUG - Processing node 69c0f157ce0c9bc43bcceff3: baseline-arm64-baylibre 2026-03-25 07:53:14,929 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f157ce0c9bc43bcceff3 2026-03-25 07:53:14,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:53:14,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f157ce0c9bc43bcceff3 HTTP/1.1" 200 None 2026-03-25 07:53:14,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:53:14,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:53:14,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:53:14,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f157ce0c9bc43bcceff3 HTTP/1.1" 200 None 2026-03-25 07:53:14,986 - lava_callback - ERROR - Error processing callback for node 69c0f157ce0c9bc43bcceff3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:54:14,843 - lava_callback - INFO - Received callback for node 69c0f1a2ce0c9bc43bcceffa INFO: 172.21.0.1:54572 - "POST /node/69c0f1a2ce0c9bc43bcceffa HTTP/1.0" 202 Accepted 2026-03-25 07:54:14,850 - lava_callback - INFO - Starting async job submit for node 69c0f1a2ce0c9bc43bcceffa 2026-03-25 07:54:14,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:54:14,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f1a2ce0c9bc43bcceffa HTTP/1.1" 200 None 2026-03-25 07:54:14,866 - lava_callback - DEBUG - Processing node 69c0f1a2ce0c9bc43bcceffa: baseline-arm64-baylibre 2026-03-25 07:54:14,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f1a2ce0c9bc43bcceffa 2026-03-25 07:54:14,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:54:14,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f1a2ce0c9bc43bcceffa HTTP/1.1" 200 None 2026-03-25 07:54:14,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:54:14,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 07:54:14,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:54:14,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f1a2ce0c9bc43bcceffa HTTP/1.1" 200 None 2026-03-25 07:54:14,945 - lava_callback - ERROR - Error processing callback for node 69c0f1a2ce0c9bc43bcceffa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40156 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:55:34,951 - lava_callback - INFO - Received callback for node 69c0f1eece0c9bc43bccf007 INFO: 172.21.0.1:60428 - "POST /node/69c0f1eece0c9bc43bccf007 HTTP/1.0" 202 Accepted 2026-03-25 07:55:34,957 - lava_callback - INFO - Starting async job submit for node 69c0f1eece0c9bc43bccf007 2026-03-25 07:55:34,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:55:34,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f1eece0c9bc43bccf007 HTTP/1.1" 200 None 2026-03-25 07:55:34,973 - lava_callback - DEBUG - Processing node 69c0f1eece0c9bc43bccf007: baseline-arm64-baylibre 2026-03-25 07:55:34,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f1eece0c9bc43bccf007 2026-03-25 07:55:34,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:55:34,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f1eece0c9bc43bccf007 HTTP/1.1" 200 None 2026-03-25 07:55:34,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:55:35,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 07:55:35,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:55:35,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f1eece0c9bc43bccf007 HTTP/1.1" 200 None 2026-03-25 07:55:35,026 - lava_callback - ERROR - Error processing callback for node 69c0f1eece0c9bc43bccf007: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41768 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:56:54,968 - lava_callback - INFO - Received callback for node 69c0f23fce0c9bc43bccf00e INFO: 172.21.0.1:36002 - "POST /node/69c0f23fce0c9bc43bccf00e HTTP/1.0" 202 Accepted 2026-03-25 07:56:54,973 - lava_callback - INFO - Starting async job submit for node 69c0f23fce0c9bc43bccf00e 2026-03-25 07:56:54,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:56:54,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f23fce0c9bc43bccf00e HTTP/1.1" 200 None 2026-03-25 07:56:54,992 - lava_callback - DEBUG - Processing node 69c0f23fce0c9bc43bccf00e: baseline-arm64-clabbe 2026-03-25 07:56:54,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f23fce0c9bc43bccf00e 2026-03-25 07:56:54,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:56:55,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f23fce0c9bc43bccf00e HTTP/1.1" 200 None 2026-03-25 07:56:55,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:56:55,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:56:55,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:56:55,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f23fce0c9bc43bccf00e HTTP/1.1" 200 None 2026-03-25 07:56:55,059 - lava_callback - ERROR - Error processing callback for node 69c0f23fce0c9bc43bccf00e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36882 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:58:15,010 - lava_callback - INFO - Received callback for node 69c0f28bce0c9bc43bccf01c INFO: 172.21.0.1:59444 - "POST /node/69c0f28bce0c9bc43bccf01c HTTP/1.0" 202 Accepted 2026-03-25 07:58:15,018 - lava_callback - INFO - Starting async job submit for node 69c0f28bce0c9bc43bccf01c 2026-03-25 07:58:15,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:58:15,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f28bce0c9bc43bccf01c HTTP/1.1" 200 None 2026-03-25 07:58:15,041 - lava_callback - DEBUG - Processing node 69c0f28bce0c9bc43bccf01c: baseline-arm64-clabbe 2026-03-25 07:58:15,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f28bce0c9bc43bccf01c 2026-03-25 07:58:15,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:58:15,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f28bce0c9bc43bccf01c HTTP/1.1" 200 None 2026-03-25 07:58:15,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:58:15,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 07:58:15,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:58:15,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f28bce0c9bc43bccf01c HTTP/1.1" 200 None 2026-03-25 07:58:15,101 - lava_callback - ERROR - Error processing callback for node 69c0f28bce0c9bc43bccf01c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49596 - "GET / HTTP/1.0" 200 OK 2026-03-25 07:59:35,069 - lava_callback - INFO - Received callback for node 69c0f2d7ce0c9bc43bccf024 INFO: 172.21.0.1:59522 - "POST /node/69c0f2d7ce0c9bc43bccf024 HTTP/1.0" 202 Accepted 2026-03-25 07:59:35,074 - lava_callback - INFO - Starting async job submit for node 69c0f2d7ce0c9bc43bccf024 2026-03-25 07:59:35,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:59:35,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f2d7ce0c9bc43bccf024 HTTP/1.1" 200 None 2026-03-25 07:59:35,092 - lava_callback - DEBUG - Processing node 69c0f2d7ce0c9bc43bccf024: baseline-arm64-clabbe 2026-03-25 07:59:35,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f2d7ce0c9bc43bccf024 2026-03-25 07:59:35,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:59:35,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f2d7ce0c9bc43bccf024 HTTP/1.1" 200 None 2026-03-25 07:59:35,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:59:35,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 07:59:35,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 07:59:35,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f2d7ce0c9bc43bccf024 HTTP/1.1" 200 None 2026-03-25 07:59:35,152 - lava_callback - ERROR - Error processing callback for node 69c0f2d7ce0c9bc43bccf024: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32908 - "GET / HTTP/1.0" 200 OK 2026-03-25 08:00:55,149 - lava_callback - INFO - Received callback for node 69c0f325ce0c9bc43bccf030 INFO: 172.21.0.1:45258 - "POST /node/69c0f325ce0c9bc43bccf030 HTTP/1.0" 202 Accepted 2026-03-25 08:00:55,155 - lava_callback - INFO - Starting async job submit for node 69c0f325ce0c9bc43bccf030 2026-03-25 08:00:55,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:00:55,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f325ce0c9bc43bccf030 HTTP/1.1" 200 None 2026-03-25 08:00:55,171 - lava_callback - DEBUG - Processing node 69c0f325ce0c9bc43bccf030: baseline-arm64-clabbe 2026-03-25 08:00:55,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f325ce0c9bc43bccf030 2026-03-25 08:00:55,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:00:55,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f325ce0c9bc43bccf030 HTTP/1.1" 200 None 2026-03-25 08:00:55,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:00:55,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 08:00:55,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:00:55,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f325ce0c9bc43bccf030 HTTP/1.1" 200 None 2026-03-25 08:00:55,229 - lava_callback - ERROR - Error processing callback for node 69c0f325ce0c9bc43bccf030: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35636 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 08:02:15,216 - lava_callback - INFO - Received callback for node 69c0f372ce0c9bc43bccf039 INFO: 172.21.0.1:35652 - "POST /node/69c0f372ce0c9bc43bccf039 HTTP/1.0" 202 Accepted 2026-03-25 08:02:15,221 - lava_callback - INFO - Starting async job submit for node 69c0f372ce0c9bc43bccf039 2026-03-25 08:02:15,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:02:15,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f372ce0c9bc43bccf039 HTTP/1.1" 200 None 2026-03-25 08:02:15,245 - lava_callback - DEBUG - Processing node 69c0f372ce0c9bc43bccf039: baseline-arm64-clabbe 2026-03-25 08:02:15,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f372ce0c9bc43bccf039 2026-03-25 08:02:15,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:02:15,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f372ce0c9bc43bccf039 HTTP/1.1" 200 None 2026-03-25 08:02:15,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:02:15,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 08:02:15,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:02:15,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f372ce0c9bc43bccf039 HTTP/1.1" 200 None 2026-03-25 08:02:15,306 - lava_callback - ERROR - Error processing callback for node 69c0f372ce0c9bc43bccf039: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35488 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 08:03:15,242 - lava_callback - INFO - Received callback for node 69c0f3bece0c9bc43bccf042 INFO: 172.21.0.1:35504 - "POST /node/69c0f3bece0c9bc43bccf042 HTTP/1.0" 202 Accepted 2026-03-25 08:03:15,247 - lava_callback - INFO - Starting async job submit for node 69c0f3bece0c9bc43bccf042 2026-03-25 08:03:15,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:03:15,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f3bece0c9bc43bccf042 HTTP/1.1" 200 None 2026-03-25 08:03:15,263 - lava_callback - DEBUG - Processing node 69c0f3bece0c9bc43bccf042: baseline-arm64-clabbe 2026-03-25 08:03:15,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f3bece0c9bc43bccf042 2026-03-25 08:03:15,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:03:15,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f3bece0c9bc43bccf042 HTTP/1.1" 200 None 2026-03-25 08:03:15,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:03:15,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 08:03:15,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:03:15,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f3bece0c9bc43bccf042 HTTP/1.1" 200 None 2026-03-25 08:03:15,331 - lava_callback - ERROR - Error processing callback for node 69c0f3bece0c9bc43bccf042: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49972 - "GET / HTTP/1.0" 200 OK 2026-03-25 08:04:35,300 - lava_callback - INFO - Received callback for node 69c0f409ce0c9bc43bccf050 INFO: 172.21.0.1:48140 - "POST /node/69c0f409ce0c9bc43bccf050 HTTP/1.0" 202 Accepted 2026-03-25 08:04:35,306 - lava_callback - INFO - Starting async job submit for node 69c0f409ce0c9bc43bccf050 2026-03-25 08:04:35,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:04:35,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f409ce0c9bc43bccf050 HTTP/1.1" 200 None 2026-03-25 08:04:35,322 - lava_callback - DEBUG - Processing node 69c0f409ce0c9bc43bccf050: baseline-arm64-clabbe 2026-03-25 08:04:35,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f409ce0c9bc43bccf050 2026-03-25 08:04:35,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:04:35,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f409ce0c9bc43bccf050 HTTP/1.1" 200 None 2026-03-25 08:04:35,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:04:35,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 08:04:35,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:04:35,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f409ce0c9bc43bccf050 HTTP/1.1" 200 None 2026-03-25 08:04:35,381 - lava_callback - ERROR - Error processing callback for node 69c0f409ce0c9bc43bccf050: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36280 - "GET / HTTP/1.0" 200 OK 2026-03-25 08:05:55,450 - lava_callback - INFO - Received callback for node 69c0f456ce0c9bc43bccf058 INFO: 172.21.0.1:59020 - "POST /node/69c0f456ce0c9bc43bccf058 HTTP/1.0" 202 Accepted 2026-03-25 08:05:55,455 - lava_callback - INFO - Starting async job submit for node 69c0f456ce0c9bc43bccf058 2026-03-25 08:05:55,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:05:55,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f456ce0c9bc43bccf058 HTTP/1.1" 200 None 2026-03-25 08:05:55,473 - lava_callback - DEBUG - Processing node 69c0f456ce0c9bc43bccf058: baseline-arm64-clabbe 2026-03-25 08:05:55,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f456ce0c9bc43bccf058 2026-03-25 08:05:55,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:05:55,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f456ce0c9bc43bccf058 HTTP/1.1" 200 None 2026-03-25 08:05:55,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:05:55,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 08:05:55,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:05:55,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f456ce0c9bc43bccf058 HTTP/1.1" 200 None 2026-03-25 08:05:55,527 - lava_callback - ERROR - Error processing callback for node 69c0f456ce0c9bc43bccf058: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42890 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-25 08:07:15,418 - lava_callback - INFO - Received callback for node 69c0f4a4ce0c9bc43bccf065 INFO: 172.21.0.1:42904 - "POST /node/69c0f4a4ce0c9bc43bccf065 HTTP/1.0" 202 Accepted 2026-03-25 08:07:15,424 - lava_callback - INFO - Starting async job submit for node 69c0f4a4ce0c9bc43bccf065 2026-03-25 08:07:15,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:07:15,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f4a4ce0c9bc43bccf065 HTTP/1.1" 200 None 2026-03-25 08:07:15,440 - lava_callback - DEBUG - Processing node 69c0f4a4ce0c9bc43bccf065: baseline-arm64-clabbe 2026-03-25 08:07:15,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f4a4ce0c9bc43bccf065 2026-03-25 08:07:15,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:07:15,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f4a4ce0c9bc43bccf065 HTTP/1.1" 200 None 2026-03-25 08:07:15,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:07:15,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 08:07:15,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:07:15,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f4a4ce0c9bc43bccf065 HTTP/1.1" 200 None 2026-03-25 08:07:15,506 - lava_callback - ERROR - Error processing callback for node 69c0f4a4ce0c9bc43bccf065: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32870 - "GET / HTTP/1.0" 200 OK 2026-03-25 08:08:35,526 - lava_callback - INFO - Received callback for node 69c0f4f2ce0c9bc43bccf06e INFO: 172.21.0.1:57504 - "POST /node/69c0f4f2ce0c9bc43bccf06e HTTP/1.0" 202 Accepted 2026-03-25 08:08:35,531 - lava_callback - INFO - Starting async job submit for node 69c0f4f2ce0c9bc43bccf06e 2026-03-25 08:08:35,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:08:35,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f4f2ce0c9bc43bccf06e HTTP/1.1" 200 None 2026-03-25 08:08:35,559 - lava_callback - DEBUG - Processing node 69c0f4f2ce0c9bc43bccf06e: baseline-arm64-clabbe 2026-03-25 08:08:35,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f4f2ce0c9bc43bccf06e 2026-03-25 08:08:35,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:08:35,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f4f2ce0c9bc43bccf06e HTTP/1.1" 200 None 2026-03-25 08:08:35,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:08:35,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde369d54b5122738c3da2 HTTP/1.1" 200 None 2026-03-25 08:08:35,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:08:35,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f4f2ce0c9bc43bccf06e HTTP/1.1" 200 None 2026-03-25 08:08:35,616 - lava_callback - ERROR - Error processing callback for node 69c0f4f2ce0c9bc43bccf06e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48096 - "GET / HTTP/1.0" 200 OK 2026-03-25 08:09:55,581 - lava_callback - INFO - Received callback for node 69c0f53dce0c9bc43bccf07b INFO: 172.21.0.1:42122 - "POST /node/69c0f53dce0c9bc43bccf07b HTTP/1.0" 202 Accepted 2026-03-25 08:09:55,588 - lava_callback - INFO - Starting async job submit for node 69c0f53dce0c9bc43bccf07b 2026-03-25 08:09:55,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:09:55,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f53dce0c9bc43bccf07b HTTP/1.1" 200 None 2026-03-25 08:09:55,606 - lava_callback - DEBUG - Processing node 69c0f53dce0c9bc43bccf07b: baseline-arm64-clabbe 2026-03-25 08:09:55,607 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0f53dce0c9bc43bccf07b 2026-03-25 08:09:55,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:09:55,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f53dce0c9bc43bccf07b HTTP/1.1" 200 None 2026-03-25 08:09:55,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:09:55,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bb40cfa1e2bfb1b33443fa HTTP/1.1" 200 None 2026-03-25 08:09:55,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-25 08:09:55,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0f53dce0c9bc43bccf07b HTTP/1.1" 200 None 2026-03-25 08:09:55,667 - lava_callback - ERROR - Error processing callback for node 69c0f53dce0c9bc43bccf07b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33574 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]