INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:41220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42942 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:32:06,315 - lava_callback - INFO - Received callback for node 69cdb88523eb053795d41efd 2026-04-02 00:32:06,322 - lava_callback - INFO - Starting async job submit for node 69cdb88523eb053795d41efd INFO: 172.20.0.1:34836 - "POST /node/69cdb88523eb053795d41efd HTTP/1.0" 202 Accepted 2026-04-02 00:32:06,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:32:06,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb88523eb053795d41efd HTTP/1.1" 200 None 2026-04-02 00:32:06,366 - lava_callback - DEBUG - Processing node 69cdb88523eb053795d41efd: baseline-arm INFO: 172.20.0.1:34852 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:32:06,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:32:06,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:32:06,719 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cdb88523eb053795d41efd/log.txt.gz 2026-04-02 00:32:06,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:32:07,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:32:07,022 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdb88523eb053795d41efd/lava-logs.yaml 2026-04-02 00:32:07,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:32:07,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:32:07,167 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdb88523eb053795d41efd/lava_callback.json.gz 2026-04-02 00:32:07,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:32:07,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb88523eb053795d41efd HTTP/1.1" 200 None 2026-04-02 00:32:07,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:32:07,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5bf23eb053795d41ebd HTTP/1.1" 200 None 2026-04-02 00:32:07,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:32:07,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdb88523eb053795d41efd HTTP/1.1" 200 None 2026-04-02 00:32:07,279 - lava_callback - INFO - Completed processing callback for node 69cdb88523eb053795d41efd INFO: 172.20.0.1:40756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33406 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:32:37,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:32:37,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42324 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 00:35:15,958 - lava_callback - INFO - Received callback for node 69cb16b78c06c2799bf5505e INFO: 172.20.0.1:42330 - "POST /node/69cb16b78c06c2799bf5505e HTTP/1.0" 202 Accepted 2026-04-02 00:35:15,964 - lava_callback - INFO - Starting async job submit for node 69cb16b78c06c2799bf5505e 2026-04-02 00:35:15,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:35:15,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb16b78c06c2799bf5505e HTTP/1.1" 200 None 2026-04-02 00:35:15,980 - lava_callback - DEBUG - Processing node 69cb16b78c06c2799bf5505e: baseline-arm-baylibre 2026-04-02 00:35:15,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb16b78c06c2799bf5505e 2026-04-02 00:35:15,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:35:15,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb16b78c06c2799bf5505e HTTP/1.1" 200 None 2026-04-02 00:35:15,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:35:16,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:35:16,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:35:16,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb16b78c06c2799bf5505e HTTP/1.1" 200 None 2026-04-02 00:35:16,050 - lava_callback - ERROR - Error processing callback for node 69cb16b78c06c2799bf5505e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44532 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:35:55,951 - lava_callback - INFO - Received callback for node 69cb16e28c06c2799bf5506e INFO: 172.20.0.1:34324 - "POST /node/69cb16e28c06c2799bf5506e HTTP/1.0" 202 Accepted 2026-04-02 00:35:55,958 - lava_callback - INFO - Starting async job submit for node 69cb16e28c06c2799bf5506e 2026-04-02 00:35:55,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:35:55,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb16e28c06c2799bf5506e HTTP/1.1" 200 None 2026-04-02 00:35:55,973 - lava_callback - DEBUG - Processing node 69cb16e28c06c2799bf5506e: baseline-arm-baylibre 2026-04-02 00:35:55,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb16e28c06c2799bf5506e 2026-04-02 00:35:55,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:35:55,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb16e28c06c2799bf5506e HTTP/1.1" 200 None 2026-04-02 00:35:55,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:35:56,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:35:56,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:35:56,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb16e28c06c2799bf5506e HTTP/1.1" 200 None 2026-04-02 00:35:56,026 - lava_callback - ERROR - Error processing callback for node 69cb16e28c06c2799bf5506e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39094 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:36:36,014 - lava_callback - INFO - Received callback for node 69cb170c8c06c2799bf5507b INFO: 172.20.0.1:35876 - "POST /node/69cb170c8c06c2799bf5507b HTTP/1.0" 202 Accepted 2026-04-02 00:36:36,020 - lava_callback - INFO - Starting async job submit for node 69cb170c8c06c2799bf5507b 2026-04-02 00:36:36,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:36:36,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb170c8c06c2799bf5507b HTTP/1.1" 200 None 2026-04-02 00:36:36,036 - lava_callback - DEBUG - Processing node 69cb170c8c06c2799bf5507b: baseline-arm-baylibre 2026-04-02 00:36:36,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb170c8c06c2799bf5507b 2026-04-02 00:36:36,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:36:36,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb170c8c06c2799bf5507b HTTP/1.1" 200 None 2026-04-02 00:36:36,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:36:36,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:36:36,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:36:36,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb170c8c06c2799bf5507b HTTP/1.1" 200 None 2026-04-02 00:36:36,100 - lava_callback - ERROR - Error processing callback for node 69cb170c8c06c2799bf5507b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35878 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:36:41,147 - lava_callback - INFO - Received callback for node 69cdb9f823eb053795d41f2d INFO: 172.20.0.1:41906 - "POST /node/69cdb9f823eb053795d41f2d HTTP/1.0" 202 Accepted 2026-04-02 00:36:41,154 - lava_callback - INFO - Starting async job submit for node 69cdb9f823eb053795d41f2d 2026-04-02 00:36:41,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:36:41,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9f823eb053795d41f2d HTTP/1.1" 200 None 2026-04-02 00:36:41,188 - lava_callback - DEBUG - Processing node 69cdb9f823eb053795d41f2d: baseline-x86 2026-04-02 00:36:41,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:36:41,510 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:36:41,512 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f823eb053795d41f2d/log.txt.gz 2026-04-02 00:36:41,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:36:41,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:36:41,827 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f823eb053795d41f2d/lava-logs.yaml 2026-04-02 00:36:41,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:36:41,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:36:41,998 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f823eb053795d41f2d/lava_callback.json.gz 2026-04-02 00:36:42,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:36:42,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9f823eb053795d41f2d HTTP/1.1" 200 None 2026-04-02 00:36:42,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:36:42,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5e523eb053795d41ede HTTP/1.1" 200 None 2026-04-02 00:36:42,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:36:42,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdb9f823eb053795d41f2d HTTP/1.1" 200 None 2026-04-02 00:36:42,096 - lava_callback - INFO - Completed processing callback for node 69cdb9f823eb053795d41f2d INFO: 172.20.0.1:41910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51402 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:37:00,525 - lava_callback - INFO - Received callback for node 69cdb9f623eb053795d41f24 INFO: 172.20.0.1:41802 - "POST /node/69cdb9f623eb053795d41f24 HTTP/1.0" 202 Accepted 2026-04-02 00:37:00,532 - lava_callback - INFO - Starting async job submit for node 69cdb9f623eb053795d41f24 2026-04-02 00:37:00,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:00,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9f623eb053795d41f24 HTTP/1.1" 200 None 2026-04-02 00:37:00,566 - lava_callback - DEBUG - Processing node 69cdb9f623eb053795d41f24: baseline-x86 2026-04-02 00:37:00,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:37:00,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:37:00,877 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f623eb053795d41f24/log.txt.gz 2026-04-02 00:37:00,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:37:01,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:37:01,275 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f623eb053795d41f24/lava-logs.yaml 2026-04-02 00:37:01,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:37:01,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:37:01,423 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f623eb053795d41f24/lava_callback.json.gz 2026-04-02 00:37:01,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:01,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9f623eb053795d41f24 HTTP/1.1" 200 None 2026-04-02 00:37:01,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:01,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5e523eb053795d41ede HTTP/1.1" 200 None 2026-04-02 00:37:01,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:01,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdb9f623eb053795d41f24 HTTP/1.1" 200 None 2026-04-02 00:37:01,508 - lava_callback - INFO - Completed processing callback for node 69cdb9f623eb053795d41f24 INFO: 172.20.0.1:41812 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:37:07,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:07,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37348 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 00:37:16,015 - lava_callback - INFO - Received callback for node 69cb17378c06c2799bf55098 INFO: 172.20.0.1:37360 - "POST /node/69cb17378c06c2799bf55098 HTTP/1.0" 202 Accepted 2026-04-02 00:37:16,021 - lava_callback - INFO - Starting async job submit for node 69cb17378c06c2799bf55098 2026-04-02 00:37:16,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:16,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17378c06c2799bf55098 HTTP/1.1" 200 None 2026-04-02 00:37:16,037 - lava_callback - DEBUG - Processing node 69cb17378c06c2799bf55098: baseline-arm-baylibre 2026-04-02 00:37:16,037 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb17378c06c2799bf55098 2026-04-02 00:37:16,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:16,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17378c06c2799bf55098 HTTP/1.1" 200 None 2026-04-02 00:37:16,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:16,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:37:16,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:16,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb17378c06c2799bf55098 HTTP/1.1" 200 None 2026-04-02 00:37:16,089 - lava_callback - ERROR - Error processing callback for node 69cb17378c06c2799bf55098: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51994 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:37:54,998 - lava_callback - INFO - Received callback for node 69cdb9f923eb053795d41f30 INFO: 172.20.0.1:59948 - "POST /node/69cdb9f923eb053795d41f30 HTTP/1.0" 202 Accepted 2026-04-02 00:37:55,004 - lava_callback - INFO - Starting async job submit for node 69cdb9f923eb053795d41f30 2026-04-02 00:37:55,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:55,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9f923eb053795d41f30 HTTP/1.1" 200 None 2026-04-02 00:37:55,036 - lava_callback - DEBUG - Processing node 69cdb9f923eb053795d41f30: baseline-x86 2026-04-02 00:37:55,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:37:55,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:37:55,531 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f923eb053795d41f30/log.txt.gz 2026-04-02 00:37:55,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:37:55,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:37:55,933 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f923eb053795d41f30/lava-logs.yaml 2026-04-02 00:37:55,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:37:56,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:37:56,095 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f923eb053795d41f30/lava_callback.json.gz 2026-04-02 00:37:56,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:56,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9f923eb053795d41f30 HTTP/1.1" 200 None 2026-04-02 00:37:56,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:56,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5e523eb053795d41ede HTTP/1.1" 200 None 2026-04-02 00:37:56,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:37:56,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdb9f923eb053795d41f30 HTTP/1.1" 200 None 2026-04-02 00:37:56,176 - lava_callback - INFO - Completed processing callback for node 69cdb9f923eb053795d41f30 INFO: 172.20.0.1:59964 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:38:04,517 - lava_callback - INFO - Received callback for node 69cdb9f623eb053795d41f27 INFO: 172.20.0.1:39244 - "POST /node/69cdb9f623eb053795d41f27 HTTP/1.0" 202 Accepted 2026-04-02 00:38:04,523 - lava_callback - INFO - Starting async job submit for node 69cdb9f623eb053795d41f27 2026-04-02 00:38:04,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:38:04,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9f623eb053795d41f27 HTTP/1.1" 200 None 2026-04-02 00:38:04,557 - lava_callback - DEBUG - Processing node 69cdb9f623eb053795d41f27: baseline-x86 2026-04-02 00:38:04,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:38:05,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:38:05,004 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f623eb053795d41f27/log.txt.gz 2026-04-02 00:38:05,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:38:05,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:38:05,388 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f623eb053795d41f27/lava-logs.yaml 2026-04-02 00:38:05,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:38:05,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:38:05,571 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f623eb053795d41f27/lava_callback.json.gz 2026-04-02 00:38:05,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:38:05,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9f623eb053795d41f27 HTTP/1.1" 200 None 2026-04-02 00:38:05,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:38:05,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5e523eb053795d41ede HTTP/1.1" 200 None 2026-04-02 00:38:05,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:38:05,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdb9f623eb053795d41f27 HTTP/1.1" 200 None 2026-04-02 00:38:05,658 - lava_callback - INFO - Completed processing callback for node 69cdb9f623eb053795d41f27 INFO: 172.20.0.1:39246 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:38:07,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:38:07,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52112 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:39:30,172 - lava_callback - INFO - Received callback for node 69cb17ad8c06c2799bf550d6 INFO: 172.20.0.1:48774 - "POST /node/69cb17ad8c06c2799bf550d6 HTTP/1.0" 202 Accepted 2026-04-02 00:39:30,179 - lava_callback - INFO - Starting async job submit for node 69cb17ad8c06c2799bf550d6 2026-04-02 00:39:30,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:39:30,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17ad8c06c2799bf550d6 HTTP/1.1" 200 None 2026-04-02 00:39:30,195 - lava_callback - DEBUG - Processing node 69cb17ad8c06c2799bf550d6: baseline-arm-clabbe 2026-04-02 00:39:30,195 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb17ad8c06c2799bf550d6 2026-04-02 00:39:30,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:39:30,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17ad8c06c2799bf550d6 HTTP/1.1" 200 None 2026-04-02 00:39:30,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:39:30,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:39:30,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:39:30,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb17ad8c06c2799bf550d6 HTTP/1.1" 200 None 2026-04-02 00:39:30,263 - lava_callback - ERROR - Error processing callback for node 69cb17ad8c06c2799bf550d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33666 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:39:56,143 - lava_callback - INFO - Received callback for node 69cb17d48c06c2799bf550f6 INFO: 172.20.0.1:40098 - "POST /node/69cb17d48c06c2799bf550f6 HTTP/1.0" 202 Accepted 2026-04-02 00:39:56,149 - lava_callback - INFO - Starting async job submit for node 69cb17d48c06c2799bf550f6 2026-04-02 00:39:56,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:39:56,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17d48c06c2799bf550f6 HTTP/1.1" 200 None 2026-04-02 00:39:56,170 - lava_callback - DEBUG - Processing node 69cb17d48c06c2799bf550f6: baseline-arm-baylibre 2026-04-02 00:39:56,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb17d48c06c2799bf550f6 2026-04-02 00:39:56,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:39:56,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb17d48c06c2799bf550f6 HTTP/1.1" 200 None 2026-04-02 00:39:56,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:39:56,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:39:56,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:39:56,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb17d48c06c2799bf550f6 HTTP/1.1" 200 None 2026-04-02 00:39:56,223 - lava_callback - ERROR - Error processing callback for node 69cb17d48c06c2799bf550f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50234 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:40:50,161 - lava_callback - INFO - Received callback for node 69cb18008c06c2799bf5510f INFO: 172.20.0.1:53378 - "POST /node/69cb18008c06c2799bf5510f HTTP/1.0" 202 Accepted 2026-04-02 00:40:50,166 - lava_callback - INFO - Starting async job submit for node 69cb18008c06c2799bf5510f 2026-04-02 00:40:50,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:40:50,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb18008c06c2799bf5510f HTTP/1.1" 200 None 2026-04-02 00:40:50,182 - lava_callback - DEBUG - Processing node 69cb18008c06c2799bf5510f: baseline-arm-clabbe 2026-04-02 00:40:50,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb18008c06c2799bf5510f 2026-04-02 00:40:50,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:40:50,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb18008c06c2799bf5510f HTTP/1.1" 200 None 2026-04-02 00:40:50,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:40:50,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:40:50,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:40:50,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb18008c06c2799bf5510f HTTP/1.1" 200 None 2026-04-02 00:40:50,289 - lava_callback - ERROR - Error processing callback for node 69cb18008c06c2799bf5510f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57104 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:42:36,343 - lava_callback - INFO - Received callback for node 69cb186f8c06c2799bf55127 INFO: 172.20.0.1:49212 - "POST /node/69cb186f8c06c2799bf55127 HTTP/1.0" 202 Accepted 2026-04-02 00:42:36,348 - lava_callback - INFO - Starting async job submit for node 69cb186f8c06c2799bf55127 2026-04-02 00:42:36,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:42:36,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb186f8c06c2799bf55127 HTTP/1.1" 200 None 2026-04-02 00:42:36,363 - lava_callback - DEBUG - Processing node 69cb186f8c06c2799bf55127: baseline-arm-baylibre 2026-04-02 00:42:36,363 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb186f8c06c2799bf55127 2026-04-02 00:42:36,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:42:36,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb186f8c06c2799bf55127 HTTP/1.1" 200 None 2026-04-02 00:42:36,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:42:36,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:42:36,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:42:36,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb186f8c06c2799bf55127 HTTP/1.1" 200 None 2026-04-02 00:42:36,426 - lava_callback - ERROR - Error processing callback for node 69cb186f8c06c2799bf55127: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36470 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 00:43:16,317 - lava_callback - INFO - Received callback for node 69cb18968c06c2799bf5512e INFO: 172.20.0.1:36478 - "POST /node/69cb18968c06c2799bf5512e HTTP/1.0" 202 Accepted 2026-04-02 00:43:16,322 - lava_callback - INFO - Starting async job submit for node 69cb18968c06c2799bf5512e 2026-04-02 00:43:16,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:43:16,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb18968c06c2799bf5512e HTTP/1.1" 200 None 2026-04-02 00:43:16,338 - lava_callback - DEBUG - Processing node 69cb18968c06c2799bf5512e: baseline-arm-baylibre 2026-04-02 00:43:16,338 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb18968c06c2799bf5512e 2026-04-02 00:43:16,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:43:16,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb18968c06c2799bf5512e HTTP/1.1" 200 None 2026-04-02 00:43:16,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:43:16,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 00:43:16,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:43:16,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb18968c06c2799bf5512e HTTP/1.1" 200 None 2026-04-02 00:43:16,418 - lava_callback - ERROR - Error processing callback for node 69cb18968c06c2799bf5512e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49528 - "GET /node/69c2d8a46778148209fd6469%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:49538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38708 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:43:56,377 - lava_callback - INFO - Received callback for node 69cb18c18c06c2799bf5513a INFO: 172.20.0.1:43494 - "POST /node/69cb18c18c06c2799bf5513a HTTP/1.0" 202 Accepted 2026-04-02 00:43:56,383 - lava_callback - INFO - Starting async job submit for node 69cb18c18c06c2799bf5513a 2026-04-02 00:43:56,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:43:56,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb18c18c06c2799bf5513a HTTP/1.1" 200 None 2026-04-02 00:43:56,400 - lava_callback - DEBUG - Processing node 69cb18c18c06c2799bf5513a: baseline-arm-baylibre 2026-04-02 00:43:56,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb18c18c06c2799bf5513a 2026-04-02 00:43:56,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:43:56,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb18c18c06c2799bf5513a HTTP/1.1" 200 None 2026-04-02 00:43:56,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:43:56,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 00:43:56,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:43:56,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb18c18c06c2799bf5513a HTTP/1.1" 200 None 2026-04-02 00:43:56,463 - lava_callback - ERROR - Error processing callback for node 69cb18c18c06c2799bf5513a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36418 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:44:36,388 - lava_callback - INFO - Received callback for node 69cb18e98c06c2799bf5514d INFO: 172.20.0.1:56372 - "POST /node/69cb18e98c06c2799bf5514d HTTP/1.0" 202 Accepted 2026-04-02 00:44:36,394 - lava_callback - INFO - Starting async job submit for node 69cb18e98c06c2799bf5514d 2026-04-02 00:44:36,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:44:36,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb18e98c06c2799bf5514d HTTP/1.1" 200 None 2026-04-02 00:44:36,409 - lava_callback - DEBUG - Processing node 69cb18e98c06c2799bf5514d: baseline-arm-baylibre 2026-04-02 00:44:36,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb18e98c06c2799bf5514d 2026-04-02 00:44:36,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:44:36,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb18e98c06c2799bf5514d HTTP/1.1" 200 None 2026-04-02 00:44:36,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:44:36,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 00:44:36,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:44:36,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb18e98c06c2799bf5514d HTTP/1.1" 200 None 2026-04-02 00:44:36,475 - lava_callback - ERROR - Error processing callback for node 69cb18e98c06c2799bf5514d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59542 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:45:30,362 - lava_callback - INFO - Received callback for node 69cb19158c06c2799bf5515c INFO: 172.20.0.1:48382 - "POST /node/69cb19158c06c2799bf5515c HTTP/1.0" 202 Accepted 2026-04-02 00:45:30,369 - lava_callback - INFO - Starting async job submit for node 69cb19158c06c2799bf5515c 2026-04-02 00:45:30,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:30,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb19158c06c2799bf5515c HTTP/1.1" 200 None 2026-04-02 00:45:30,387 - lava_callback - DEBUG - Processing node 69cb19158c06c2799bf5515c: baseline-arm-clabbe 2026-04-02 00:45:30,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb19158c06c2799bf5515c 2026-04-02 00:45:30,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:30,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb19158c06c2799bf5515c HTTP/1.1" 200 None 2026-04-02 00:45:30,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:30,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:45:30,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:30,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb19158c06c2799bf5515c HTTP/1.1" 200 None 2026-04-02 00:45:30,474 - lava_callback - ERROR - Error processing callback for node 69cb19158c06c2799bf5515c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:45:30,564 - lava_callback - INFO - Received callback for node 69cdbc1723eb053795d42007 INFO: 172.20.0.1:48388 - "POST /node/69cdbc1723eb053795d42007 HTTP/1.0" 202 Accepted 2026-04-02 00:45:30,571 - lava_callback - INFO - Starting async job submit for node 69cdbc1723eb053795d42007 2026-04-02 00:45:30,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:30,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1723eb053795d42007 HTTP/1.1" 200 None 2026-04-02 00:45:30,608 - lava_callback - DEBUG - Processing node 69cdbc1723eb053795d42007: baseline-arm 2026-04-02 00:45:30,618 - lava_callback - WARNING - Failed to upload log 2026-04-02 00:45:30,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:45:30,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:45:30,774 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1723eb053795d42007/lava-logs.yaml 2026-04-02 00:45:30,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:45:31,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:45:31,017 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1723eb053795d42007/lava_callback.json.gz 2026-04-02 00:45:31,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:31,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1723eb053795d42007 HTTP/1.1" 200 None 2026-04-02 00:45:31,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:31,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:45:31,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:31,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc1723eb053795d42007 HTTP/1.1" 200 None 2026-04-02 00:45:31,107 - lava_callback - INFO - Completed processing callback for node 69cdbc1723eb053795d42007 INFO: 172.20.0.1:48392 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:45:37,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:37,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37276 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:45:49,610 - lava_callback - INFO - Received callback for node 69cdbc1a23eb053795d42013 INFO: 172.20.0.1:37286 - "POST /node/69cdbc1a23eb053795d42013 HTTP/1.0" 202 Accepted 2026-04-02 00:45:49,658 - lava_callback - INFO - Starting async job submit for node 69cdbc1a23eb053795d42013 2026-04-02 00:45:49,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:49,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1a23eb053795d42013 HTTP/1.1" 200 None 2026-04-02 00:45:49,691 - lava_callback - DEBUG - Processing node 69cdbc1a23eb053795d42013: baseline-arm 2026-04-02 00:45:49,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:45:49,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:45:49,957 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1a23eb053795d42013/log.txt.gz 2026-04-02 00:45:50,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:45:50,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:45:50,229 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1a23eb053795d42013/lava-logs.yaml 2026-04-02 00:45:50,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:45:50,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:45:50,375 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1a23eb053795d42013/lava_callback.json.gz 2026-04-02 00:45:50,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:50,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1a23eb053795d42013 HTTP/1.1" 200 None 2026-04-02 00:45:50,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:50,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:45:50,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:50,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc1a23eb053795d42013 HTTP/1.1" 200 None 2026-04-02 00:45:50,474 - lava_callback - INFO - Completed processing callback for node 69cdbc1a23eb053795d42013 2026-04-02 00:45:52,711 - lava_callback - INFO - Received callback for node 69cdbc1823eb053795d4200a INFO: 172.20.0.1:49082 - "POST /node/69cdbc1823eb053795d4200a HTTP/1.0" 202 Accepted 2026-04-02 00:45:52,759 - lava_callback - INFO - Starting async job submit for node 69cdbc1823eb053795d4200a 2026-04-02 00:45:52,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:52,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1823eb053795d4200a HTTP/1.1" 200 None 2026-04-02 00:45:52,798 - lava_callback - DEBUG - Processing node 69cdbc1823eb053795d4200a: baseline-arm 2026-04-02 00:45:52,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:45:53,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:45:53,067 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1823eb053795d4200a/log.txt.gz 2026-04-02 00:45:53,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:45:53,323 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:45:53,325 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1823eb053795d4200a/lava-logs.yaml 2026-04-02 00:45:53,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:45:53,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:45:53,485 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1823eb053795d4200a/lava_callback.json.gz 2026-04-02 00:45:53,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:53,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1823eb053795d4200a HTTP/1.1" 200 None 2026-04-02 00:45:53,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:53,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:45:53,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:45:53,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc1823eb053795d4200a HTTP/1.1" 200 None 2026-04-02 00:45:53,571 - lava_callback - INFO - Completed processing callback for node 69cdbc1823eb053795d4200a INFO: 172.20.0.1:49090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38802 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:46:07,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:07,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 00:46:10,246 - lava_callback - INFO - Received callback for node 69cb193f8c06c2799bf55165 INFO: 172.20.0.1:45720 - "POST /node/69cb193f8c06c2799bf55165 HTTP/1.0" 202 Accepted 2026-04-02 00:46:10,255 - lava_callback - INFO - Starting async job submit for node 69cb193f8c06c2799bf55165 2026-04-02 00:46:10,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:10,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb193f8c06c2799bf55165 HTTP/1.1" 200 None 2026-04-02 00:46:10,278 - lava_callback - DEBUG - Processing node 69cb193f8c06c2799bf55165: baseline-arm-clabbe 2026-04-02 00:46:10,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb193f8c06c2799bf55165 2026-04-02 00:46:10,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:10,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb193f8c06c2799bf55165 HTTP/1.1" 200 None 2026-04-02 00:46:10,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:10,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 00:46:10,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:10,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb193f8c06c2799bf55165 HTTP/1.1" 200 None 2026-04-02 00:46:10,379 - lava_callback - ERROR - Error processing callback for node 69cb193f8c06c2799bf55165: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39116 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:46:24,567 - lava_callback - INFO - Received callback for node 69cdbc1923eb053795d42010 INFO: 172.20.0.1:39118 - "POST /node/69cdbc1923eb053795d42010 HTTP/1.0" 202 Accepted 2026-04-02 00:46:24,574 - lava_callback - INFO - Starting async job submit for node 69cdbc1923eb053795d42010 2026-04-02 00:46:24,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:24,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1923eb053795d42010 HTTP/1.1" 200 None 2026-04-02 00:46:24,673 - lava_callback - DEBUG - Processing node 69cdbc1923eb053795d42010: baseline-arm 2026-04-02 00:46:24,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:46:25,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:46:25,031 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1923eb053795d42010/log.txt.gz 2026-04-02 00:46:25,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:46:25,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:46:25,373 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1923eb053795d42010/lava-logs.yaml 2026-04-02 00:46:25,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:46:25,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:46:25,612 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1923eb053795d42010/lava_callback.json.gz 2026-04-02 00:46:25,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:25,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1923eb053795d42010 HTTP/1.1" 200 None 2026-04-02 00:46:25,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:25,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:46:25,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:25,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc1923eb053795d42010 HTTP/1.1" 200 None 2026-04-02 00:46:25,701 - lava_callback - INFO - Completed processing callback for node 69cdbc1923eb053795d42010 INFO: 172.20.0.1:39130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44068 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:46:37,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:37,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 00:46:44,674 - lava_callback - INFO - Received callback for node 69cdbc1623eb053795d42004 INFO: 172.20.0.1:43810 - "POST /node/69cdbc1623eb053795d42004 HTTP/1.0" 202 Accepted 2026-04-02 00:46:44,679 - lava_callback - INFO - Starting async job submit for node 69cdbc1623eb053795d42004 2026-04-02 00:46:44,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:44,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1623eb053795d42004 HTTP/1.1" 200 None 2026-04-02 00:46:44,730 - lava_callback - DEBUG - Processing node 69cdbc1623eb053795d42004: baseline-arm 2026-04-02 00:46:44,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:46:45,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:46:45,058 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1623eb053795d42004/log.txt.gz 2026-04-02 00:46:45,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:46:45,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:46:45,354 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1623eb053795d42004/lava-logs.yaml 2026-04-02 00:46:45,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:46:45,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:46:45,532 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1623eb053795d42004/lava_callback.json.gz 2026-04-02 00:46:45,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:45,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1623eb053795d42004 HTTP/1.1" 200 None 2026-04-02 00:46:45,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:45,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:46:45,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:45,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc1623eb053795d42004 HTTP/1.1" 200 None 2026-04-02 00:46:45,627 - lava_callback - INFO - Completed processing callback for node 69cdbc1623eb053795d42004 INFO: 172.20.0.1:43814 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:46:48,053 - lava_callback - INFO - Received callback for node 69cdbc2823eb053795d42045 INFO: 172.20.0.1:43830 - "POST /node/69cdbc2823eb053795d42045 HTTP/1.0" 202 Accepted 2026-04-02 00:46:48,059 - lava_callback - INFO - Starting async job submit for node 69cdbc2823eb053795d42045 2026-04-02 00:46:48,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:48,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2823eb053795d42045 HTTP/1.1" 200 None 2026-04-02 00:46:48,098 - lava_callback - DEBUG - Processing node 69cdbc2823eb053795d42045: baseline-arm-broonie 2026-04-02 00:46:48,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:46:48,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:46:48,473 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2823eb053795d42045/log.txt.gz 2026-04-02 00:46:48,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:46:48,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:46:48,800 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2823eb053795d42045/lava-logs.yaml 2026-04-02 00:46:48,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:46:48,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:46:48,964 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2823eb053795d42045/lava_callback.json.gz 2026-04-02 00:46:49,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:49,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2823eb053795d42045 HTTP/1.1" 200 None 2026-04-02 00:46:49,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:49,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:46:49,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:49,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc2823eb053795d42045 HTTP/1.1" 200 None 2026-04-02 00:46:49,084 - lava_callback - INFO - Completed processing callback for node 69cdbc2823eb053795d42045 2026-04-02 00:46:50,461 - lava_callback - INFO - Received callback for node 69cb19688c06c2799bf5516f INFO: 172.20.0.1:42758 - "POST /node/69cb19688c06c2799bf5516f HTTP/1.0" 202 Accepted 2026-04-02 00:46:50,466 - lava_callback - INFO - Starting async job submit for node 69cb19688c06c2799bf5516f 2026-04-02 00:46:50,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:50,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb19688c06c2799bf5516f HTTP/1.1" 200 None 2026-04-02 00:46:50,483 - lava_callback - DEBUG - Processing node 69cb19688c06c2799bf5516f: baseline-arm-clabbe 2026-04-02 00:46:50,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb19688c06c2799bf5516f 2026-04-02 00:46:50,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:50,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb19688c06c2799bf5516f HTTP/1.1" 200 None 2026-04-02 00:46:50,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:50,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 00:46:50,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:46:50,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb19688c06c2799bf5516f HTTP/1.1" 200 None 2026-04-02 00:46:50,549 - lava_callback - ERROR - Error processing callback for node 69cb19688c06c2799bf5516f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42774 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:47:02,163 - lava_callback - INFO - Received callback for node 69cdbc3b23eb053795d420af INFO: 172.20.0.1:38408 - "POST /node/69cdbc3b23eb053795d420af HTTP/1.0" 202 Accepted 2026-04-02 00:47:02,213 - lava_callback - INFO - Starting async job submit for node 69cdbc3b23eb053795d420af 2026-04-02 00:47:02,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:02,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3b23eb053795d420af HTTP/1.1" 200 None 2026-04-02 00:47:02,255 - lava_callback - DEBUG - Processing node 69cdbc3b23eb053795d420af: baseline-arm-pengutronix 2026-04-02 00:47:02,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:02,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:02,519 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3b23eb053795d420af/log.txt.gz 2026-04-02 00:47:02,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:02,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:02,825 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3b23eb053795d420af/lava-logs.yaml 2026-04-02 00:47:02,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:02,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:02,965 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3b23eb053795d420af/lava_callback.json.gz 2026-04-02 00:47:02,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:03,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3b23eb053795d420af HTTP/1.1" 200 None 2026-04-02 00:47:03,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:03,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:47:03,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:03,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc3b23eb053795d420af HTTP/1.1" 200 None 2026-04-02 00:47:03,077 - lava_callback - INFO - Completed processing callback for node 69cdbc3b23eb053795d420af INFO: 172.20.0.1:38416 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:47:06,889 - lava_callback - INFO - Received callback for node 69cdbc2923eb053795d42048 INFO: 172.20.0.1:38424 - "POST /node/69cdbc2923eb053795d42048 HTTP/1.0" 202 Accepted 2026-04-02 00:47:06,899 - lava_callback - INFO - Starting async job submit for node 69cdbc2923eb053795d42048 2026-04-02 00:47:06,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:06,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2923eb053795d42048 HTTP/1.1" 200 None 2026-04-02 00:47:06,944 - lava_callback - DEBUG - Processing node 69cdbc2923eb053795d42048: baseline-arm-broonie 2026-04-02 00:47:07,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:07,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:07,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2923eb053795d42048/log.txt.gz 2026-04-02 00:47:07,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:07,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:07,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 00:47:07,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:07,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2923eb053795d42048/lava-logs.yaml 2026-04-02 00:47:07,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:07,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:07,851 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2923eb053795d42048/lava_callback.json.gz 2026-04-02 00:47:07,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:07,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2923eb053795d42048 HTTP/1.1" 200 None 2026-04-02 00:47:07,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:07,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:47:07,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:07,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc2923eb053795d42048 HTTP/1.1" 200 None 2026-04-02 00:47:07,940 - lava_callback - INFO - Completed processing callback for node 69cdbc2923eb053795d42048 2026-04-02 00:47:10,102 - lava_callback - INFO - Received callback for node 69cdbc2a23eb053795d4204d INFO: 172.20.0.1:34270 - "POST /node/69cdbc2a23eb053795d4204d HTTP/1.0" 202 Accepted 2026-04-02 00:47:10,108 - lava_callback - INFO - Starting async job submit for node 69cdbc2a23eb053795d4204d 2026-04-02 00:47:10,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:10,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2a23eb053795d4204d HTTP/1.1" 200 None 2026-04-02 00:47:10,146 - lava_callback - DEBUG - Processing node 69cdbc2a23eb053795d4204d: baseline-arm-broonie 2026-04-02 00:47:10,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:10,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:10,509 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2a23eb053795d4204d/log.txt.gz 2026-04-02 00:47:10,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:10,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:10,836 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2a23eb053795d4204d/lava-logs.yaml 2026-04-02 00:47:10,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:11,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:11,002 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2a23eb053795d4204d/lava_callback.json.gz 2026-04-02 00:47:11,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:11,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2a23eb053795d4204d HTTP/1.1" 200 None 2026-04-02 00:47:11,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:11,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:47:11,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:11,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc2a23eb053795d4204d HTTP/1.1" 200 None 2026-04-02 00:47:11,113 - lava_callback - INFO - Completed processing callback for node 69cdbc2a23eb053795d4204d INFO: 172.20.0.1:34274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55322 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:47:30,459 - lava_callback - INFO - Received callback for node 69cb19918c06c2799bf55178 INFO: 172.20.0.1:52914 - "POST /node/69cb19918c06c2799bf55178 HTTP/1.0" 202 Accepted 2026-04-02 00:47:30,468 - lava_callback - INFO - Starting async job submit for node 69cb19918c06c2799bf55178 2026-04-02 00:47:30,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:30,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb19918c06c2799bf55178 HTTP/1.1" 200 None 2026-04-02 00:47:30,483 - lava_callback - DEBUG - Processing node 69cb19918c06c2799bf55178: baseline-arm-clabbe 2026-04-02 00:47:30,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb19918c06c2799bf55178 2026-04-02 00:47:30,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:30,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb19918c06c2799bf55178 HTTP/1.1" 200 None 2026-04-02 00:47:30,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:30,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 00:47:30,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:30,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb19918c06c2799bf55178 HTTP/1.1" 200 None 2026-04-02 00:47:30,554 - lava_callback - ERROR - Error processing callback for node 69cb19918c06c2799bf55178: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:47:33,056 - lava_callback - INFO - Received callback for node 69cdbc2f23eb053795d42062 INFO: 172.20.0.1:52922 - "POST /node/69cdbc2f23eb053795d42062 HTTP/1.0" 202 Accepted 2026-04-02 00:47:33,064 - lava_callback - INFO - Starting async job submit for node 69cdbc2f23eb053795d42062 2026-04-02 00:47:33,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:33,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2f23eb053795d42062 HTTP/1.1" 200 None 2026-04-02 00:47:33,141 - lava_callback - DEBUG - Processing node 69cdbc2f23eb053795d42062: baseline-arm-clabbe 2026-04-02 00:47:33,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:33,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:33,497 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cdbc2f23eb053795d42062/log.txt.gz 2026-04-02 00:47:33,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:33,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:33,835 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cdbc2f23eb053795d42062/lava-logs.yaml 2026-04-02 00:47:33,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:33,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:33,976 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cdbc2f23eb053795d42062/lava_callback.json.gz 2026-04-02 00:47:34,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:34,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2f23eb053795d42062 HTTP/1.1" 200 None 2026-04-02 00:47:34,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:34,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:47:34,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:34,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc2f23eb053795d42062 HTTP/1.1" 200 None 2026-04-02 00:47:34,062 - lava_callback - INFO - Completed processing callback for node 69cdbc2f23eb053795d42062 INFO: 172.20.0.1:52930 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:47:37,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:37,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60716 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:47:50,330 - lava_callback - INFO - Received callback for node 69cdbc2623eb053795d4203e INFO: 172.20.0.1:47680 - "POST /node/69cdbc2623eb053795d4203e HTTP/1.0" 202 Accepted 2026-04-02 00:47:50,340 - lava_callback - INFO - Starting async job submit for node 69cdbc2623eb053795d4203e 2026-04-02 00:47:50,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:50,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2623eb053795d4203e HTTP/1.1" 200 None 2026-04-02 00:47:50,400 - lava_callback - DEBUG - Processing node 69cdbc2623eb053795d4203e: baseline-arm-broonie 2026-04-02 00:47:50,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:50,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:50,803 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2623eb053795d4203e/log.txt.gz 2026-04-02 00:47:50,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:51,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:51,162 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2623eb053795d4203e/lava-logs.yaml 2026-04-02 00:47:51,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:47:51,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:47:51,349 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cdbc2623eb053795d4203e/lava_callback.json.gz 2026-04-02 00:47:51,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:51,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2623eb053795d4203e HTTP/1.1" 200 None 2026-04-02 00:47:51,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:51,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:47:51,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:47:51,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc2623eb053795d4203e HTTP/1.1" 200 None 2026-04-02 00:47:51,458 - lava_callback - INFO - Completed processing callback for node 69cdbc2623eb053795d4203e INFO: 172.20.0.1:47692 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:48:01,005 - lava_callback - INFO - Received callback for node 69cdbcb623eb053795d4215e INFO: 172.20.0.1:33016 - "POST /node/69cdbcb623eb053795d4215e HTTP/1.0" 202 Accepted 2026-04-02 00:48:01,011 - lava_callback - INFO - Starting async job submit for node 69cdbcb623eb053795d4215e 2026-04-02 00:48:01,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:01,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbcb623eb053795d4215e HTTP/1.1" 200 None 2026-04-02 00:48:01,029 - lava_callback - DEBUG - Processing node 69cdbcb623eb053795d4215e: baseline-arm 2026-04-02 00:48:01,035 - lava_callback - WARNING - Failed to upload log 2026-04-02 00:48:01,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:48:01,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:01,202 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbcb623eb053795d4215e/lava-logs.yaml 2026-04-02 00:48:01,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:48:01,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:01,343 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbcb623eb053795d4215e/lava_callback.json.gz 2026-04-02 00:48:01,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:01,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbcb623eb053795d4215e HTTP/1.1" 200 None 2026-04-02 00:48:01,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:01,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:48:01,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:01,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbcb623eb053795d4215e HTTP/1.1" 200 None 2026-04-02 00:48:01,410 - lava_callback - INFO - Completed processing callback for node 69cdbcb623eb053795d4215e 2026-04-02 00:48:03,687 - lava_callback - INFO - Received callback for node 69cdbc5223eb053795d420d4 INFO: 172.20.0.1:33018 - "POST /node/69cdbc5223eb053795d420d4 HTTP/1.0" 202 Accepted 2026-04-02 00:48:03,697 - lava_callback - INFO - Starting async job submit for node 69cdbc5223eb053795d420d4 2026-04-02 00:48:03,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:03,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5223eb053795d420d4 HTTP/1.1" 200 None 2026-04-02 00:48:03,745 - lava_callback - DEBUG - Processing node 69cdbc5223eb053795d420d4: ltp-dio 2026-04-02 00:48:04,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:48:04,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:04,369 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69cdbc5223eb053795d420d4/log.txt.gz 2026-04-02 00:48:04,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:48:04,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:04,811 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69cdbc5223eb053795d420d4/lava-logs.yaml 2026-04-02 00:48:04,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:48:04,939 - lava_callback - INFO - Received callback for node 69cdbc3923eb053795d42085 2026-04-02 00:48:04,945 - lava_callback - INFO - Starting async job submit for node 69cdbc3923eb053795d42085 INFO: 172.20.0.1:33026 - "POST /node/69cdbc3923eb053795d42085 HTTP/1.0" 202 Accepted 2026-04-02 00:48:04,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:04,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:04,975 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69cdbc5223eb053795d420d4/lava_callback.json.gz 2026-04-02 00:48:05,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:05,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3923eb053795d42085 HTTP/1.1" 200 None 2026-04-02 00:48:05,009 - lava_callback - DEBUG - Processing node 69cdbc3923eb053795d42085: baseline-arm-pengutronix 2026-04-02 00:48:05,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5223eb053795d420d4 HTTP/1.1" 200 None 2026-04-02 00:48:05,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:05,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:48:05,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:05,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:48:05,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5223eb053795d420d4 HTTP/1.1" 200 None 2026-04-02 00:48:05,213 - lava_callback - INFO - Completed processing callback for node 69cdbc5223eb053795d420d4 2026-04-02 00:48:05,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:05,371 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3923eb053795d42085/log.txt.gz 2026-04-02 00:48:05,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:48:05,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:05,673 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3923eb053795d42085/lava-logs.yaml 2026-04-02 00:48:05,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:48:05,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:05,961 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3923eb053795d42085/lava_callback.json.gz 2026-04-02 00:48:06,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:06,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3923eb053795d42085 HTTP/1.1" 200 None 2026-04-02 00:48:06,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:06,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:48:06,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:06,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc3923eb053795d42085 HTTP/1.1" 200 None 2026-04-02 00:48:06,144 - lava_callback - INFO - Completed processing callback for node 69cdbc3923eb053795d42085 2026-04-02 00:48:06,163 - lava_callback - INFO - Received callback for node 69cdbc3823eb053795d42080 INFO: 172.20.0.1:33042 - "POST /node/69cdbc3823eb053795d42080 HTTP/1.0" 202 Accepted 2026-04-02 00:48:06,173 - lava_callback - INFO - Starting async job submit for node 69cdbc3823eb053795d42080 2026-04-02 00:48:06,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:06,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3823eb053795d42080 HTTP/1.1" 200 None 2026-04-02 00:48:06,266 - lava_callback - DEBUG - Processing node 69cdbc3823eb053795d42080: baseline-arm-pengutronix 2026-04-02 00:48:06,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33054 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:48:06,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:06,586 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3823eb053795d42080/log.txt.gz 2026-04-02 00:48:06,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:48:06,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:06,878 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3823eb053795d42080/lava-logs.yaml 2026-04-02 00:48:06,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:48:07,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:48:07,029 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3823eb053795d42080/lava_callback.json.gz 2026-04-02 00:48:07,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:07,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3823eb053795d42080 HTTP/1.1" 200 None 2026-04-02 00:48:07,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:07,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:48:07,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:07,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc3823eb053795d42080 HTTP/1.1" 200 None 2026-04-02 00:48:07,175 - lava_callback - INFO - Completed processing callback for node 69cdbc3823eb053795d42080 2026-04-02 00:48:07,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:07,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-02 00:48:10,296 - lava_callback - INFO - Received callback for node 69cb19ba8c06c2799bf55185 INFO: 172.20.0.1:34572 - "POST /node/69cb19ba8c06c2799bf55185 HTTP/1.0" 202 Accepted 2026-04-02 00:48:10,305 - lava_callback - INFO - Starting async job submit for node 69cb19ba8c06c2799bf55185 2026-04-02 00:48:10,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:10,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb19ba8c06c2799bf55185 HTTP/1.1" 200 None 2026-04-02 00:48:10,330 - lava_callback - DEBUG - Processing node 69cb19ba8c06c2799bf55185: baseline-arm-clabbe 2026-04-02 00:48:10,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb19ba8c06c2799bf55185 2026-04-02 00:48:10,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:10,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb19ba8c06c2799bf55185 HTTP/1.1" 200 None 2026-04-02 00:48:10,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:10,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 00:48:10,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:10,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb19ba8c06c2799bf55185 HTTP/1.1" 200 None 2026-04-02 00:48:10,396 - lava_callback - ERROR - Error processing callback for node 69cb19ba8c06c2799bf55185: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45206 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:48:50,352 - lava_callback - INFO - Received callback for node 69cb19e58c06c2799bf55190 INFO: 172.20.0.1:56452 - "POST /node/69cb19e58c06c2799bf55190 HTTP/1.0" 202 Accepted 2026-04-02 00:48:50,373 - lava_callback - INFO - Starting async job submit for node 69cb19e58c06c2799bf55190 2026-04-02 00:48:50,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:50,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb19e58c06c2799bf55190 HTTP/1.1" 200 None 2026-04-02 00:48:50,397 - lava_callback - DEBUG - Processing node 69cb19e58c06c2799bf55190: baseline-arm-clabbe 2026-04-02 00:48:50,398 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb19e58c06c2799bf55190 2026-04-02 00:48:50,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:50,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb19e58c06c2799bf55190 HTTP/1.1" 200 None 2026-04-02 00:48:50,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:50,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 00:48:50,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:48:50,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb19e58c06c2799bf55190 HTTP/1.1" 200 None 2026-04-02 00:48:50,458 - lava_callback - ERROR - Error processing callback for node 69cb19e58c06c2799bf55190: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42832 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:49:30,427 - lava_callback - INFO - Received callback for node 69cb1a0e8c06c2799bf5519a INFO: 172.20.0.1:45824 - "POST /node/69cb1a0e8c06c2799bf5519a HTTP/1.0" 202 Accepted 2026-04-02 00:49:30,433 - lava_callback - INFO - Starting async job submit for node 69cb1a0e8c06c2799bf5519a 2026-04-02 00:49:30,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:30,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1a0e8c06c2799bf5519a HTTP/1.1" 200 None 2026-04-02 00:49:30,449 - lava_callback - DEBUG - Processing node 69cb1a0e8c06c2799bf5519a: baseline-arm-clabbe 2026-04-02 00:49:30,449 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1a0e8c06c2799bf5519a 2026-04-02 00:49:30,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:30,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1a0e8c06c2799bf5519a HTTP/1.1" 200 None 2026-04-02 00:49:30,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:30,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:49:30,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:30,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1a0e8c06c2799bf5519a HTTP/1.1" 200 None 2026-04-02 00:49:30,519 - lava_callback - ERROR - Error processing callback for node 69cb1a0e8c06c2799bf5519a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:49:32,121 - lava_callback - INFO - Received callback for node 69cdbc3223eb053795d4206f INFO: 172.20.0.1:45838 - "POST /node/69cdbc3223eb053795d4206f HTTP/1.0" 202 Accepted 2026-04-02 00:49:32,128 - lava_callback - INFO - Starting async job submit for node 69cdbc3223eb053795d4206f 2026-04-02 00:49:32,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:32,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3223eb053795d4206f HTTP/1.1" 200 None 2026-04-02 00:49:32,168 - lava_callback - DEBUG - Processing node 69cdbc3223eb053795d4206f: baseline-arm-clabbe 2026-04-02 00:49:32,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:49:32,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:49:32,583 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cdbc3223eb053795d4206f/log.txt.gz 2026-04-02 00:49:32,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:49:32,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:49:32,906 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cdbc3223eb053795d4206f/lava-logs.yaml 2026-04-02 00:49:32,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:49:33,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:49:33,048 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cdbc3223eb053795d4206f/lava_callback.json.gz 2026-04-02 00:49:33,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:33,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3223eb053795d4206f HTTP/1.1" 200 None 2026-04-02 00:49:33,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:33,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:49:33,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:33,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc3223eb053795d4206f HTTP/1.1" 200 None 2026-04-02 00:49:33,139 - lava_callback - INFO - Completed processing callback for node 69cdbc3223eb053795d4206f INFO: 172.20.0.1:45842 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:49:37,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:37,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56472 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:49:47,181 - lava_callback - INFO - Received callback for node 69cdb88723eb053795d41f00 INFO: 172.20.0.1:56480 - "POST /node/69cdb88723eb053795d41f00 HTTP/1.0" 202 Accepted 2026-04-02 00:49:47,189 - lava_callback - INFO - Starting async job submit for node 69cdb88723eb053795d41f00 2026-04-02 00:49:47,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:47,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb88723eb053795d41f00 HTTP/1.1" 200 None 2026-04-02 00:49:47,293 - lava_callback - DEBUG - Processing node 69cdb88723eb053795d41f00: ltp-smoke 2026-04-02 00:49:47,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:49:48,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:49:48,034 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdb88723eb053795d41f00/log.txt.gz 2026-04-02 00:49:48,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:49:48,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:49:48,737 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdb88723eb053795d41f00/lava-logs.yaml 2026-04-02 00:49:48,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:49:48,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:49:48,906 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdb88723eb053795d41f00/lava_callback.json.gz 2026-04-02 00:49:49,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:49,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb88723eb053795d41f00 HTTP/1.1" 200 None 2026-04-02 00:49:49,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:49,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5bf23eb053795d41ebd HTTP/1.1" 200 None 2026-04-02 00:49:49,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:49:49,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdb88723eb053795d41f00 HTTP/1.1" 200 None 2026-04-02 00:49:49,207 - lava_callback - INFO - Completed processing callback for node 69cdb88723eb053795d41f00 INFO: 172.20.0.1:41572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36074 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:50:07,505 - lava_callback - INFO - Received callback for node 69cdbc4d23eb053795d420c5 INFO: 172.20.0.1:36088 - "POST /node/69cdbc4d23eb053795d420c5 HTTP/1.0" 202 Accepted 2026-04-02 00:50:07,511 - lava_callback - INFO - Starting async job submit for node 69cdbc4d23eb053795d420c5 2026-04-02 00:50:07,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:07,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:07,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc4d23eb053795d420c5 HTTP/1.1" 200 None 2026-04-02 00:50:07,556 - lava_callback - DEBUG - Processing node 69cdbc4d23eb053795d420c5: ltp-containers 2026-04-02 00:50:07,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 00:50:07,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:08,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:08,090 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69cdbc4d23eb053795d420c5/log.txt.gz 2026-04-02 00:50:08,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:08,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:08,506 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69cdbc4d23eb053795d420c5/lava-logs.yaml 2026-04-02 00:50:08,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:08,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:08,670 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69cdbc4d23eb053795d420c5/lava_callback.json.gz 2026-04-02 00:50:08,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:08,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc4d23eb053795d420c5 HTTP/1.1" 200 None 2026-04-02 00:50:08,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:08,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:50:08,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:08,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc4d23eb053795d420c5 HTTP/1.1" 200 None 2026-04-02 00:50:08,772 - lava_callback - INFO - Completed processing callback for node 69cdbc4d23eb053795d420c5 2026-04-02 00:50:10,460 - lava_callback - INFO - Received callback for node 69cdbc4b23eb053795d420be INFO: 172.20.0.1:54884 - "POST /node/69cdbc4b23eb053795d420be HTTP/1.0" 202 Accepted 2026-04-02 00:50:10,466 - lava_callback - INFO - Starting async job submit for node 69cdbc4b23eb053795d420be 2026-04-02 00:50:10,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:10,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc4b23eb053795d420be HTTP/1.1" 200 None 2026-04-02 00:50:10,508 - lava_callback - DEBUG - Processing node 69cdbc4b23eb053795d420be: ltp-cap_bounds 2026-04-02 00:50:10,603 - lava_callback - INFO - Received callback for node 69cb1a398c06c2799bf551a8 INFO: 172.20.0.1:54898 - "POST /node/69cb1a398c06c2799bf551a8 HTTP/1.0" 202 Accepted 2026-04-02 00:50:10,625 - lava_callback - INFO - Starting async job submit for node 69cb1a398c06c2799bf551a8 2026-04-02 00:50:10,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:10,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1a398c06c2799bf551a8 HTTP/1.1" 200 None 2026-04-02 00:50:10,754 - lava_callback - DEBUG - Processing node 69cb1a398c06c2799bf551a8: baseline-arm-clabbe 2026-04-02 00:50:10,755 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1a398c06c2799bf551a8 2026-04-02 00:50:10,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:10,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1a398c06c2799bf551a8 HTTP/1.1" 200 None 2026-04-02 00:50:10,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:10,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:10,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 00:50:10,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:10,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1a398c06c2799bf551a8 HTTP/1.1" 200 None 2026-04-02 00:50:10,965 - lava_callback - ERROR - Error processing callback for node 69cb1a398c06c2799bf551a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:50:11,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:11,074 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cdbc4b23eb053795d420be/log.txt.gz 2026-04-02 00:50:11,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:11,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:11,572 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cdbc4b23eb053795d420be/lava-logs.yaml 2026-04-02 00:50:11,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:11,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:11,758 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cdbc4b23eb053795d420be/lava_callback.json.gz 2026-04-02 00:50:11,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:11,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc4b23eb053795d420be HTTP/1.1" 200 None 2026-04-02 00:50:11,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:11,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:50:11,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:11,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc4b23eb053795d420be HTTP/1.1" 200 None 2026-04-02 00:50:11,855 - lava_callback - INFO - Completed processing callback for node 69cdbc4b23eb053795d420be 2026-04-02 00:50:12,022 - lava_callback - INFO - Received callback for node 69cdbc4e23eb053795d420c8 INFO: 172.20.0.1:54910 - "POST /node/69cdbc4e23eb053795d420c8 HTTP/1.0" 202 Accepted 2026-04-02 00:50:12,032 - lava_callback - INFO - Starting async job submit for node 69cdbc4e23eb053795d420c8 2026-04-02 00:50:12,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:12,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc4e23eb053795d420c8 HTTP/1.1" 200 None 2026-04-02 00:50:12,086 - lava_callback - DEBUG - Processing node 69cdbc4e23eb053795d420c8: ltp-controllers 2026-04-02 00:50:12,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:12,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:12,687 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69cdbc4e23eb053795d420c8/log.txt.gz 2026-04-02 00:50:12,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:13,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:13,199 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69cdbc4e23eb053795d420c8/lava-logs.yaml 2026-04-02 00:50:13,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:13,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:13,372 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69cdbc4e23eb053795d420c8/lava_callback.json.gz 2026-04-02 00:50:13,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:13,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc4e23eb053795d420c8 HTTP/1.1" 200 None 2026-04-02 00:50:13,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:13,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:50:13,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:13,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc4e23eb053795d420c8 HTTP/1.1" 200 None 2026-04-02 00:50:13,482 - lava_callback - INFO - Completed processing callback for node 69cdbc4e23eb053795d420c8 INFO: 172.20.0.1:54918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45018 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:50:30,266 - lava_callback - INFO - Received callback for node 69cdbc5d23eb053795d42100 INFO: 172.20.0.1:40356 - "POST /node/69cdbc5d23eb053795d42100 HTTP/1.0" 202 Accepted 2026-04-02 00:50:30,273 - lava_callback - INFO - Starting async job submit for node 69cdbc5d23eb053795d42100 2026-04-02 00:50:30,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:30,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5d23eb053795d42100 HTTP/1.1" 200 None 2026-04-02 00:50:30,320 - lava_callback - DEBUG - Processing node 69cdbc5d23eb053795d42100: ltp-syscalls 2026-04-02 00:50:31,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:31,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:31,417 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cdbc5d23eb053795d42100/log.txt.gz 2026-04-02 00:50:31,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:32,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:32,111 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cdbc5d23eb053795d42100/lava-logs.yaml 2026-04-02 00:50:32,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:50:32,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:50:32,317 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cdbc5d23eb053795d42100/lava_callback.json.gz 2026-04-02 00:50:32,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:32,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5d23eb053795d42100 HTTP/1.1" 200 None 2026-04-02 00:50:32,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:32,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:50:32,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:32,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5d23eb053795d42100 HTTP/1.1" 200 None 2026-04-02 00:50:32,430 - lava_callback - INFO - Completed processing callback for node 69cdbc5d23eb053795d42100 INFO: 172.20.0.1:40372 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:50:37,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:50:37,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:54906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46734 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:51:23,267 - lava_callback - INFO - Received callback for node 69cdbc3323eb053795d42072 INFO: 172.20.0.1:46740 - "POST /node/69cdbc3323eb053795d42072 HTTP/1.0" 202 Accepted 2026-04-02 00:51:23,273 - lava_callback - INFO - Starting async job submit for node 69cdbc3323eb053795d42072 2026-04-02 00:51:23,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:23,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3323eb053795d42072 HTTP/1.1" 200 None 2026-04-02 00:51:23,309 - lava_callback - DEBUG - Processing node 69cdbc3323eb053795d42072: baseline-arm-clabbe 2026-04-02 00:51:23,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:51:23,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:51:23,649 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cdbc3323eb053795d42072/log.txt.gz 2026-04-02 00:51:23,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:51:23,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:51:23,942 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cdbc3323eb053795d42072/lava-logs.yaml 2026-04-02 00:51:23,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:51:24,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:51:24,079 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cdbc3323eb053795d42072/lava_callback.json.gz 2026-04-02 00:51:24,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:24,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3323eb053795d42072 HTTP/1.1" 200 None 2026-04-02 00:51:24,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:24,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:51:24,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:24,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc3323eb053795d42072 HTTP/1.1" 200 None 2026-04-02 00:51:24,178 - lava_callback - INFO - Completed processing callback for node 69cdbc3323eb053795d42072 INFO: 172.20.0.1:46754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51952 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:51:37,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:37,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47736 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:51:56,898 - lava_callback - INFO - Received callback for node 69cb1a9e8c06c2799bf551bd INFO: 172.20.0.1:47738 - "POST /node/69cb1a9e8c06c2799bf551bd HTTP/1.0" 202 Accepted 2026-04-02 00:51:56,904 - lava_callback - INFO - Starting async job submit for node 69cb1a9e8c06c2799bf551bd 2026-04-02 00:51:56,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:56,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1a9e8c06c2799bf551bd HTTP/1.1" 200 None 2026-04-02 00:51:56,920 - lava_callback - DEBUG - Processing node 69cb1a9e8c06c2799bf551bd: baseline-arm-baylibre 2026-04-02 00:51:56,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1a9e8c06c2799bf551bd 2026-04-02 00:51:56,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:56,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1a9e8c06c2799bf551bd HTTP/1.1" 200 None 2026-04-02 00:51:56,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:56,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-04-02 00:51:56,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:56,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1a9e8c06c2799bf551bd HTTP/1.1" 200 None 2026-04-02 00:51:56,978 - lava_callback - ERROR - Error processing callback for node 69cb1a9e8c06c2799bf551bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:51:57,495 - lava_callback - INFO - Received callback for node 69cdb9f723eb053795d41f2a INFO: 172.20.0.1:47740 - "POST /node/69cdb9f723eb053795d41f2a HTTP/1.0" 202 Accepted 2026-04-02 00:51:57,503 - lava_callback - INFO - Starting async job submit for node 69cdb9f723eb053795d41f2a 2026-04-02 00:51:57,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:57,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9f723eb053795d41f2a HTTP/1.1" 200 None 2026-04-02 00:51:57,561 - lava_callback - DEBUG - Processing node 69cdb9f723eb053795d41f2a: baseline-x86 2026-04-02 00:51:58,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:51:58,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:51:58,643 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f723eb053795d41f2a/log.txt.gz 2026-04-02 00:51:59,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:51:59,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:51:59,310 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f723eb053795d41f2a/lava-logs.yaml 2026-04-02 00:51:59,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:51:59,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:51:59,526 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cdb9f723eb053795d41f2a/lava_callback.json.gz DEBUG: Setup failed for 69cdb9f723eb053795d41f2a. Transitting job node result: incomplete -> fail 2026-04-02 00:51:59,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:59,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9f723eb053795d41f2a HTTP/1.1" 200 None 2026-04-02 00:51:59,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:59,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5e523eb053795d41ede HTTP/1.1" 200 None 2026-04-02 00:51:59,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:51:59,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdb9f723eb053795d41f2a HTTP/1.1" 200 None 2026-04-02 00:51:59,780 - lava_callback - INFO - Completed processing callback for node 69cdb9f723eb053795d41f2a INFO: 172.20.0.1:35184 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:52:07,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:07,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 00:52:10,458 - lava_callback - INFO - Received callback for node 69cdbc5123eb053795d420d1 INFO: 172.20.0.1:38830 - "POST /node/69cdbc5123eb053795d420d1 HTTP/1.0" 202 Accepted 2026-04-02 00:52:10,465 - lava_callback - INFO - Starting async job submit for node 69cdbc5123eb053795d420d1 2026-04-02 00:52:10,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:10,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5123eb053795d420d1 HTTP/1.1" 200 None 2026-04-02 00:52:10,538 - lava_callback - DEBUG - Processing node 69cdbc5123eb053795d420d1: ltp-cve 2026-04-02 00:52:10,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:52:11,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:52:11,067 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69cdbc5123eb053795d420d1/log.txt.gz 2026-04-02 00:52:11,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:52:11,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:52:11,478 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69cdbc5123eb053795d420d1/lava-logs.yaml 2026-04-02 00:52:11,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:52:11,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:52:11,648 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69cdbc5123eb053795d420d1/lava_callback.json.gz 2026-04-02 00:52:11,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:11,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5123eb053795d420d1 HTTP/1.1" 200 None 2026-04-02 00:52:11,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:11,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:52:11,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:11,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5123eb053795d420d1 HTTP/1.1" 200 None 2026-04-02 00:52:11,737 - lava_callback - INFO - Completed processing callback for node 69cdbc5123eb053795d420d1 INFO: 172.20.0.1:38846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36320 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:52:36,874 - lava_callback - INFO - Received callback for node 69cb1ac88c06c2799bf551d1 INFO: 172.20.0.1:36322 - "POST /node/69cb1ac88c06c2799bf551d1 HTTP/1.0" 202 Accepted 2026-04-02 00:52:36,880 - lava_callback - INFO - Starting async job submit for node 69cb1ac88c06c2799bf551d1 2026-04-02 00:52:36,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:36,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1ac88c06c2799bf551d1 HTTP/1.1" 200 None 2026-04-02 00:52:36,897 - lava_callback - DEBUG - Processing node 69cb1ac88c06c2799bf551d1: baseline-arm-baylibre 2026-04-02 00:52:36,897 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1ac88c06c2799bf551d1 2026-04-02 00:52:36,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:36,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1ac88c06c2799bf551d1 HTTP/1.1" 200 None 2026-04-02 00:52:36,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:36,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-04-02 00:52:36,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:36,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1ac88c06c2799bf551d1 HTTP/1.1" 200 None 2026-04-02 00:52:36,949 - lava_callback - ERROR - Error processing callback for node 69cb1ac88c06c2799bf551d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:52:37,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:37,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42650 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:52:50,646 - lava_callback - INFO - Received callback for node 69cdbc5323eb053795d420d9 INFO: 172.20.0.1:50088 - "POST /node/69cdbc5323eb053795d420d9 HTTP/1.0" 202 Accepted 2026-04-02 00:52:50,657 - lava_callback - INFO - Starting async job submit for node 69cdbc5323eb053795d420d9 2026-04-02 00:52:50,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:50,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5323eb053795d420d9 HTTP/1.1" 200 None 2026-04-02 00:52:50,736 - lava_callback - DEBUG - Processing node 69cdbc5323eb053795d420d9: ltp-fs_bind 2026-04-02 00:52:51,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:52:51,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:52:51,734 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cdbc5323eb053795d420d9/log.txt.gz 2026-04-02 00:52:52,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:52:52,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:52:52,733 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cdbc5323eb053795d420d9/lava-logs.yaml 2026-04-02 00:52:52,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:52:53,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:52:53,310 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cdbc5323eb053795d420d9/lava_callback.json.gz 2026-04-02 00:52:53,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:53,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5323eb053795d420d9 HTTP/1.1" 200 None 2026-04-02 00:52:53,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:53,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:52:53,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:52:53,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5323eb053795d420d9 HTTP/1.1" 200 None 2026-04-02 00:52:53,454 - lava_callback - INFO - Completed processing callback for node 69cdbc5323eb053795d420d9 INFO: 172.20.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50590 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:53:07,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:53:07,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 00:53:10,727 - lava_callback - INFO - Received callback for node 69cb1af18c06c2799bf551dc INFO: 172.20.0.1:36694 - "POST /node/69cb1af18c06c2799bf551dc HTTP/1.0" 202 Accepted 2026-04-02 00:53:10,735 - lava_callback - INFO - Starting async job submit for node 69cb1af18c06c2799bf551dc 2026-04-02 00:53:10,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:53:10,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1af18c06c2799bf551dc HTTP/1.1" 200 None 2026-04-02 00:53:10,769 - lava_callback - DEBUG - Processing node 69cb1af18c06c2799bf551dc: baseline-arm-clabbe 2026-04-02 00:53:10,770 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1af18c06c2799bf551dc 2026-04-02 00:53:10,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:53:10,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1af18c06c2799bf551dc HTTP/1.1" 200 None 2026-04-02 00:53:10,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:53:10,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-04-02 00:53:10,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:53:10,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1af18c06c2799bf551dc HTTP/1.1" 200 None 2026-04-02 00:53:10,868 - lava_callback - ERROR - Error processing callback for node 69cb1af18c06c2799bf551dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41274 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:53:50,704 - lava_callback - INFO - Received callback for node 69cb1b1b8c06c2799bf551e7 INFO: 172.20.0.1:40926 - "POST /node/69cb1b1b8c06c2799bf551e7 HTTP/1.0" 202 Accepted 2026-04-02 00:53:50,713 - lava_callback - INFO - Starting async job submit for node 69cb1b1b8c06c2799bf551e7 2026-04-02 00:53:50,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:53:50,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1b1b8c06c2799bf551e7 HTTP/1.1" 200 None 2026-04-02 00:53:50,744 - lava_callback - DEBUG - Processing node 69cb1b1b8c06c2799bf551e7: baseline-arm-clabbe 2026-04-02 00:53:50,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1b1b8c06c2799bf551e7 2026-04-02 00:53:50,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:53:50,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1b1b8c06c2799bf551e7 HTTP/1.1" 200 None 2026-04-02 00:53:50,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:53:50,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-04-02 00:53:50,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:53:50,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1b1b8c06c2799bf551e7 HTTP/1.1" 200 None 2026-04-02 00:53:50,849 - lava_callback - ERROR - Error processing callback for node 69cb1b1b8c06c2799bf551e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55692 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:54:07,382 - lava_callback - INFO - Received callback for node 69cdbc5523eb053795d420dc INFO: 172.20.0.1:55708 - "POST /node/69cdbc5523eb053795d420dc HTTP/1.0" 202 Accepted 2026-04-02 00:54:07,393 - lava_callback - INFO - Starting async job submit for node 69cdbc5523eb053795d420dc 2026-04-02 00:54:07,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:07,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5523eb053795d420dc HTTP/1.1" 200 None 2026-04-02 00:54:07,475 - lava_callback - DEBUG - Processing node 69cdbc5523eb053795d420dc: ltp-fsx 2026-04-02 00:54:08,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:54:08,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:08,504 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69cdbc5523eb053795d420dc/log.txt.gz 2026-04-02 00:54:08,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:54:09,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:09,617 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69cdbc5523eb053795d420dc/lava-logs.yaml 2026-04-02 00:54:09,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:54:10,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:10,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69cdbc5523eb053795d420dc/lava_callback.json.gz 2026-04-02 00:54:10,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:10,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5523eb053795d420dc HTTP/1.1" 200 None 2026-04-02 00:54:10,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:10,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:54:10,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:10,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5523eb053795d420dc HTTP/1.1" 200 None 2026-04-02 00:54:10,767 - lava_callback - INFO - Completed processing callback for node 69cdbc5523eb053795d420dc INFO: 172.20.0.1:49558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53626 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:54:26,108 - lava_callback - INFO - Received callback for node 69cdbe2b23eb053795d4223b INFO: 172.20.0.1:53636 - "POST /node/69cdbe2b23eb053795d4223b HTTP/1.0" 202 Accepted 2026-04-02 00:54:26,118 - lava_callback - INFO - Starting async job submit for node 69cdbe2b23eb053795d4223b 2026-04-02 00:54:26,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:26,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbe2b23eb053795d4223b HTTP/1.1" 200 None 2026-04-02 00:54:26,156 - lava_callback - DEBUG - Processing node 69cdbe2b23eb053795d4223b: baseline-arm 2026-04-02 00:54:26,165 - lava_callback - WARNING - Failed to upload log 2026-04-02 00:54:26,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:53638 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:54:26,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:26,962 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbe2b23eb053795d4223b/lava-logs.yaml 2026-04-02 00:54:26,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:54:27,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:27,383 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbe2b23eb053795d4223b/lava_callback.json.gz 2026-04-02 00:54:27,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:27,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbe2b23eb053795d4223b HTTP/1.1" 200 None 2026-04-02 00:54:27,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:27,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:54:27,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:27,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbe2b23eb053795d4223b HTTP/1.1" 200 None 2026-04-02 00:54:27,483 - lava_callback - INFO - Completed processing callback for node 69cdbe2b23eb053795d4223b 2026-04-02 00:54:30,493 - lava_callback - INFO - Received callback for node 69cdbc5623eb053795d420df INFO: 172.20.0.1:56864 - "POST /node/69cdbc5623eb053795d420df HTTP/1.0" 202 Accepted 2026-04-02 00:54:30,503 - lava_callback - INFO - Starting async job submit for node 69cdbc5623eb053795d420df 2026-04-02 00:54:30,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:30,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5623eb053795d420df HTTP/1.1" 200 None 2026-04-02 00:54:30,603 - lava_callback - DEBUG - Processing node 69cdbc5623eb053795d420df: ltp-input 2026-04-02 00:54:30,782 - lava_callback - INFO - Received callback for node 69cb1b448c06c2799bf551f5 INFO: 172.20.0.1:56878 - "POST /node/69cb1b448c06c2799bf551f5 HTTP/1.0" 202 Accepted 2026-04-02 00:54:30,798 - lava_callback - INFO - Starting async job submit for node 69cb1b448c06c2799bf551f5 2026-04-02 00:54:30,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:30,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1b448c06c2799bf551f5 HTTP/1.1" 200 None 2026-04-02 00:54:30,866 - lava_callback - DEBUG - Processing node 69cb1b448c06c2799bf551f5: baseline-arm-clabbe 2026-04-02 00:54:30,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1b448c06c2799bf551f5 2026-04-02 00:54:30,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:30,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1b448c06c2799bf551f5 HTTP/1.1" 200 None 2026-04-02 00:54:30,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:30,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-04-02 00:54:30,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:31,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1b448c06c2799bf551f5 HTTP/1.1" 200 None 2026-04-02 00:54:31,101 - lava_callback - ERROR - Error processing callback for node 69cb1b448c06c2799bf551f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:54:31,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:54:31,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:31,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69cdbc5623eb053795d420df/log.txt.gz 2026-04-02 00:54:32,136 - lava_callback - INFO - Received callback for node 69cdbc5823eb053795d420e2 INFO: 172.20.0.1:56890 - "POST /node/69cdbc5823eb053795d420e2 HTTP/1.0" 202 Accepted 2026-04-02 00:54:32,147 - lava_callback - INFO - Starting async job submit for node 69cdbc5823eb053795d420e2 2026-04-02 00:54:32,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:32,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5823eb053795d420e2 HTTP/1.1" 200 None 2026-04-02 00:54:32,316 - lava_callback - DEBUG - Processing node 69cdbc5823eb053795d420e2: ltp-ipc 2026-04-02 00:54:32,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:54:33,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:54:33,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:33,436 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69cdbc5823eb053795d420e2/log.txt.gz 2026-04-02 00:54:33,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:33,539 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69cdbc5623eb053795d420df/lava-logs.yaml 2026-04-02 00:54:33,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:54:33,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:54:34,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:34,466 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69cdbc5623eb053795d420df/lava_callback.json.gz 2026-04-02 00:54:34,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:34,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5623eb053795d420df HTTP/1.1" 200 None 2026-04-02 00:54:34,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:34,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:54:34,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:34,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5623eb053795d420df HTTP/1.1" 200 None 2026-04-02 00:54:34,624 - lava_callback - INFO - Completed processing callback for node 69cdbc5623eb053795d420df 2026-04-02 00:54:34,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:34,855 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69cdbc5823eb053795d420e2/lava-logs.yaml 2026-04-02 00:54:34,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:54:35,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:54:35,686 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69cdbc5823eb053795d420e2/lava_callback.json.gz 2026-04-02 00:54:35,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:35,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5823eb053795d420e2 HTTP/1.1" 200 None 2026-04-02 00:54:35,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:35,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:54:35,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:35,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5823eb053795d420e2 HTTP/1.1" 200 None 2026-04-02 00:54:35,839 - lava_callback - INFO - Completed processing callback for node 69cdbc5823eb053795d420e2 INFO: 172.20.0.1:56898 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:54:37,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:54:37,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:53340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55802 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:55:30,870 - lava_callback - INFO - Received callback for node 69cb1b6e8c06c2799bf551fd INFO: 172.20.0.1:38092 - "POST /node/69cb1b6e8c06c2799bf551fd HTTP/1.0" 202 Accepted 2026-04-02 00:55:30,887 - lava_callback - INFO - Starting async job submit for node 69cb1b6e8c06c2799bf551fd 2026-04-02 00:55:30,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:55:30,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1b6e8c06c2799bf551fd HTTP/1.1" 200 None 2026-04-02 00:55:30,929 - lava_callback - DEBUG - Processing node 69cb1b6e8c06c2799bf551fd: baseline-arm-clabbe 2026-04-02 00:55:30,929 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1b6e8c06c2799bf551fd 2026-04-02 00:55:30,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:55:30,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1b6e8c06c2799bf551fd HTTP/1.1" 200 None 2026-04-02 00:55:30,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:55:30,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-04-02 00:55:30,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:55:31,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1b6e8c06c2799bf551fd HTTP/1.1" 200 None 2026-04-02 00:55:31,017 - lava_callback - ERROR - Error processing callback for node 69cb1b6e8c06c2799bf551fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41838 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:56:10,816 - lava_callback - INFO - Received callback for node 69cb1b978c06c2799bf55206 INFO: 172.20.0.1:48722 - "POST /node/69cb1b978c06c2799bf55206 HTTP/1.0" 202 Accepted 2026-04-02 00:56:10,825 - lava_callback - INFO - Starting async job submit for node 69cb1b978c06c2799bf55206 2026-04-02 00:56:10,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:10,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1b978c06c2799bf55206 HTTP/1.1" 200 None 2026-04-02 00:56:10,851 - lava_callback - DEBUG - Processing node 69cb1b978c06c2799bf55206: baseline-arm-clabbe 2026-04-02 00:56:10,851 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1b978c06c2799bf55206 2026-04-02 00:56:10,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:10,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1b978c06c2799bf55206 HTTP/1.1" 200 None 2026-04-02 00:56:10,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:10,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-04-02 00:56:10,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:10,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1b978c06c2799bf55206 HTTP/1.1" 200 None 2026-04-02 00:56:10,927 - lava_callback - ERROR - Error processing callback for node 69cb1b978c06c2799bf55206: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60044 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:56:37,103 - lava_callback - INFO - Received callback for node 69cb1bbd8c06c2799bf5522b INFO: 172.20.0.1:60056 - "POST /node/69cb1bbd8c06c2799bf5522b HTTP/1.0" 202 Accepted 2026-04-02 00:56:37,110 - lava_callback - INFO - Starting async job submit for node 69cb1bbd8c06c2799bf5522b 2026-04-02 00:56:37,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:37,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bbd8c06c2799bf5522b HTTP/1.1" 200 None 2026-04-02 00:56:37,129 - lava_callback - DEBUG - Processing node 69cb1bbd8c06c2799bf5522b: baseline-arm-baylibre 2026-04-02 00:56:37,129 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bbd8c06c2799bf5522b 2026-04-02 00:56:37,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:37,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bbd8c06c2799bf5522b HTTP/1.1" 200 None 2026-04-02 00:56:37,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:37,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:56:37,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:37,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bbd8c06c2799bf5522b HTTP/1.1" 200 None 2026-04-02 00:56:37,212 - lava_callback - ERROR - Error processing callback for node 69cb1bbd8c06c2799bf5522b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:56:37,786 - lava_callback - INFO - Received callback for node 69cb1bbf8c06c2799bf5522e INFO: 172.20.0.1:60058 - "POST /node/69cb1bbf8c06c2799bf5522e HTTP/1.0" 202 Accepted 2026-04-02 00:56:37,795 - lava_callback - INFO - Starting async job submit for node 69cb1bbf8c06c2799bf5522e 2026-04-02 00:56:37,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:37,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bbf8c06c2799bf5522e HTTP/1.1" 200 None 2026-04-02 00:56:37,818 - lava_callback - DEBUG - Processing node 69cb1bbf8c06c2799bf5522e: baseline-arm-baylibre 2026-04-02 00:56:37,819 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bbf8c06c2799bf5522e 2026-04-02 00:56:37,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:37,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bbf8c06c2799bf5522e HTTP/1.1" 200 None 2026-04-02 00:56:37,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:37,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:56:37,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:37,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bbf8c06c2799bf5522e HTTP/1.1" 200 None 2026-04-02 00:56:37,906 - lava_callback - ERROR - Error processing callback for node 69cb1bbf8c06c2799bf5522e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:56:38,472 - lava_callback - INFO - Received callback for node 69cb1bc08c06c2799bf55231 INFO: 172.20.0.1:60070 - "POST /node/69cb1bc08c06c2799bf55231 HTTP/1.0" 202 Accepted 2026-04-02 00:56:38,481 - lava_callback - INFO - Starting async job submit for node 69cb1bc08c06c2799bf55231 2026-04-02 00:56:38,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:38,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bc08c06c2799bf55231 HTTP/1.1" 200 None 2026-04-02 00:56:38,519 - lava_callback - DEBUG - Processing node 69cb1bc08c06c2799bf55231: baseline-arm-baylibre 2026-04-02 00:56:38,519 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bc08c06c2799bf55231 2026-04-02 00:56:38,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:38,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bc08c06c2799bf55231 HTTP/1.1" 200 None 2026-04-02 00:56:38,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:38,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:56:38,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:38,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bc08c06c2799bf55231 HTTP/1.1" 200 None 2026-04-02 00:56:38,606 - lava_callback - ERROR - Error processing callback for node 69cb1bc08c06c2799bf55231: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:56:39,153 - lava_callback - INFO - Received callback for node 69cb1bc18c06c2799bf55234 INFO: 172.20.0.1:60074 - "POST /node/69cb1bc18c06c2799bf55234 HTTP/1.0" 202 Accepted 2026-04-02 00:56:39,164 - lava_callback - INFO - Starting async job submit for node 69cb1bc18c06c2799bf55234 2026-04-02 00:56:39,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:39,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bc18c06c2799bf55234 HTTP/1.1" 200 None 2026-04-02 00:56:39,193 - lava_callback - DEBUG - Processing node 69cb1bc18c06c2799bf55234: baseline-arm-baylibre 2026-04-02 00:56:39,193 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bc18c06c2799bf55234 2026-04-02 00:56:39,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:39,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bc18c06c2799bf55234 HTTP/1.1" 200 None 2026-04-02 00:56:39,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:39,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:56:39,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:39,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bc18c06c2799bf55234 HTTP/1.1" 200 None 2026-04-02 00:56:39,255 - lava_callback - ERROR - Error processing callback for node 69cb1bc18c06c2799bf55234: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42462 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:56:50,964 - lava_callback - INFO - Received callback for node 69cb1bce8c06c2799bf5525b INFO: 172.20.0.1:58022 - "POST /node/69cb1bce8c06c2799bf5525b HTTP/1.0" 202 Accepted 2026-04-02 00:56:50,973 - lava_callback - INFO - Starting async job submit for node 69cb1bce8c06c2799bf5525b 2026-04-02 00:56:50,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:51,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bce8c06c2799bf5525b HTTP/1.1" 200 None 2026-04-02 00:56:51,002 - lava_callback - DEBUG - Processing node 69cb1bce8c06c2799bf5525b: baseline-arm-clabbe 2026-04-02 00:56:51,002 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bce8c06c2799bf5525b 2026-04-02 00:56:51,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:51,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bce8c06c2799bf5525b HTTP/1.1" 200 None 2026-04-02 00:56:51,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:51,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:56:51,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:51,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bce8c06c2799bf5525b HTTP/1.1" 200 None 2026-04-02 00:56:51,087 - lava_callback - ERROR - Error processing callback for node 69cb1bce8c06c2799bf5525b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:56:57,099 - lava_callback - INFO - Received callback for node 69cb1bc48c06c2799bf5523f INFO: 172.20.0.1:58040 - "POST /node/69cb1bc48c06c2799bf5523f HTTP/1.0" 202 Accepted 2026-04-02 00:56:57,105 - lava_callback - INFO - Starting async job submit for node 69cb1bc48c06c2799bf5523f 2026-04-02 00:56:57,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:57,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bc48c06c2799bf5523f HTTP/1.1" 200 None 2026-04-02 00:56:57,121 - lava_callback - DEBUG - Processing node 69cb1bc48c06c2799bf5523f: baseline-arm-baylibre 2026-04-02 00:56:57,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bc48c06c2799bf5523f 2026-04-02 00:56:57,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:57,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bc48c06c2799bf5523f HTTP/1.1" 200 None 2026-04-02 00:56:57,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:57,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:56:57,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:57,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bc48c06c2799bf5523f HTTP/1.1" 200 None 2026-04-02 00:56:57,172 - lava_callback - ERROR - Error processing callback for node 69cb1bc48c06c2799bf5523f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:56:57,868 - lava_callback - INFO - Received callback for node 69cb1bc68c06c2799bf55242 INFO: 172.20.0.1:58046 - "POST /node/69cb1bc68c06c2799bf55242 HTTP/1.0" 202 Accepted 2026-04-02 00:56:57,877 - lava_callback - INFO - Starting async job submit for node 69cb1bc68c06c2799bf55242 2026-04-02 00:56:57,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:57,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bc68c06c2799bf55242 HTTP/1.1" 200 None 2026-04-02 00:56:57,902 - lava_callback - DEBUG - Processing node 69cb1bc68c06c2799bf55242: baseline-arm-baylibre 2026-04-02 00:56:57,903 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bc68c06c2799bf55242 2026-04-02 00:56:57,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:57,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bc68c06c2799bf55242 HTTP/1.1" 200 None 2026-04-02 00:56:57,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:57,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:56:57,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:57,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bc68c06c2799bf55242 HTTP/1.1" 200 None 2026-04-02 00:56:57,981 - lava_callback - ERROR - Error processing callback for node 69cb1bc68c06c2799bf55242: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:56:59,487 - lava_callback - INFO - Received callback for node 69cdbe5b23eb053795d42261 INFO: 172.20.0.1:58048 - "POST /node/69cdbe5b23eb053795d42261 HTTP/1.0" 202 Accepted 2026-04-02 00:56:59,493 - lava_callback - INFO - Starting async job submit for node 69cdbe5b23eb053795d42261 2026-04-02 00:56:59,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:56:59,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbe5b23eb053795d42261 HTTP/1.1" 200 None 2026-04-02 00:56:59,548 - lava_callback - DEBUG - Processing node 69cdbe5b23eb053795d42261: baseline-arm-pengutronix 2026-04-02 00:56:59,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:57:00,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:57:00,025 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbe5b23eb053795d42261/log.txt.gz 2026-04-02 00:57:00,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:57:00,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:57:00,413 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbe5b23eb053795d42261/lava-logs.yaml 2026-04-02 00:57:00,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:57:00,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:57:00,628 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbe5b23eb053795d42261/lava_callback.json.gz 2026-04-02 00:57:00,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:00,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbe5b23eb053795d42261 HTTP/1.1" 200 None 2026-04-02 00:57:00,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:00,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:57:00,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:00,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbe5b23eb053795d42261 HTTP/1.1" 200 None 2026-04-02 00:57:00,819 - lava_callback - INFO - Completed processing callback for node 69cdbe5b23eb053795d42261 INFO: 172.20.0.1:47580 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:57:07,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:07,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 00:57:10,919 - lava_callback - INFO - Received callback for node 69cb1bd28c06c2799bf5526a INFO: 172.20.0.1:60290 - "POST /node/69cb1bd28c06c2799bf5526a HTTP/1.0" 202 Accepted 2026-04-02 00:57:10,925 - lava_callback - INFO - Starting async job submit for node 69cb1bd28c06c2799bf5526a 2026-04-02 00:57:10,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:10,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd28c06c2799bf5526a HTTP/1.1" 200 None 2026-04-02 00:57:10,943 - lava_callback - DEBUG - Processing node 69cb1bd28c06c2799bf5526a: baseline-arm-clabbe 2026-04-02 00:57:10,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bd28c06c2799bf5526a 2026-04-02 00:57:10,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:10,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd28c06c2799bf5526a HTTP/1.1" 200 None 2026-04-02 00:57:10,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:10,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:57:10,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:11,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bd28c06c2799bf5526a HTTP/1.1" 200 None 2026-04-02 00:57:11,001 - lava_callback - ERROR - Error processing callback for node 69cb1bd28c06c2799bf5526a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:57:11,545 - lava_callback - INFO - Received callback for node 69cb1bd38c06c2799bf5526d INFO: 172.20.0.1:60302 - "POST /node/69cb1bd38c06c2799bf5526d HTTP/1.0" 202 Accepted 2026-04-02 00:57:11,551 - lava_callback - INFO - Starting async job submit for node 69cb1bd38c06c2799bf5526d 2026-04-02 00:57:11,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:11,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd38c06c2799bf5526d HTTP/1.1" 200 None 2026-04-02 00:57:11,571 - lava_callback - DEBUG - Processing node 69cb1bd38c06c2799bf5526d: baseline-arm-clabbe 2026-04-02 00:57:11,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bd38c06c2799bf5526d 2026-04-02 00:57:11,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:11,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd38c06c2799bf5526d HTTP/1.1" 200 None 2026-04-02 00:57:11,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:11,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:57:11,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:11,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bd38c06c2799bf5526d HTTP/1.1" 200 None 2026-04-02 00:57:11,668 - lava_callback - ERROR - Error processing callback for node 69cb1bd38c06c2799bf5526d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:57:12,178 - lava_callback - INFO - Received callback for node 69cb1bd48c06c2799bf55270 INFO: 172.20.0.1:60306 - "POST /node/69cb1bd48c06c2799bf55270 HTTP/1.0" 202 Accepted 2026-04-02 00:57:12,187 - lava_callback - INFO - Starting async job submit for node 69cb1bd48c06c2799bf55270 2026-04-02 00:57:12,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:12,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd48c06c2799bf55270 HTTP/1.1" 200 None 2026-04-02 00:57:12,213 - lava_callback - DEBUG - Processing node 69cb1bd48c06c2799bf55270: baseline-arm-clabbe 2026-04-02 00:57:12,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bd48c06c2799bf55270 2026-04-02 00:57:12,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:12,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd48c06c2799bf55270 HTTP/1.1" 200 None 2026-04-02 00:57:12,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:12,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:57:12,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:12,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bd48c06c2799bf55270 HTTP/1.1" 200 None 2026-04-02 00:57:12,327 - lava_callback - ERROR - Error processing callback for node 69cb1bd48c06c2799bf55270: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 00:57:12,813 - lava_callback - INFO - Received callback for node 69cb1bd78c06c2799bf552a4 INFO: 172.20.0.1:60316 - "POST /node/69cb1bd78c06c2799bf552a4 HTTP/1.0" 202 Accepted 2026-04-02 00:57:12,822 - lava_callback - INFO - Starting async job submit for node 69cb1bd78c06c2799bf552a4 2026-04-02 00:57:12,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:12,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd78c06c2799bf552a4 HTTP/1.1" 200 None 2026-04-02 00:57:12,846 - lava_callback - DEBUG - Processing node 69cb1bd78c06c2799bf552a4: baseline-arm-clabbe 2026-04-02 00:57:12,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1bd78c06c2799bf552a4 2026-04-02 00:57:12,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:12,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1bd78c06c2799bf552a4 HTTP/1.1" 200 None 2026-04-02 00:57:12,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:12,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 00:57:12,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:12,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1bd78c06c2799bf552a4 HTTP/1.1" 200 None 2026-04-02 00:57:12,932 - lava_callback - ERROR - Error processing callback for node 69cb1bd78c06c2799bf552a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58682 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:57:30,669 - lava_callback - INFO - Received callback for node 69cdbc5a23eb053795d420fa INFO: 172.20.0.1:51156 - "POST /node/69cdbc5a23eb053795d420fa HTTP/1.0" 202 Accepted 2026-04-02 00:57:30,679 - lava_callback - INFO - Starting async job submit for node 69cdbc5a23eb053795d420fa 2026-04-02 00:57:30,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:30,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5a23eb053795d420fa HTTP/1.1" 200 None 2026-04-02 00:57:30,743 - lava_callback - DEBUG - Processing node 69cdbc5a23eb053795d420fa: ltp-sched 2026-04-02 00:57:31,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:57:31,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:57:31,438 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69cdbc5a23eb053795d420fa/log.txt.gz 2026-04-02 00:57:31,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:57:31,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:57:31,927 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69cdbc5a23eb053795d420fa/lava-logs.yaml 2026-04-02 00:57:31,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:57:32,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:57:32,167 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69cdbc5a23eb053795d420fa/lava_callback.json.gz 2026-04-02 00:57:32,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:32,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5a23eb053795d420fa HTTP/1.1" 200 None 2026-04-02 00:57:32,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:32,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:57:32,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:32,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5a23eb053795d420fa HTTP/1.1" 200 None 2026-04-02 00:57:32,281 - lava_callback - INFO - Completed processing callback for node 69cdbc5a23eb053795d420fa INFO: 172.20.0.1:51164 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:57:37,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:37,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 00:57:38,330 - lava_callback - INFO - Received callback for node 69cdbe7f23eb053795d42269 INFO: 172.20.0.1:51168 - "POST /node/69cdbe7f23eb053795d42269 HTTP/1.0" 202 Accepted 2026-04-02 00:57:38,338 - lava_callback - INFO - Starting async job submit for node 69cdbe7f23eb053795d42269 2026-04-02 00:57:38,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:38,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbe7f23eb053795d42269 HTTP/1.1" 200 None 2026-04-02 00:57:38,381 - lava_callback - DEBUG - Processing node 69cdbe7f23eb053795d42269: baseline-arm-pengutronix 2026-04-02 00:57:38,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:57:38,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:57:38,705 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbe7f23eb053795d42269/log.txt.gz 2026-04-02 00:57:38,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:57:39,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:57:39,059 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbe7f23eb053795d42269/lava-logs.yaml 2026-04-02 00:57:39,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:57:39,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:57:39,306 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbe7f23eb053795d42269/lava_callback.json.gz 2026-04-02 00:57:39,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:39,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbe7f23eb053795d42269 HTTP/1.1" 200 None 2026-04-02 00:57:39,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:39,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:57:39,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:57:39,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbe7f23eb053795d42269 HTTP/1.1" 200 None 2026-04-02 00:57:39,518 - lava_callback - INFO - Completed processing callback for node 69cdbe7f23eb053795d42269 INFO: 172.20.0.1:38370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39254 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:58:07,495 - lava_callback - INFO - Received callback for node 69cdbc5c23eb053795d420fd INFO: 172.20.0.1:39266 - "POST /node/69cdbc5c23eb053795d420fd HTTP/1.0" 202 Accepted 2026-04-02 00:58:07,501 - lava_callback - INFO - Starting async job submit for node 69cdbc5c23eb053795d420fd 2026-04-02 00:58:07,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:07,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5c23eb053795d420fd HTTP/1.1" 200 None 2026-04-02 00:58:07,543 - lava_callback - DEBUG - Processing node 69cdbc5c23eb053795d420fd: ltp-smoke 2026-04-02 00:58:07,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:07,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 00:58:07,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:58:08,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:58:08,142 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdbc5c23eb053795d420fd/log.txt.gz 2026-04-02 00:58:08,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:58:08,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:58:08,657 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdbc5c23eb053795d420fd/lava-logs.yaml 2026-04-02 00:58:08,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:58:08,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:58:08,835 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdbc5c23eb053795d420fd/lava_callback.json.gz 2026-04-02 00:58:08,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:08,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5c23eb053795d420fd HTTP/1.1" 200 None 2026-04-02 00:58:08,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:08,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:58:08,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:08,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5c23eb053795d420fd HTTP/1.1" 200 None 2026-04-02 00:58:08,944 - lava_callback - INFO - Completed processing callback for node 69cdbc5c23eb053795d420fd INFO: 172.20.0.1:54748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38728 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:58:37,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:38,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:58:50,720 - lava_callback - INFO - Received callback for node 69cdbc5f23eb053795d42104 INFO: 172.20.0.1:50452 - "POST /node/69cdbc5f23eb053795d42104 HTTP/1.0" 202 Accepted 2026-04-02 00:58:50,728 - lava_callback - INFO - Starting async job submit for node 69cdbc5f23eb053795d42104 2026-04-02 00:58:50,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:50,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5f23eb053795d42104 HTTP/1.1" 200 None 2026-04-02 00:58:50,777 - lava_callback - DEBUG - Processing node 69cdbc5f23eb053795d42104: ltp-syscalls-ipc 2026-04-02 00:58:51,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:58:51,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:58:51,433 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cdbc5f23eb053795d42104/log.txt.gz 2026-04-02 00:58:51,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:58:52,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:58:52,062 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cdbc5f23eb053795d42104/lava-logs.yaml 2026-04-02 00:58:52,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:58:52,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:58:52,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cdbc5f23eb053795d42104/lava_callback.json.gz 2026-04-02 00:58:52,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:52,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5f23eb053795d42104 HTTP/1.1" 200 None 2026-04-02 00:58:52,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:52,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:58:52,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:52,355 - lava_callback - INFO - Received callback for node 69cdbc6023eb053795d42107 INFO: 172.20.0.1:50462 - "POST /node/69cdbc6023eb053795d42107 HTTP/1.0" 202 Accepted 2026-04-02 00:58:52,364 - lava_callback - INFO - Starting async job submit for node 69cdbc6023eb053795d42107 2026-04-02 00:58:52,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:52,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5f23eb053795d42104 HTTP/1.1" 200 None 2026-04-02 00:58:52,423 - lava_callback - INFO - Completed processing callback for node 69cdbc5f23eb053795d42104 2026-04-02 00:58:52,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc6023eb053795d42107 HTTP/1.1" 200 None 2026-04-02 00:58:52,447 - lava_callback - DEBUG - Processing node 69cdbc6023eb053795d42107: ltp-timers 2026-04-02 00:58:53,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:58:53,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:58:53,233 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69cdbc6023eb053795d42107/log.txt.gz 2026-04-02 00:58:53,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:58:53,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:58:53,648 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69cdbc6023eb053795d42107/lava-logs.yaml 2026-04-02 00:58:53,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 00:58:53,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 00:58:53,822 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69cdbc6023eb053795d42107/lava_callback.json.gz 2026-04-02 00:58:53,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:53,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc6023eb053795d42107 HTTP/1.1" 200 None 2026-04-02 00:58:53,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:53,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 00:58:53,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:58:53,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc6023eb053795d42107 HTTP/1.1" 200 None 2026-04-02 00:58:53,968 - lava_callback - INFO - Completed processing callback for node 69cdbc6023eb053795d42107 INFO: 172.20.0.1:50478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56696 - "GET / HTTP/1.0" 200 OK 2026-04-02 00:59:08,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 00:59:08,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34152 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:02:00,710 - lava_callback - INFO - Received callback for node 69cdbc1823eb053795d4200d INFO: 172.20.0.1:59996 - "POST /node/69cdbc1823eb053795d4200d HTTP/1.0" 202 Accepted 2026-04-02 01:02:00,719 - lava_callback - INFO - Starting async job submit for node 69cdbc1823eb053795d4200d 2026-04-02 01:02:00,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:00,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1823eb053795d4200d HTTP/1.1" 200 None 2026-04-02 01:02:00,860 - lava_callback - DEBUG - Processing node 69cdbc1823eb053795d4200d: baseline-arm 2026-04-02 01:02:01,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:02:02,676 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:02:02,679 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1823eb053795d4200d/log.txt.gz 2026-04-02 01:02:03,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:02:04,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:02:04,185 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1823eb053795d4200d/lava-logs.yaml 2026-04-02 01:02:04,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:02:05,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:02:05,062 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdbc1823eb053795d4200d/lava_callback.json.gz DEBUG: Setup failed for 69cdbc1823eb053795d4200d. Transitting job node result: incomplete -> fail 2026-04-02 01:02:05,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:05,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1823eb053795d4200d HTTP/1.1" 200 None 2026-04-02 01:02:05,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:05,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:02:05,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:05,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc1823eb053795d4200d HTTP/1.1" 200 None 2026-04-02 01:02:05,478 - lava_callback - INFO - Completed processing callback for node 69cdbc1823eb053795d4200d INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:02:08,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:08,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 01:02:10,833 - lava_callback - INFO - Received callback for node 69cdbc6223eb053795d4210e INFO: 172.20.0.1:46422 - "POST /node/69cdbc6223eb053795d4210e HTTP/1.0" 202 Accepted 2026-04-02 01:02:10,844 - lava_callback - INFO - Starting async job submit for node 69cdbc6223eb053795d4210e 2026-04-02 01:02:10,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:10,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc6223eb053795d4210e HTTP/1.1" 200 None 2026-04-02 01:02:10,935 - lava_callback - DEBUG - Processing node 69cdbc6223eb053795d4210e: ltp-watchqueue 2026-04-02 01:02:11,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:02:12,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:02:12,388 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cdbc6223eb053795d4210e/log.txt.gz 2026-04-02 01:02:12,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:02:13,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:02:13,498 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cdbc6223eb053795d4210e/lava-logs.yaml 2026-04-02 01:02:13,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:02:14,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:02:14,334 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cdbc6223eb053795d4210e/lava_callback.json.gz 2026-04-02 01:02:14,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:14,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc6223eb053795d4210e HTTP/1.1" 200 None 2026-04-02 01:02:14,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:14,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:02:14,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:14,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc6223eb053795d4210e HTTP/1.1" 200 None 2026-04-02 01:02:14,481 - lava_callback - INFO - Completed processing callback for node 69cdbc6223eb053795d4210e INFO: 172.20.0.1:46432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36098 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:02:38,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:38,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48450 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:02:47,788 - lava_callback - INFO - Received callback for node 69cdc02523eb053795d42324 INFO: 172.20.0.1:48466 - "POST /node/69cdc02523eb053795d42324 HTTP/1.0" 202 Accepted 2026-04-02 01:02:47,799 - lava_callback - INFO - Starting async job submit for node 69cdc02523eb053795d42324 2026-04-02 01:02:47,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:47,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc02523eb053795d42324 HTTP/1.1" 200 None 2026-04-02 01:02:47,832 - lava_callback - DEBUG - Processing node 69cdc02523eb053795d42324: baseline-arm 2026-04-02 01:02:47,840 - lava_callback - WARNING - Failed to upload log 2026-04-02 01:02:47,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:02:48,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:02:48,694 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdc02523eb053795d42324/lava-logs.yaml 2026-04-02 01:02:48,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:02:49,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:02:49,470 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdc02523eb053795d42324/lava_callback.json.gz 2026-04-02 01:02:49,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:49,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc02523eb053795d42324 HTTP/1.1" 200 None 2026-04-02 01:02:49,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:49,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:02:49,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:02:49,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc02523eb053795d42324 HTTP/1.1" 200 None 2026-04-02 01:02:49,568 - lava_callback - INFO - Completed processing callback for node 69cdc02523eb053795d42324 INFO: 172.20.0.1:45750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59222 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:03:08,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:03:08,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52792 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:04:14,025 - lava_callback - INFO - Received callback for node 69cdc02223eb053795d42321 INFO: 172.20.0.1:48842 - "POST /node/69cdc02223eb053795d42321 HTTP/1.0" 202 Accepted 2026-04-02 01:04:14,031 - lava_callback - INFO - Starting async job submit for node 69cdc02223eb053795d42321 2026-04-02 01:04:14,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:04:14,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc02223eb053795d42321 HTTP/1.1" 200 None 2026-04-02 01:04:14,064 - lava_callback - DEBUG - Processing node 69cdc02223eb053795d42321: baseline-x86 2026-04-02 01:04:14,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:04:14,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:04:14,479 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cdc02223eb053795d42321/log.txt.gz 2026-04-02 01:04:14,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:04:14,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:04:14,849 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cdc02223eb053795d42321/lava-logs.yaml 2026-04-02 01:04:14,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:04:15,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:04:15,002 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cdc02223eb053795d42321/lava_callback.json.gz 2026-04-02 01:04:15,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:04:15,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc02223eb053795d42321 HTTP/1.1" 200 None 2026-04-02 01:04:15,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:04:15,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5e523eb053795d41ede HTTP/1.1" 200 None 2026-04-02 01:04:15,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:04:15,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc02223eb053795d42321 HTTP/1.1" 200 None 2026-04-02 01:04:15,090 - lava_callback - INFO - Completed processing callback for node 69cdc02223eb053795d42321 INFO: 172.20.0.1:48854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48008 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:04:38,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:04:38,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46954 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:05:29,128 - lava_callback - INFO - Received callback for node 69cdbc3a23eb053795d420ac INFO: 172.20.0.1:46968 - "POST /node/69cdbc3a23eb053795d420ac HTTP/1.0" 202 Accepted 2026-04-02 01:05:29,177 - lava_callback - INFO - Starting async job submit for node 69cdbc3a23eb053795d420ac 2026-04-02 01:05:29,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:05:29,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3a23eb053795d420ac HTTP/1.1" 200 None 2026-04-02 01:05:29,254 - lava_callback - DEBUG - Processing node 69cdbc3a23eb053795d420ac: baseline-arm-pengutronix 2026-04-02 01:05:29,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:05:29,572 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:05:29,574 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3a23eb053795d420ac/log.txt.gz 2026-04-02 01:05:29,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:05:30,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:05:30,206 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3a23eb053795d420ac/lava-logs.yaml 2026-04-02 01:05:30,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:05:30,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:05:30,351 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdbc3a23eb053795d420ac/lava_callback.json.gz DEBUG: Setup failed for 69cdbc3a23eb053795d420ac. Transitting job node result: incomplete -> fail 2026-04-02 01:05:30,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:05:30,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3a23eb053795d420ac HTTP/1.1" 200 None 2026-04-02 01:05:30,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:05:30,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:05:30,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:05:30,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc3a23eb053795d420ac HTTP/1.1" 200 None 2026-04-02 01:05:30,637 - lava_callback - INFO - Completed processing callback for node 69cdbc3a23eb053795d420ac INFO: 172.20.0.1:53914 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:05:38,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:05:38,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58730 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:06:57,661 - lava_callback - INFO - Received callback for node 69cb1e2a8c06c2799bf579d4 INFO: 172.20.0.1:58732 - "POST /node/69cb1e2a8c06c2799bf579d4 HTTP/1.0" 202 Accepted 2026-04-02 01:06:57,670 - lava_callback - INFO - Starting async job submit for node 69cb1e2a8c06c2799bf579d4 2026-04-02 01:06:57,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:06:57,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1e2a8c06c2799bf579d4 HTTP/1.1" 200 None 2026-04-02 01:06:57,709 - lava_callback - DEBUG - Processing node 69cb1e2a8c06c2799bf579d4: baseline-arm64-baylibre 2026-04-02 01:06:57,709 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1e2a8c06c2799bf579d4 2026-04-02 01:06:57,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:06:57,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1e2a8c06c2799bf579d4 HTTP/1.1" 200 None 2026-04-02 01:06:57,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:06:57,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 01:06:57,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:06:57,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1e2a8c06c2799bf579d4 HTTP/1.1" 200 None 2026-04-02 01:06:57,804 - lava_callback - ERROR - Error processing callback for node 69cb1e2a8c06c2799bf579d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41792 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:08:03,402 - lava_callback - INFO - Received callback for node 69cdc15b23eb053795d423c3 INFO: 172.20.0.1:41668 - "POST /node/69cdc15b23eb053795d423c3 HTTP/1.0" 202 Accepted 2026-04-02 01:08:03,408 - lava_callback - INFO - Starting async job submit for node 69cdc15b23eb053795d423c3 2026-04-02 01:08:03,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:03,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15b23eb053795d423c3 HTTP/1.1" 200 None 2026-04-02 01:08:03,441 - lava_callback - DEBUG - Processing node 69cdc15b23eb053795d423c3: baseline-arm64 2026-04-02 01:08:03,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:03,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:03,752 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15b23eb053795d423c3/log.txt.gz 2026-04-02 01:08:03,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:04,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:04,024 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15b23eb053795d423c3/lava-logs.yaml 2026-04-02 01:08:04,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:04,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:04,251 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15b23eb053795d423c3/lava_callback.json.gz 2026-04-02 01:08:04,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:04,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15b23eb053795d423c3 HTTP/1.1" 200 None 2026-04-02 01:08:04,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:04,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:08:04,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:04,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15b23eb053795d423c3 HTTP/1.1" 200 None 2026-04-02 01:08:04,347 - lava_callback - INFO - Completed processing callback for node 69cdc15b23eb053795d423c3 2026-04-02 01:08:04,486 - lava_callback - INFO - Received callback for node 69cdc14f23eb053795d423a2 INFO: 172.20.0.1:41680 - "POST /node/69cdc14f23eb053795d423a2 HTTP/1.0" 202 Accepted 2026-04-02 01:08:04,496 - lava_callback - INFO - Starting async job submit for node 69cdc14f23eb053795d423a2 2026-04-02 01:08:04,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:04,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc14f23eb053795d423a2 HTTP/1.1" 200 None 2026-04-02 01:08:04,549 - lava_callback - DEBUG - Processing node 69cdc14f23eb053795d423a2: baseline-arm64 2026-04-02 01:08:04,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:04,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:04,953 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc14f23eb053795d423a2/log.txt.gz 2026-04-02 01:08:05,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:05,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:05,323 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc14f23eb053795d423a2/lava-logs.yaml 2026-04-02 01:08:05,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:05,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:05,481 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc14f23eb053795d423a2/lava_callback.json.gz 2026-04-02 01:08:05,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:05,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc14f23eb053795d423a2 HTTP/1.1" 200 None 2026-04-02 01:08:05,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:05,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:08:05,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:05,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc14f23eb053795d423a2 HTTP/1.1" 200 None 2026-04-02 01:08:05,573 - lava_callback - INFO - Completed processing callback for node 69cdc14f23eb053795d423a2 INFO: 172.20.0.1:41696 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:08:08,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:08,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48364 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:08:24,121 - lava_callback - INFO - Received callback for node 69cdc16b23eb053795d42407 INFO: 172.20.0.1:48378 - "POST /node/69cdc16b23eb053795d42407 HTTP/1.0" 202 Accepted 2026-04-02 01:08:24,127 - lava_callback - INFO - Starting async job submit for node 69cdc16b23eb053795d42407 2026-04-02 01:08:24,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:24,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16b23eb053795d42407 HTTP/1.1" 200 None 2026-04-02 01:08:24,156 - lava_callback - DEBUG - Processing node 69cdc16b23eb053795d42407: baseline-nfs-arm64 2026-04-02 01:08:24,158 - lava_callback - WARNING - Failed to upload log 2026-04-02 01:08:24,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:24,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:24,318 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16b23eb053795d42407/lava-logs.yaml 2026-04-02 01:08:24,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:24,488 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:24,489 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16b23eb053795d42407/lava_callback.json.gz 2026-04-02 01:08:24,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:24,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16b23eb053795d42407 HTTP/1.1" 200 None 2026-04-02 01:08:24,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:24,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:08:24,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:24,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16b23eb053795d42407 HTTP/1.1" 200 None 2026-04-02 01:08:24,551 - lava_callback - INFO - Completed processing callback for node 69cdc16b23eb053795d42407 INFO: 172.20.0.1:48384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49028 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:08:38,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:38,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 01:08:41,946 - lava_callback - INFO - Received callback for node 69cdc15123eb053795d423a8 INFO: 172.20.0.1:59556 - "POST /node/69cdc15123eb053795d423a8 HTTP/1.0" 202 Accepted 2026-04-02 01:08:41,953 - lava_callback - INFO - Starting async job submit for node 69cdc15123eb053795d423a8 2026-04-02 01:08:41,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:41,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15123eb053795d423a8 HTTP/1.1" 200 None 2026-04-02 01:08:41,993 - lava_callback - DEBUG - Processing node 69cdc15123eb053795d423a8: baseline-arm64 2026-04-02 01:08:42,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:42,538 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:42,540 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15123eb053795d423a8/log.txt.gz 2026-04-02 01:08:42,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:43,006 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:43,008 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15123eb053795d423a8/lava-logs.yaml 2026-04-02 01:08:43,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:08:43,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:08:43,149 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15123eb053795d423a8/lava_callback.json.gz 2026-04-02 01:08:43,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:43,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15123eb053795d423a8 HTTP/1.1" 200 None 2026-04-02 01:08:43,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:43,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:08:43,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:08:43,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15123eb053795d423a8 HTTP/1.1" 200 None 2026-04-02 01:08:43,232 - lava_callback - INFO - Completed processing callback for node 69cdc15123eb053795d423a8 INFO: 172.20.0.1:59566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41058 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:09:08,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:08,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 01:09:08,499 - lava_callback - INFO - Received callback for node 69cdc15023eb053795d423a5 INFO: 172.20.0.1:41062 - "POST /node/69cdc15023eb053795d423a5 HTTP/1.0" 202 Accepted 2026-04-02 01:09:08,505 - lava_callback - INFO - Starting async job submit for node 69cdc15023eb053795d423a5 2026-04-02 01:09:08,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:08,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15023eb053795d423a5 HTTP/1.1" 200 None 2026-04-02 01:09:08,556 - lava_callback - DEBUG - Processing node 69cdc15023eb053795d423a5: baseline-arm64 2026-04-02 01:09:08,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:08,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:08,947 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15023eb053795d423a5/log.txt.gz 2026-04-02 01:09:09,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:09,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:09,252 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15023eb053795d423a5/lava-logs.yaml 2026-04-02 01:09:09,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:09,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:09,393 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15023eb053795d423a5/lava_callback.json.gz 2026-04-02 01:09:09,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:09,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15023eb053795d423a5 HTTP/1.1" 200 None 2026-04-02 01:09:09,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:09,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:09:09,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:09,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15023eb053795d423a5 HTTP/1.1" 200 None 2026-04-02 01:09:09,476 - lava_callback - INFO - Completed processing callback for node 69cdc15023eb053795d423a5 INFO: 172.20.0.1:58258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58260 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:09:20,917 - lava_callback - INFO - Received callback for node 69cdc17423eb053795d42419 INFO: 172.20.0.1:44050 - "POST /node/69cdc17423eb053795d42419 HTTP/1.0" 202 Accepted 2026-04-02 01:09:20,923 - lava_callback - INFO - Starting async job submit for node 69cdc17423eb053795d42419 2026-04-02 01:09:20,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:20,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17423eb053795d42419 HTTP/1.1" 200 None 2026-04-02 01:09:20,965 - lava_callback - DEBUG - Processing node 69cdc17423eb053795d42419: baseline-arm-pengutronix 2026-04-02 01:09:21,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:21,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:21,304 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc17423eb053795d42419/log.txt.gz 2026-04-02 01:09:21,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:21,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:21,618 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc17423eb053795d42419/lava-logs.yaml 2026-04-02 01:09:21,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:21,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:21,789 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc17423eb053795d42419/lava_callback.json.gz 2026-04-02 01:09:21,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:21,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17423eb053795d42419 HTTP/1.1" 200 None 2026-04-02 01:09:21,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:21,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:09:21,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:21,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc17423eb053795d42419 HTTP/1.1" 200 None 2026-04-02 01:09:21,878 - lava_callback - INFO - Completed processing callback for node 69cdc17423eb053795d42419 INFO: 172.20.0.1:44062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44068 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:09:33,261 - lava_callback - INFO - Received callback for node 69cdc19823eb053795d4249c INFO: 172.20.0.1:37322 - "POST /node/69cdc19823eb053795d4249c HTTP/1.0" 202 Accepted 2026-04-02 01:09:33,268 - lava_callback - INFO - Starting async job submit for node 69cdc19823eb053795d4249c 2026-04-02 01:09:33,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:33,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19823eb053795d4249c HTTP/1.1" 200 None 2026-04-02 01:09:33,320 - lava_callback - DEBUG - Processing node 69cdc19823eb053795d4249c: baseline-arm64-foundriesio 2026-04-02 01:09:33,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:33,775 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:33,777 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cdc19823eb053795d4249c/log.txt.gz 2026-04-02 01:09:33,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:34,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:34,160 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cdc19823eb053795d4249c/lava-logs.yaml 2026-04-02 01:09:34,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:34,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:34,338 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cdc19823eb053795d4249c/lava_callback.json.gz 2026-04-02 01:09:34,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:34,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19823eb053795d4249c HTTP/1.1" 200 None 2026-04-02 01:09:34,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:34,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:09:34,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:34,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc19823eb053795d4249c HTTP/1.1" 200 None 2026-04-02 01:09:34,445 - lava_callback - INFO - Completed processing callback for node 69cdc19823eb053795d4249c INFO: 172.20.0.1:37334 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:09:37,805 - lava_callback - INFO - Received callback for node 69cb1ec98c06c2799bf579fa INFO: 172.20.0.1:37340 - "POST /node/69cb1ec98c06c2799bf579fa HTTP/1.0" 202 Accepted 2026-04-02 01:09:37,814 - lava_callback - INFO - Starting async job submit for node 69cb1ec98c06c2799bf579fa 2026-04-02 01:09:37,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:37,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1ec98c06c2799bf579fa HTTP/1.1" 200 None 2026-04-02 01:09:37,835 - lava_callback - DEBUG - Processing node 69cb1ec98c06c2799bf579fa: baseline-arm64-baylibre 2026-04-02 01:09:37,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1ec98c06c2799bf579fa 2026-04-02 01:09:37,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:37,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1ec98c06c2799bf579fa HTTP/1.1" 200 None 2026-04-02 01:09:37,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:37,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 01:09:37,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:37,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1ec98c06c2799bf579fa HTTP/1.1" 200 None 2026-04-02 01:09:37,899 - lava_callback - ERROR - Error processing callback for node 69cb1ec98c06c2799bf579fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:09:38,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:38,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 01:09:40,143 - lava_callback - INFO - Received callback for node 69cdc19423eb053795d4248f INFO: 172.20.0.1:47792 - "POST /node/69cdc19423eb053795d4248f HTTP/1.0" 202 Accepted 2026-04-02 01:09:40,151 - lava_callback - INFO - Starting async job submit for node 69cdc19423eb053795d4248f 2026-04-02 01:09:40,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:40,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19423eb053795d4248f HTTP/1.1" 200 None 2026-04-02 01:09:40,199 - lava_callback - DEBUG - Processing node 69cdc19423eb053795d4248f: baseline-arm64-pengutronix 2026-04-02 01:09:40,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:40,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:40,549 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cdc19423eb053795d4248f/log.txt.gz 2026-04-02 01:09:40,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:40,860 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:40,862 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cdc19423eb053795d4248f/lava-logs.yaml 2026-04-02 01:09:40,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:41,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:41,030 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cdc19423eb053795d4248f/lava_callback.json.gz 2026-04-02 01:09:41,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:41,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19423eb053795d4248f HTTP/1.1" 200 None 2026-04-02 01:09:41,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:41,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:09:41,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:41,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc19423eb053795d4248f HTTP/1.1" 200 None 2026-04-02 01:09:41,111 - lava_callback - INFO - Completed processing callback for node 69cdc19423eb053795d4248f INFO: 172.20.0.1:47804 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:09:47,780 - lava_callback - INFO - Received callback for node 69cdc17f23eb053795d42440 INFO: 172.20.0.1:47808 - "POST /node/69cdc17f23eb053795d42440 HTTP/1.0" 202 Accepted 2026-04-02 01:09:47,787 - lava_callback - INFO - Starting async job submit for node 69cdc17f23eb053795d42440 2026-04-02 01:09:47,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:47,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17f23eb053795d42440 HTTP/1.1" 200 None 2026-04-02 01:09:47,829 - lava_callback - DEBUG - Processing node 69cdc17f23eb053795d42440: baseline-arm64-broonie 2026-04-02 01:09:48,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:48,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:48,235 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc17f23eb053795d42440/log.txt.gz 2026-04-02 01:09:48,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:48,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:48,597 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc17f23eb053795d42440/lava-logs.yaml 2026-04-02 01:09:48,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:48,742 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:48,744 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc17f23eb053795d42440/lava_callback.json.gz 2026-04-02 01:09:48,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:48,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17f23eb053795d42440 HTTP/1.1" 200 None 2026-04-02 01:09:48,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:48,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:09:48,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:48,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc17f23eb053795d42440 HTTP/1.1" 200 None 2026-04-02 01:09:48,830 - lava_callback - INFO - Completed processing callback for node 69cdc17f23eb053795d42440 2026-04-02 01:09:49,274 - lava_callback - INFO - Received callback for node 69cdc18623eb053795d4246c INFO: 172.20.0.1:47814 - "POST /node/69cdc18623eb053795d4246c HTTP/1.0" 202 Accepted 2026-04-02 01:09:49,281 - lava_callback - INFO - Starting async job submit for node 69cdc18623eb053795d4246c 2026-04-02 01:09:49,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:49,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18623eb053795d4246c HTTP/1.1" 200 None 2026-04-02 01:09:49,331 - lava_callback - DEBUG - Processing node 69cdc18623eb053795d4246c: baseline-arm64-broonie 2026-04-02 01:09:49,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:49,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:49,756 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18623eb053795d4246c/log.txt.gz 2026-04-02 01:09:49,894 - lava_callback - INFO - Received callback for node 69cdc19523eb053795d42495 INFO: 172.20.0.1:51260 - "POST /node/69cdc19523eb053795d42495 HTTP/1.0" 202 Accepted 2026-04-02 01:09:49,926 - lava_callback - INFO - Starting async job submit for node 69cdc19523eb053795d42495 2026-04-02 01:09:49,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:50,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19523eb053795d42495 HTTP/1.1" 200 None 2026-04-02 01:09:50,059 - lava_callback - DEBUG - Processing node 69cdc19523eb053795d42495: baseline-arm64-pengutronix 2026-04-02 01:09:50,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:50,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:50,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:50,474 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18623eb053795d4246c/lava-logs.yaml 2026-04-02 01:09:50,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:50,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:50,590 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cdc19523eb053795d42495/log.txt.gz 2026-04-02 01:09:50,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:50,690 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18623eb053795d4246c/lava_callback.json.gz 2026-04-02 01:09:50,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:50,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:50,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18623eb053795d4246c HTTP/1.1" 200 None 2026-04-02 01:09:50,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:50,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:09:50,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:50,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18623eb053795d4246c HTTP/1.1" 200 None 2026-04-02 01:09:50,865 - lava_callback - INFO - Completed processing callback for node 69cdc18623eb053795d4246c 2026-04-02 01:09:51,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:51,007 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cdc19523eb053795d42495/lava-logs.yaml 2026-04-02 01:09:51,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:51,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:51,161 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cdc19523eb053795d42495/lava_callback.json.gz 2026-04-02 01:09:51,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:51,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19523eb053795d42495 HTTP/1.1" 200 None 2026-04-02 01:09:51,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:51,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:09:51,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:51,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc19523eb053795d42495 HTTP/1.1" 200 None 2026-04-02 01:09:51,242 - lava_callback - INFO - Completed processing callback for node 69cdc19523eb053795d42495 INFO: 172.20.0.1:51268 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:09:56,798 - lava_callback - INFO - Received callback for node 69cdc19723eb053795d42499 INFO: 172.20.0.1:51272 - "POST /node/69cdc19723eb053795d42499 HTTP/1.0" 202 Accepted 2026-04-02 01:09:56,804 - lava_callback - INFO - Starting async job submit for node 69cdc19723eb053795d42499 2026-04-02 01:09:56,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:56,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19723eb053795d42499 HTTP/1.1" 200 None 2026-04-02 01:09:56,839 - lava_callback - DEBUG - Processing node 69cdc19723eb053795d42499: baseline-arm64-pengutronix 2026-04-02 01:09:57,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:57,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:57,198 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cdc19723eb053795d42499/log.txt.gz 2026-04-02 01:09:57,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:57,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:57,558 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cdc19723eb053795d42499/lava-logs.yaml 2026-04-02 01:09:57,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:09:57,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:09:57,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cdc19723eb053795d42499/lava_callback.json.gz 2026-04-02 01:09:57,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:57,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19723eb053795d42499 HTTP/1.1" 200 None 2026-04-02 01:09:57,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:57,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:09:57,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:09:57,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc19723eb053795d42499 HTTP/1.1" 200 None 2026-04-02 01:09:57,818 - lava_callback - INFO - Completed processing callback for node 69cdc19723eb053795d42499 INFO: 172.20.0.1:46088 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:10:08,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:08,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-02 01:10:11,024 - lava_callback - INFO - Received callback for node 69cdc18a23eb053795d42474 INFO: 172.20.0.1:59344 - "POST /node/69cdc18a23eb053795d42474 HTTP/1.0" 202 Accepted 2026-04-02 01:10:11,030 - lava_callback - INFO - Starting async job submit for node 69cdc18a23eb053795d42474 2026-04-02 01:10:11,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:11,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18a23eb053795d42474 HTTP/1.1" 200 None 2026-04-02 01:10:11,070 - lava_callback - DEBUG - Processing node 69cdc18a23eb053795d42474: baseline-arm64-broonie 2026-04-02 01:10:11,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:11,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:11,480 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18a23eb053795d42474/log.txt.gz 2026-04-02 01:10:11,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:11,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:11,814 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18a23eb053795d42474/lava-logs.yaml 2026-04-02 01:10:11,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:11,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:11,983 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18a23eb053795d42474/lava_callback.json.gz 2026-04-02 01:10:12,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:12,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18a23eb053795d42474 HTTP/1.1" 200 None 2026-04-02 01:10:12,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:12,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:10:12,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:12,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18a23eb053795d42474 HTTP/1.1" 200 None 2026-04-02 01:10:12,067 - lava_callback - INFO - Completed processing callback for node 69cdc18a23eb053795d42474 INFO: 172.20.0.1:59350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37200 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:10:27,875 - lava_callback - INFO - Received callback for node 69cdc18b23eb053795d42477 INFO: 172.20.0.1:37206 - "POST /node/69cdc18b23eb053795d42477 HTTP/1.0" 202 Accepted 2026-04-02 01:10:27,881 - lava_callback - INFO - Starting async job submit for node 69cdc18b23eb053795d42477 2026-04-02 01:10:27,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:27,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18b23eb053795d42477 HTTP/1.1" 200 None 2026-04-02 01:10:27,920 - lava_callback - DEBUG - Processing node 69cdc18b23eb053795d42477: baseline-arm64-broonie 2026-04-02 01:10:28,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:28,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:28,406 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18b23eb053795d42477/log.txt.gz 2026-04-02 01:10:28,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:28,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:28,779 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18b23eb053795d42477/lava-logs.yaml 2026-04-02 01:10:28,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:28,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:28,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18b23eb053795d42477/lava_callback.json.gz 2026-04-02 01:10:28,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:28,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18b23eb053795d42477 HTTP/1.1" 200 None 2026-04-02 01:10:28,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:29,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:10:29,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:29,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18b23eb053795d42477 HTTP/1.1" 200 None 2026-04-02 01:10:29,062 - lava_callback - INFO - Completed processing callback for node 69cdc18b23eb053795d42477 2026-04-02 01:10:31,017 - lava_callback - INFO - Received callback for node 69cdc18223eb053795d42462 INFO: 172.20.0.1:58186 - "POST /node/69cdc18223eb053795d42462 HTTP/1.0" 202 Accepted 2026-04-02 01:10:31,027 - lava_callback - INFO - Starting async job submit for node 69cdc18223eb053795d42462 2026-04-02 01:10:31,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:31,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18223eb053795d42462 HTTP/1.1" 200 None 2026-04-02 01:10:31,087 - lava_callback - DEBUG - Processing node 69cdc18223eb053795d42462: baseline-arm64-broonie 2026-04-02 01:10:31,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:31,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:31,641 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18223eb053795d42462/log.txt.gz 2026-04-02 01:10:31,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:32,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:32,095 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18223eb053795d42462/lava-logs.yaml 2026-04-02 01:10:32,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:32,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:32,266 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18223eb053795d42462/lava_callback.json.gz 2026-04-02 01:10:32,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:32,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18223eb053795d42462 HTTP/1.1" 200 None 2026-04-02 01:10:32,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:32,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:10:32,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:32,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18223eb053795d42462 HTTP/1.1" 200 None 2026-04-02 01:10:32,364 - lava_callback - INFO - Completed processing callback for node 69cdc18223eb053795d42462 2026-04-02 01:10:35,548 - lava_callback - INFO - Received callback for node 69cdc19023eb053795d42486 INFO: 172.20.0.1:58192 - "POST /node/69cdc19023eb053795d42486 HTTP/1.0" 202 Accepted 2026-04-02 01:10:35,554 - lava_callback - INFO - Starting async job submit for node 69cdc19023eb053795d42486 2026-04-02 01:10:35,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:35,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19023eb053795d42486 HTTP/1.1" 200 None 2026-04-02 01:10:35,594 - lava_callback - DEBUG - Processing node 69cdc19023eb053795d42486: baseline-arm64-clabbe 2026-04-02 01:10:35,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:35,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:35,942 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cdc19023eb053795d42486/log.txt.gz 2026-04-02 01:10:36,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:36,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:36,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cdc19023eb053795d42486/lava-logs.yaml 2026-04-02 01:10:36,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:36,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:36,427 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cdc19023eb053795d42486/lava_callback.json.gz 2026-04-02 01:10:36,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:36,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19023eb053795d42486 HTTP/1.1" 200 None 2026-04-02 01:10:36,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:36,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:10:36,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:36,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc19023eb053795d42486 HTTP/1.1" 200 None 2026-04-02 01:10:36,510 - lava_callback - INFO - Completed processing callback for node 69cdc19023eb053795d42486 INFO: 172.20.0.1:58204 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:10:38,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:38,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:40066 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:10:51,123 - lava_callback - INFO - Received callback for node 69cdc18323eb053795d42465 INFO: 172.20.0.1:41310 - "POST /node/69cdc18323eb053795d42465 HTTP/1.0" 202 Accepted 2026-04-02 01:10:51,133 - lava_callback - INFO - Starting async job submit for node 69cdc18323eb053795d42465 2026-04-02 01:10:51,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:51,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18323eb053795d42465 HTTP/1.1" 200 None 2026-04-02 01:10:51,192 - lava_callback - DEBUG - Processing node 69cdc18323eb053795d42465: baseline-arm64-broonie 2026-04-02 01:10:51,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:51,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:51,743 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18323eb053795d42465/log.txt.gz 2026-04-02 01:10:51,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:52,171 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:52,173 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18323eb053795d42465/lava-logs.yaml 2026-04-02 01:10:52,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:52,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:52,310 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18323eb053795d42465/lava_callback.json.gz 2026-04-02 01:10:52,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:52,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18323eb053795d42465 HTTP/1.1" 200 None 2026-04-02 01:10:52,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:52,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:10:52,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:52,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18323eb053795d42465 HTTP/1.1" 200 None 2026-04-02 01:10:52,413 - lava_callback - INFO - Completed processing callback for node 69cdc18323eb053795d42465 2026-04-02 01:10:52,547 - lava_callback - INFO - Received callback for node 69cdc18823eb053795d4246f INFO: 172.20.0.1:41324 - "POST /node/69cdc18823eb053795d4246f HTTP/1.0" 202 Accepted 2026-04-02 01:10:52,558 - lava_callback - INFO - Starting async job submit for node 69cdc18823eb053795d4246f 2026-04-02 01:10:52,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:52,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18823eb053795d4246f HTTP/1.1" 200 None 2026-04-02 01:10:52,612 - lava_callback - DEBUG - Processing node 69cdc18823eb053795d4246f: baseline-arm64-broonie 2026-04-02 01:10:52,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:53,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:53,002 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18823eb053795d4246f/log.txt.gz 2026-04-02 01:10:53,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:53,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:53,413 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18823eb053795d4246f/lava-logs.yaml 2026-04-02 01:10:53,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:10:53,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:10:53,575 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18823eb053795d4246f/lava_callback.json.gz 2026-04-02 01:10:53,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:53,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18823eb053795d4246f HTTP/1.1" 200 None 2026-04-02 01:10:53,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:53,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:10:53,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:10:53,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18823eb053795d4246f HTTP/1.1" 200 None 2026-04-02 01:10:53,666 - lava_callback - INFO - Completed processing callback for node 69cdc18823eb053795d4246f INFO: 172.20.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44614 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:11:07,926 - lava_callback - INFO - Received callback for node 69cdc18023eb053795d42443 INFO: 172.20.0.1:44630 - "POST /node/69cdc18023eb053795d42443 HTTP/1.0" 202 Accepted 2026-04-02 01:11:07,933 - lava_callback - INFO - Starting async job submit for node 69cdc18023eb053795d42443 2026-04-02 01:11:07,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:07,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18023eb053795d42443 HTTP/1.1" 200 None 2026-04-02 01:11:07,972 - lava_callback - DEBUG - Processing node 69cdc18023eb053795d42443: baseline-arm64-broonie 2026-04-02 01:11:08,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:08,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:08,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 01:11:08,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:08,313 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18023eb053795d42443/log.txt.gz 2026-04-02 01:11:08,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:08,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:08,621 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18023eb053795d42443/lava-logs.yaml 2026-04-02 01:11:08,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:08,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:08,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18023eb053795d42443/lava_callback.json.gz 2026-04-02 01:11:08,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:08,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18023eb053795d42443 HTTP/1.1" 200 None 2026-04-02 01:11:08,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:08,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:08,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:08,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18023eb053795d42443 HTTP/1.1" 200 None 2026-04-02 01:11:08,851 - lava_callback - INFO - Completed processing callback for node 69cdc18023eb053795d42443 INFO: 172.20.0.1:50982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50996 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:11:19,737 - lava_callback - INFO - Received callback for node 69cdc15423eb053795d423b1 INFO: 172.20.0.1:42616 - "POST /node/69cdc15423eb053795d423b1 HTTP/1.0" 202 Accepted 2026-04-02 01:11:19,745 - lava_callback - INFO - Starting async job submit for node 69cdc15423eb053795d423b1 2026-04-02 01:11:19,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:19,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15423eb053795d423b1 HTTP/1.1" 200 None 2026-04-02 01:11:19,791 - lava_callback - DEBUG - Processing node 69cdc15423eb053795d423b1: baseline-arm64 2026-04-02 01:11:19,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:20,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:20,094 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15423eb053795d423b1/log.txt.gz 2026-04-02 01:11:20,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:20,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:20,340 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15423eb053795d423b1/lava-logs.yaml 2026-04-02 01:11:20,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:20,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:20,542 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15423eb053795d423b1/lava_callback.json.gz 2026-04-02 01:11:20,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:20,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15423eb053795d423b1 HTTP/1.1" 200 None 2026-04-02 01:11:20,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:20,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:20,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:20,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15423eb053795d423b1 HTTP/1.1" 200 None 2026-04-02 01:11:20,716 - lava_callback - INFO - Completed processing callback for node 69cdc15423eb053795d423b1 2026-04-02 01:11:22,081 - lava_callback - INFO - Received callback for node 69cdc15323eb053795d423ae INFO: 172.20.0.1:42628 - "POST /node/69cdc15323eb053795d423ae HTTP/1.0" 202 Accepted 2026-04-02 01:11:22,088 - lava_callback - INFO - Starting async job submit for node 69cdc15323eb053795d423ae 2026-04-02 01:11:22,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:22,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15323eb053795d423ae HTTP/1.1" 200 None 2026-04-02 01:11:22,134 - lava_callback - DEBUG - Processing node 69cdc15323eb053795d423ae: baseline-arm64 2026-04-02 01:11:22,275 - lava_callback - WARNING - Failed to upload log 2026-04-02 01:11:22,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:22,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:22,516 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15323eb053795d423ae/lava-logs.yaml 2026-04-02 01:11:22,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:22,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:22,661 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15323eb053795d423ae/lava_callback.json.gz 2026-04-02 01:11:22,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:22,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15323eb053795d423ae HTTP/1.1" 200 None 2026-04-02 01:11:22,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:22,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:22,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:22,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15323eb053795d423ae HTTP/1.1" 200 None 2026-04-02 01:11:22,857 - lava_callback - INFO - Completed processing callback for node 69cdc15323eb053795d423ae 2026-04-02 01:11:23,252 - lava_callback - INFO - Received callback for node 69cdc16023eb053795d423cf INFO: 172.20.0.1:42630 - "POST /node/69cdc16023eb053795d423cf HTTP/1.0" 202 Accepted 2026-04-02 01:11:23,259 - lava_callback - INFO - Starting async job submit for node 69cdc16023eb053795d423cf 2026-04-02 01:11:23,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:23,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16023eb053795d423cf HTTP/1.1" 200 None 2026-04-02 01:11:23,298 - lava_callback - DEBUG - Processing node 69cdc16023eb053795d423cf: baseline-arm64 INFO: 172.20.0.1:42636 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:11:23,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:23,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:23,793 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc16023eb053795d423cf/log.txt.gz 2026-04-02 01:11:23,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:24,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:24,150 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc16023eb053795d423cf/lava-logs.yaml 2026-04-02 01:11:24,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:24,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:24,322 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc16023eb053795d423cf/lava_callback.json.gz 2026-04-02 01:11:24,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:24,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16023eb053795d423cf HTTP/1.1" 200 None 2026-04-02 01:11:24,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:24,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:24,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:24,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16023eb053795d423cf HTTP/1.1" 200 None 2026-04-02 01:11:24,417 - lava_callback - INFO - Completed processing callback for node 69cdc16023eb053795d423cf INFO: 172.20.0.1:42652 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:11:28,637 - lava_callback - INFO - Received callback for node 69cdc15c23eb053795d423c6 INFO: 172.20.0.1:42660 - "POST /node/69cdc15c23eb053795d423c6 HTTP/1.0" 202 Accepted 2026-04-02 01:11:28,644 - lava_callback - INFO - Starting async job submit for node 69cdc15c23eb053795d423c6 2026-04-02 01:11:28,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:28,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15c23eb053795d423c6 HTTP/1.1" 200 None 2026-04-02 01:11:28,689 - lava_callback - DEBUG - Processing node 69cdc15c23eb053795d423c6: baseline-arm64 2026-04-02 01:11:28,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:29,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:29,094 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15c23eb053795d423c6/log.txt.gz 2026-04-02 01:11:29,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:29,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:29,476 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15c23eb053795d423c6/lava-logs.yaml 2026-04-02 01:11:29,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:29,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:29,626 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15c23eb053795d423c6/lava_callback.json.gz 2026-04-02 01:11:29,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:29,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15c23eb053795d423c6 HTTP/1.1" 200 None 2026-04-02 01:11:29,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:29,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:29,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:29,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15c23eb053795d423c6 HTTP/1.1" 200 None 2026-04-02 01:11:29,704 - lava_callback - INFO - Completed processing callback for node 69cdc15c23eb053795d423c6 2026-04-02 01:11:35,732 - lava_callback - INFO - Received callback for node 69cdc15623eb053795d423b7 INFO: 172.20.0.1:51142 - "POST /node/69cdc15623eb053795d423b7 HTTP/1.0" 202 Accepted 2026-04-02 01:11:35,738 - lava_callback - INFO - Starting async job submit for node 69cdc15623eb053795d423b7 2026-04-02 01:11:35,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:35,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15623eb053795d423b7 HTTP/1.1" 200 None 2026-04-02 01:11:35,786 - lava_callback - DEBUG - Processing node 69cdc15623eb053795d423b7: baseline-arm64 2026-04-02 01:11:36,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:36,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:36,208 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15623eb053795d423b7/log.txt.gz 2026-04-02 01:11:36,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:51152 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:11:36,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:36,556 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15623eb053795d423b7/lava-logs.yaml 2026-04-02 01:11:36,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:36,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:36,748 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15623eb053795d423b7/lava_callback.json.gz 2026-04-02 01:11:36,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:36,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15623eb053795d423b7 HTTP/1.1" 200 None 2026-04-02 01:11:36,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:36,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:36,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:36,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15623eb053795d423b7 HTTP/1.1" 200 None 2026-04-02 01:11:36,848 - lava_callback - INFO - Completed processing callback for node 69cdc15623eb053795d423b7 2026-04-02 01:11:38,163 - lava_callback - INFO - Received callback for node 69cb1f448c06c2799bf57a7e INFO: 172.20.0.1:51156 - "POST /node/69cb1f448c06c2799bf57a7e HTTP/1.0" 202 Accepted 2026-04-02 01:11:38,169 - lava_callback - INFO - Starting async job submit for node 69cb1f448c06c2799bf57a7e 2026-04-02 01:11:38,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:38,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f448c06c2799bf57a7e HTTP/1.1" 200 None 2026-04-02 01:11:38,188 - lava_callback - DEBUG - Processing node 69cb1f448c06c2799bf57a7e: baseline-arm64-baylibre 2026-04-02 01:11:38,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f448c06c2799bf57a7e 2026-04-02 01:11:38,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:38,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f448c06c2799bf57a7e HTTP/1.1" 200 None 2026-04-02 01:11:38,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:38,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 01:11:38,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:38,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f448c06c2799bf57a7e HTTP/1.1" 200 None 2026-04-02 01:11:38,245 - lava_callback - ERROR - Error processing callback for node 69cb1f448c06c2799bf57a7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:11:38,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:38,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-02 01:11:38,621 - lava_callback - INFO - Received callback for node 69cdc15923eb053795d423bd INFO: 172.20.0.1:51168 - "POST /node/69cdc15923eb053795d423bd HTTP/1.0" 202 Accepted 2026-04-02 01:11:38,628 - lava_callback - INFO - Starting async job submit for node 69cdc15923eb053795d423bd 2026-04-02 01:11:38,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:38,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15923eb053795d423bd HTTP/1.1" 200 None 2026-04-02 01:11:38,668 - lava_callback - DEBUG - Processing node 69cdc15923eb053795d423bd: baseline-arm64 2026-04-02 01:11:39,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:39,270 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:39,272 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15923eb053795d423bd/log.txt.gz 2026-04-02 01:11:39,311 - lava_callback - INFO - Received callback for node 69cdc16123eb053795d423d2 INFO: 172.20.0.1:51174 - "POST /node/69cdc16123eb053795d423d2 HTTP/1.0" 202 Accepted 2026-04-02 01:11:39,337 - lava_callback - INFO - Starting async job submit for node 69cdc16123eb053795d423d2 2026-04-02 01:11:39,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:39,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16123eb053795d423d2 HTTP/1.1" 200 None 2026-04-02 01:11:39,472 - lava_callback - DEBUG - Processing node 69cdc16123eb053795d423d2: baseline-arm64 2026-04-02 01:11:39,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:40,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:40,142 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15923eb053795d423bd/lava-logs.yaml 2026-04-02 01:11:40,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:40,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:40,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:40,319 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc16123eb053795d423d2/log.txt.gz 2026-04-02 01:11:40,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:40,407 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15923eb053795d423bd/lava_callback.json.gz 2026-04-02 01:11:40,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:40,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:40,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15923eb053795d423bd HTTP/1.1" 200 None 2026-04-02 01:11:40,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:40,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:40,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:40,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15923eb053795d423bd HTTP/1.1" 200 None 2026-04-02 01:11:40,606 - lava_callback - INFO - Completed processing callback for node 69cdc15923eb053795d423bd 2026-04-02 01:11:40,714 - lava_callback - INFO - Received callback for node 69cdc15f23eb053795d423cc INFO: 172.20.0.1:41100 - "POST /node/69cdc15f23eb053795d423cc HTTP/1.0" 202 Accepted 2026-04-02 01:11:40,724 - lava_callback - INFO - Starting async job submit for node 69cdc15f23eb053795d423cc 2026-04-02 01:11:40,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:40,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15f23eb053795d423cc HTTP/1.1" 200 None 2026-04-02 01:11:40,781 - lava_callback - DEBUG - Processing node 69cdc15f23eb053795d423cc: baseline-arm64 2026-04-02 01:11:40,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:40,883 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc16123eb053795d423d2/lava-logs.yaml 2026-04-02 01:11:40,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:41,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:41,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:41,189 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc16123eb053795d423d2/lava_callback.json.gz 2026-04-02 01:11:41,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:41,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:41,216 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15f23eb053795d423cc/log.txt.gz 2026-04-02 01:11:41,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16123eb053795d423d2 HTTP/1.1" 200 None 2026-04-02 01:11:41,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:41,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:41,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:41,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:41,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16123eb053795d423d2 HTTP/1.1" 200 None 2026-04-02 01:11:41,422 - lava_callback - INFO - Completed processing callback for node 69cdc16123eb053795d423d2 2026-04-02 01:11:41,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:41,630 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15f23eb053795d423cc/lava-logs.yaml 2026-04-02 01:11:41,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:41,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:41,766 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15f23eb053795d423cc/lava_callback.json.gz 2026-04-02 01:11:41,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:41,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15f23eb053795d423cc HTTP/1.1" 200 None 2026-04-02 01:11:41,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:41,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:41,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:41,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15f23eb053795d423cc HTTP/1.1" 200 None 2026-04-02 01:11:41,847 - lava_callback - INFO - Completed processing callback for node 69cdc15f23eb053795d423cc 2026-04-02 01:11:43,227 - lava_callback - INFO - Received callback for node 69cdc15823eb053795d423ba INFO: 172.20.0.1:41106 - "POST /node/69cdc15823eb053795d423ba HTTP/1.0" 202 Accepted 2026-04-02 01:11:43,237 - lava_callback - INFO - Starting async job submit for node 69cdc15823eb053795d423ba 2026-04-02 01:11:43,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:43,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15823eb053795d423ba HTTP/1.1" 200 None 2026-04-02 01:11:43,277 - lava_callback - DEBUG - Processing node 69cdc15823eb053795d423ba: baseline-arm64 2026-04-02 01:11:43,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:43,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:43,797 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15823eb053795d423ba/log.txt.gz 2026-04-02 01:11:44,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:44,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:44,215 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15823eb053795d423ba/lava-logs.yaml 2026-04-02 01:11:44,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:44,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:44,357 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15823eb053795d423ba/lava_callback.json.gz 2026-04-02 01:11:44,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:44,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15823eb053795d423ba HTTP/1.1" 200 None 2026-04-02 01:11:44,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:44,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:44,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:44,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15823eb053795d423ba HTTP/1.1" 200 None 2026-04-02 01:11:44,440 - lava_callback - INFO - Completed processing callback for node 69cdc15823eb053795d423ba 2026-04-02 01:11:46,388 - lava_callback - INFO - Received callback for node 69cdc19923eb053795d4249f INFO: 172.20.0.1:41122 - "POST /node/69cdc19923eb053795d4249f HTTP/1.0" 202 Accepted 2026-04-02 01:11:46,396 - lava_callback - INFO - Starting async job submit for node 69cdc19923eb053795d4249f 2026-04-02 01:11:46,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:46,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19923eb053795d4249f HTTP/1.1" 200 None 2026-04-02 01:11:46,453 - lava_callback - DEBUG - Processing node 69cdc19923eb053795d4249f: baseline-arm64-foundriesio INFO: 172.20.0.1:41130 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:11:47,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:47,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:47,421 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cdc19923eb053795d4249f/log.txt.gz 2026-04-02 01:11:47,786 - lava_callback - INFO - Received callback for node 69cdc17123eb053795d42413 INFO: 172.20.0.1:41134 - "POST /node/69cdc17123eb053795d42413 HTTP/1.0" 202 Accepted 2026-04-02 01:11:47,811 - lava_callback - INFO - Starting async job submit for node 69cdc17123eb053795d42413 2026-04-02 01:11:47,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:47,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17123eb053795d42413 HTTP/1.1" 200 None 2026-04-02 01:11:47,958 - lava_callback - DEBUG - Processing node 69cdc17123eb053795d42413: baseline-nfs-arm64 2026-04-02 01:11:48,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:48,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:48,433 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cdc19923eb053795d4249f/lava-logs.yaml 2026-04-02 01:11:48,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:48,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:48,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:48,620 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc17123eb053795d42413/log.txt.gz 2026-04-02 01:11:48,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:48,734 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cdc19923eb053795d4249f/lava_callback.json.gz DEBUG: 69cdc19923eb053795d4249f Transitting job node result: pass -> incomplete 2026-04-02 01:11:48,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:48,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19923eb053795d4249f HTTP/1.1" 200 None 2026-04-02 01:11:48,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:48,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:48,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:48,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:48,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc19923eb053795d4249f HTTP/1.1" 200 None 2026-04-02 01:11:48,979 - lava_callback - INFO - Completed processing callback for node 69cdc19923eb053795d4249f 2026-04-02 01:11:49,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:49,214 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc17123eb053795d42413/lava-logs.yaml 2026-04-02 01:11:49,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:49,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:49,363 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc17123eb053795d42413/lava_callback.json.gz 2026-04-02 01:11:49,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:49,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17123eb053795d42413 HTTP/1.1" 200 None 2026-04-02 01:11:49,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:49,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:49,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:49,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc17123eb053795d42413 HTTP/1.1" 200 None 2026-04-02 01:11:49,442 - lava_callback - INFO - Completed processing callback for node 69cdc17123eb053795d42413 2026-04-02 01:11:49,808 - lava_callback - INFO - Received callback for node 69cdc16923eb053795d42401 INFO: 172.20.0.1:34162 - "POST /node/69cdc16923eb053795d42401 HTTP/1.0" 202 Accepted 2026-04-02 01:11:49,815 - lava_callback - INFO - Starting async job submit for node 69cdc16923eb053795d42401 2026-04-02 01:11:49,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:49,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16923eb053795d42401 HTTP/1.1" 200 None 2026-04-02 01:11:49,855 - lava_callback - DEBUG - Processing node 69cdc16923eb053795d42401: baseline-nfs-arm64 2026-04-02 01:11:50,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:50,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:50,460 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16923eb053795d42401/log.txt.gz 2026-04-02 01:11:50,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:51,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:51,116 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16923eb053795d42401/lava-logs.yaml 2026-04-02 01:11:51,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:11:51,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:11:51,287 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16923eb053795d42401/lava_callback.json.gz 2026-04-02 01:11:51,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:51,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16923eb053795d42401 HTTP/1.1" 200 None 2026-04-02 01:11:51,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:51,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:11:51,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:51,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16923eb053795d42401 HTTP/1.1" 200 None 2026-04-02 01:11:51,373 - lava_callback - INFO - Completed processing callback for node 69cdc16923eb053795d42401 INFO: 172.20.0.1:34178 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:11:57,904 - lava_callback - INFO - Received callback for node 69cb1f4d8c06c2799bf57a94 INFO: 172.20.0.1:34180 - "POST /node/69cb1f4d8c06c2799bf57a94 HTTP/1.0" 202 Accepted 2026-04-02 01:11:57,910 - lava_callback - INFO - Starting async job submit for node 69cb1f4d8c06c2799bf57a94 2026-04-02 01:11:57,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:57,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f4d8c06c2799bf57a94 HTTP/1.1" 200 None 2026-04-02 01:11:57,930 - lava_callback - DEBUG - Processing node 69cb1f4d8c06c2799bf57a94: baseline-arm64-baylibre 2026-04-02 01:11:57,930 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f4d8c06c2799bf57a94 2026-04-02 01:11:57,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:57,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f4d8c06c2799bf57a94 HTTP/1.1" 200 None 2026-04-02 01:11:57,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:57,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:11:57,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:57,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f4d8c06c2799bf57a94 HTTP/1.1" 200 None 2026-04-02 01:11:57,989 - lava_callback - ERROR - Error processing callback for node 69cb1f4d8c06c2799bf57a94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:11:58,634 - lava_callback - INFO - Received callback for node 69cb1f4f8c06c2799bf57a97 INFO: 172.20.0.1:34194 - "POST /node/69cb1f4f8c06c2799bf57a97 HTTP/1.0" 202 Accepted 2026-04-02 01:11:58,640 - lava_callback - INFO - Starting async job submit for node 69cb1f4f8c06c2799bf57a97 2026-04-02 01:11:58,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:58,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f4f8c06c2799bf57a97 HTTP/1.1" 200 None 2026-04-02 01:11:58,656 - lava_callback - DEBUG - Processing node 69cb1f4f8c06c2799bf57a97: baseline-arm64-baylibre 2026-04-02 01:11:58,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f4f8c06c2799bf57a97 2026-04-02 01:11:58,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:58,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f4f8c06c2799bf57a97 HTTP/1.1" 200 None 2026-04-02 01:11:58,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:58,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:11:58,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:58,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f4f8c06c2799bf57a97 HTTP/1.1" 200 None 2026-04-02 01:11:58,714 - lava_callback - ERROR - Error processing callback for node 69cb1f4f8c06c2799bf57a97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:11:59,296 - lava_callback - INFO - Received callback for node 69cb1f508c06c2799bf57a9a INFO: 172.20.0.1:34196 - "POST /node/69cb1f508c06c2799bf57a9a HTTP/1.0" 202 Accepted 2026-04-02 01:11:59,303 - lava_callback - INFO - Starting async job submit for node 69cb1f508c06c2799bf57a9a 2026-04-02 01:11:59,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:59,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f508c06c2799bf57a9a HTTP/1.1" 200 None 2026-04-02 01:11:59,325 - lava_callback - DEBUG - Processing node 69cb1f508c06c2799bf57a9a: baseline-arm64-baylibre 2026-04-02 01:11:59,326 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f508c06c2799bf57a9a 2026-04-02 01:11:59,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:59,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f508c06c2799bf57a9a HTTP/1.1" 200 None 2026-04-02 01:11:59,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:59,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:11:59,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:59,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f508c06c2799bf57a9a HTTP/1.1" 200 None 2026-04-02 01:11:59,387 - lava_callback - ERROR - Error processing callback for node 69cb1f508c06c2799bf57a9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:11:59,901 - lava_callback - INFO - Received callback for node 69cdc16f23eb053795d42410 INFO: 172.20.0.1:53226 - "POST /node/69cdc16f23eb053795d42410 HTTP/1.0" 202 Accepted 2026-04-02 01:11:59,909 - lava_callback - INFO - Starting async job submit for node 69cdc16f23eb053795d42410 2026-04-02 01:11:59,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:11:59,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16f23eb053795d42410 HTTP/1.1" 200 None 2026-04-02 01:11:59,966 - lava_callback - DEBUG - Processing node 69cdc16f23eb053795d42410: baseline-nfs-arm64 2026-04-02 01:11:59,995 - lava_callback - INFO - Received callback for node 69cb1f538c06c2799bf57aa1 INFO: 172.20.0.1:53236 - "POST /node/69cb1f538c06c2799bf57aa1 HTTP/1.0" 202 Accepted 2026-04-02 01:12:00,015 - lava_callback - INFO - Starting async job submit for node 69cb1f538c06c2799bf57aa1 2026-04-02 01:12:00,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:00,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f538c06c2799bf57aa1 HTTP/1.1" 200 None 2026-04-02 01:12:00,138 - lava_callback - DEBUG - Processing node 69cb1f538c06c2799bf57aa1: baseline-arm64-baylibre 2026-04-02 01:12:00,143 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f538c06c2799bf57aa1 2026-04-02 01:12:00,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:00,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f538c06c2799bf57aa1 HTTP/1.1" 200 None 2026-04-02 01:12:00,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:00,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:12:00,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:00,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:00,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f538c06c2799bf57aa1 HTTP/1.1" 200 None 2026-04-02 01:12:00,432 - lava_callback - ERROR - Error processing callback for node 69cb1f538c06c2799bf57aa1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:12:00,653 - lava_callback - INFO - Received callback for node 69cb1f558c06c2799bf57aa4 INFO: 172.20.0.1:53252 - "POST /node/69cb1f558c06c2799bf57aa4 HTTP/1.0" 202 Accepted 2026-04-02 01:12:00,660 - lava_callback - INFO - Starting async job submit for node 69cb1f558c06c2799bf57aa4 2026-04-02 01:12:00,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:00,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f558c06c2799bf57aa4 HTTP/1.1" 200 None 2026-04-02 01:12:00,682 - lava_callback - DEBUG - Processing node 69cb1f558c06c2799bf57aa4: baseline-arm64-baylibre 2026-04-02 01:12:00,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f558c06c2799bf57aa4 2026-04-02 01:12:00,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:00,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:00,698 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16f23eb053795d42410/log.txt.gz 2026-04-02 01:12:00,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f558c06c2799bf57aa4 HTTP/1.1" 200 None 2026-04-02 01:12:00,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:00,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:12:00,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:00,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:00,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f558c06c2799bf57aa4 HTTP/1.1" 200 None 2026-04-02 01:12:00,942 - lava_callback - ERROR - Error processing callback for node 69cb1f558c06c2799bf57aa4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:12:01,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:01,221 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16f23eb053795d42410/lava-logs.yaml 2026-04-02 01:12:01,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:01,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:01,386 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16f23eb053795d42410/lava_callback.json.gz 2026-04-02 01:12:01,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:01,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16f23eb053795d42410 HTTP/1.1" 200 None 2026-04-02 01:12:01,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:01,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:12:01,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:01,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16f23eb053795d42410 HTTP/1.1" 200 None 2026-04-02 01:12:01,474 - lava_callback - INFO - Completed processing callback for node 69cdc16f23eb053795d42410 2026-04-02 01:12:03,890 - lava_callback - INFO - Received callback for node 69cdc15223eb053795d423ab INFO: 172.20.0.1:53264 - "POST /node/69cdc15223eb053795d423ab HTTP/1.0" 202 Accepted 2026-04-02 01:12:03,897 - lava_callback - INFO - Starting async job submit for node 69cdc15223eb053795d423ab 2026-04-02 01:12:03,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:03,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15223eb053795d423ab HTTP/1.1" 200 None 2026-04-02 01:12:03,932 - lava_callback - DEBUG - Processing node 69cdc15223eb053795d423ab: baseline-arm64 2026-04-02 01:12:04,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:04,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:04,299 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15223eb053795d423ab/log.txt.gz 2026-04-02 01:12:04,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:04,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:04,610 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15223eb053795d423ab/lava-logs.yaml 2026-04-02 01:12:04,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:04,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:04,762 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15223eb053795d423ab/lava_callback.json.gz 2026-04-02 01:12:04,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:04,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15223eb053795d423ab HTTP/1.1" 200 None 2026-04-02 01:12:04,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:04,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:12:04,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:04,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15223eb053795d423ab HTTP/1.1" 200 None 2026-04-02 01:12:04,882 - lava_callback - INFO - Completed processing callback for node 69cdc15223eb053795d423ab INFO: 172.20.0.1:53268 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:12:07,830 - lava_callback - INFO - Received callback for node 69cdc18523eb053795d42468 INFO: 172.20.0.1:53272 - "POST /node/69cdc18523eb053795d42468 HTTP/1.0" 202 Accepted 2026-04-02 01:12:07,837 - lava_callback - INFO - Starting async job submit for node 69cdc18523eb053795d42468 2026-04-02 01:12:07,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:07,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18523eb053795d42468 HTTP/1.1" 200 None 2026-04-02 01:12:07,894 - lava_callback - DEBUG - Processing node 69cdc18523eb053795d42468: baseline-arm64-broonie 2026-04-02 01:12:08,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:08,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:08,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-02 01:12:08,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:08,380 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18523eb053795d42468/log.txt.gz 2026-04-02 01:12:08,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:08,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:08,761 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18523eb053795d42468/lava-logs.yaml 2026-04-02 01:12:08,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:08,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:08,919 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cdc18523eb053795d42468/lava_callback.json.gz 2026-04-02 01:12:08,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:08,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18523eb053795d42468 HTTP/1.1" 200 None 2026-04-02 01:12:08,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:08,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:12:08,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:09,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18523eb053795d42468 HTTP/1.1" 200 None 2026-04-02 01:12:09,011 - lava_callback - INFO - Completed processing callback for node 69cdc18523eb053795d42468 2026-04-02 01:12:11,665 - lava_callback - INFO - Received callback for node 69cb1f688c06c2799bf57af0 INFO: 172.20.0.1:58906 - "POST /node/69cb1f688c06c2799bf57af0 HTTP/1.0" 202 Accepted 2026-04-02 01:12:11,671 - lava_callback - INFO - Starting async job submit for node 69cb1f688c06c2799bf57af0 2026-04-02 01:12:11,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:11,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f688c06c2799bf57af0 HTTP/1.1" 200 None 2026-04-02 01:12:11,691 - lava_callback - DEBUG - Processing node 69cb1f688c06c2799bf57af0: baseline-arm64-clabbe 2026-04-02 01:12:11,691 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f688c06c2799bf57af0 2026-04-02 01:12:11,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:11,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f688c06c2799bf57af0 HTTP/1.1" 200 None 2026-04-02 01:12:11,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:11,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:12:11,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:11,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f688c06c2799bf57af0 HTTP/1.1" 200 None 2026-04-02 01:12:11,759 - lava_callback - ERROR - Error processing callback for node 69cb1f688c06c2799bf57af0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:12:12,328 - lava_callback - INFO - Received callback for node 69cb1f6a8c06c2799bf57af3 INFO: 172.20.0.1:58916 - "POST /node/69cb1f6a8c06c2799bf57af3 HTTP/1.0" 202 Accepted 2026-04-02 01:12:12,334 - lava_callback - INFO - Starting async job submit for node 69cb1f6a8c06c2799bf57af3 2026-04-02 01:12:12,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:12,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6a8c06c2799bf57af3 HTTP/1.1" 200 None 2026-04-02 01:12:12,349 - lava_callback - DEBUG - Processing node 69cb1f6a8c06c2799bf57af3: baseline-arm64-clabbe 2026-04-02 01:12:12,349 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f6a8c06c2799bf57af3 2026-04-02 01:12:12,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:12,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6a8c06c2799bf57af3 HTTP/1.1" 200 None 2026-04-02 01:12:12,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:12,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:12:12,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:12,388 - lava_callback - INFO - Received callback for node 69cdc16423eb053795d423f5 INFO: 172.20.0.1:58918 - "POST /node/69cdc16423eb053795d423f5 HTTP/1.0" 202 Accepted 2026-04-02 01:12:12,396 - lava_callback - INFO - Starting async job submit for node 69cdc16423eb053795d423f5 2026-04-02 01:12:12,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:12,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f6a8c06c2799bf57af3 HTTP/1.1" 200 None 2026-04-02 01:12:12,433 - lava_callback - ERROR - Error processing callback for node 69cb1f6a8c06c2799bf57af3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:12:12,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16423eb053795d423f5 HTTP/1.1" 200 None 2026-04-02 01:12:12,471 - lava_callback - DEBUG - Processing node 69cdc16423eb053795d423f5: baseline-nfs-arm64 2026-04-02 01:12:12,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:12,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:12,995 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16423eb053795d423f5/log.txt.gz 2026-04-02 01:12:13,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:13,360 - lava_callback - INFO - Received callback for node 69cdc16a23eb053795d42404 INFO: 172.20.0.1:58934 - "POST /node/69cdc16a23eb053795d42404 HTTP/1.0" 202 Accepted 2026-04-02 01:12:13,368 - lava_callback - INFO - Starting async job submit for node 69cdc16a23eb053795d42404 2026-04-02 01:12:13,394 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:13,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:13,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16423eb053795d423f5/lava-logs.yaml 2026-04-02 01:12:13,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:13,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16a23eb053795d42404 HTTP/1.1" 200 None 2026-04-02 01:12:13,429 - lava_callback - DEBUG - Processing node 69cdc16a23eb053795d42404: baseline-nfs-arm64 2026-04-02 01:12:13,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:13,681 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16423eb053795d423f5/lava_callback.json.gz 2026-04-02 01:12:13,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:13,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16423eb053795d423f5 HTTP/1.1" 200 None 2026-04-02 01:12:13,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:13,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:12:13,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:14,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16423eb053795d423f5 HTTP/1.1" 200 None 2026-04-02 01:12:14,113 - lava_callback - INFO - Completed processing callback for node 69cdc16423eb053795d423f5 2026-04-02 01:12:14,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:14,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:14,300 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16a23eb053795d42404/log.txt.gz 2026-04-02 01:12:14,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:14,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:14,921 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16a23eb053795d42404/lava-logs.yaml 2026-04-02 01:12:14,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:58938 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 01:12:15,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:15,174 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16a23eb053795d42404/lava_callback.json.gz 2026-04-02 01:12:15,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:15,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16a23eb053795d42404 HTTP/1.1" 200 None 2026-04-02 01:12:15,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:15,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:12:15,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:15,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16a23eb053795d42404 HTTP/1.1" 200 None 2026-04-02 01:12:15,263 - lava_callback - INFO - Completed processing callback for node 69cdc16a23eb053795d42404 INFO: 172.20.0.1:58942 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:12:23,404 - lava_callback - INFO - Received callback for node 69cdc17223eb053795d42416 INFO: 172.20.0.1:44454 - "POST /node/69cdc17223eb053795d42416 HTTP/1.0" 202 Accepted 2026-04-02 01:12:23,411 - lava_callback - INFO - Starting async job submit for node 69cdc17223eb053795d42416 2026-04-02 01:12:23,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:23,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17223eb053795d42416 HTTP/1.1" 200 None 2026-04-02 01:12:23,473 - lava_callback - DEBUG - Processing node 69cdc17223eb053795d42416: baseline-nfs-arm64 INFO: 172.20.0.1:44462 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:12:23,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:24,066 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:24,068 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc17223eb053795d42416/log.txt.gz 2026-04-02 01:12:24,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:24,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:24,530 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc17223eb053795d42416/lava-logs.yaml 2026-04-02 01:12:24,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:24,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:24,703 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc17223eb053795d42416/lava_callback.json.gz 2026-04-02 01:12:24,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:24,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17223eb053795d42416 HTTP/1.1" 200 None 2026-04-02 01:12:24,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:24,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:12:24,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:24,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc17223eb053795d42416 HTTP/1.1" 200 None 2026-04-02 01:12:24,789 - lava_callback - INFO - Completed processing callback for node 69cdc17223eb053795d42416 INFO: 172.20.0.1:44472 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:12:30,607 - lava_callback - INFO - Received callback for node 69cdc16c23eb053795d4240a INFO: 172.20.0.1:56458 - "POST /node/69cdc16c23eb053795d4240a HTTP/1.0" 202 Accepted 2026-04-02 01:12:30,618 - lava_callback - INFO - Starting async job submit for node 69cdc16c23eb053795d4240a 2026-04-02 01:12:30,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:30,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16c23eb053795d4240a HTTP/1.1" 200 None 2026-04-02 01:12:30,667 - lava_callback - DEBUG - Processing node 69cdc16c23eb053795d4240a: baseline-nfs-arm64 2026-04-02 01:12:31,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:31,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:31,254 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16c23eb053795d4240a/log.txt.gz 2026-04-02 01:12:31,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:31,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:31,647 - lava_callback - INFO - Received callback for node 69cb1f6b8c06c2799bf57af6 INFO: 172.20.0.1:56472 - "POST /node/69cb1f6b8c06c2799bf57af6 HTTP/1.0" 202 Accepted 2026-04-02 01:12:31,653 - lava_callback - INFO - Starting async job submit for node 69cb1f6b8c06c2799bf57af6 2026-04-02 01:12:31,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:31,654 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16c23eb053795d4240a/lava-logs.yaml 2026-04-02 01:12:31,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:31,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6b8c06c2799bf57af6 HTTP/1.1" 200 None 2026-04-02 01:12:31,669 - lava_callback - DEBUG - Processing node 69cb1f6b8c06c2799bf57af6: baseline-arm64-clabbe 2026-04-02 01:12:31,669 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f6b8c06c2799bf57af6 2026-04-02 01:12:31,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:31,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6b8c06c2799bf57af6 HTTP/1.1" 200 None 2026-04-02 01:12:31,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:31,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:12:31,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:31,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f6b8c06c2799bf57af6 HTTP/1.1" 200 None 2026-04-02 01:12:31,735 - lava_callback - ERROR - Error processing callback for node 69cb1f6b8c06c2799bf57af6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:12:31,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:31,818 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16c23eb053795d4240a/lava_callback.json.gz 2026-04-02 01:12:31,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:31,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16c23eb053795d4240a HTTP/1.1" 200 None 2026-04-02 01:12:31,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:31,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:12:31,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:31,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16c23eb053795d4240a HTTP/1.1" 200 None 2026-04-02 01:12:31,948 - lava_callback - INFO - Completed processing callback for node 69cdc16c23eb053795d4240a 2026-04-02 01:12:32,287 - lava_callback - INFO - Received callback for node 69cb1f6c8c06c2799bf57af9 INFO: 172.20.0.1:56478 - "POST /node/69cb1f6c8c06c2799bf57af9 HTTP/1.0" 202 Accepted 2026-04-02 01:12:32,293 - lava_callback - INFO - Starting async job submit for node 69cb1f6c8c06c2799bf57af9 2026-04-02 01:12:32,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:32,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6c8c06c2799bf57af9 HTTP/1.1" 200 None 2026-04-02 01:12:32,309 - lava_callback - DEBUG - Processing node 69cb1f6c8c06c2799bf57af9: baseline-arm64-clabbe 2026-04-02 01:12:32,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f6c8c06c2799bf57af9 2026-04-02 01:12:32,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:32,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6c8c06c2799bf57af9 HTTP/1.1" 200 None 2026-04-02 01:12:32,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:32,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:12:32,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:32,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f6c8c06c2799bf57af9 HTTP/1.1" 200 None 2026-04-02 01:12:32,366 - lava_callback - ERROR - Error processing callback for node 69cb1f6c8c06c2799bf57af9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:12:32,929 - lava_callback - INFO - Received callback for node 69cb1f6f8c06c2799bf57b02 INFO: 172.20.0.1:56494 - "POST /node/69cb1f6f8c06c2799bf57b02 HTTP/1.0" 202 Accepted 2026-04-02 01:12:32,937 - lava_callback - INFO - Starting async job submit for node 69cb1f6f8c06c2799bf57b02 2026-04-02 01:12:32,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:32,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6f8c06c2799bf57b02 HTTP/1.1" 200 None 2026-04-02 01:12:32,954 - lava_callback - DEBUG - Processing node 69cb1f6f8c06c2799bf57b02: baseline-arm64-clabbe 2026-04-02 01:12:32,954 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f6f8c06c2799bf57b02 2026-04-02 01:12:32,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:32,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6f8c06c2799bf57b02 HTTP/1.1" 200 None 2026-04-02 01:12:32,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:32,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:12:32,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:33,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f6f8c06c2799bf57b02 HTTP/1.1" 200 None 2026-04-02 01:12:33,031 - lava_callback - ERROR - Error processing callback for node 69cb1f6f8c06c2799bf57b02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:12:33,597 - lava_callback - INFO - Received callback for node 69cb1f6e8c06c2799bf57aff INFO: 172.20.0.1:56504 - "POST /node/69cb1f6e8c06c2799bf57aff HTTP/1.0" 202 Accepted 2026-04-02 01:12:33,603 - lava_callback - INFO - Starting async job submit for node 69cb1f6e8c06c2799bf57aff 2026-04-02 01:12:33,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:33,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6e8c06c2799bf57aff HTTP/1.1" 200 None 2026-04-02 01:12:33,618 - lava_callback - DEBUG - Processing node 69cb1f6e8c06c2799bf57aff: baseline-arm64-clabbe 2026-04-02 01:12:33,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1f6e8c06c2799bf57aff 2026-04-02 01:12:33,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:33,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1f6e8c06c2799bf57aff HTTP/1.1" 200 None 2026-04-02 01:12:33,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:33,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 01:12:33,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:33,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1f6e8c06c2799bf57aff HTTP/1.1" 200 None 2026-04-02 01:12:33,679 - lava_callback - ERROR - Error processing callback for node 69cb1f6e8c06c2799bf57aff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56510 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:12:37,741 - lava_callback - INFO - Received callback for node 69cdc15a23eb053795d423c0 INFO: 172.20.0.1:56518 - "POST /node/69cdc15a23eb053795d423c0 HTTP/1.0" 202 Accepted 2026-04-02 01:12:37,748 - lava_callback - INFO - Starting async job submit for node 69cdc15a23eb053795d423c0 2026-04-02 01:12:37,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:37,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15a23eb053795d423c0 HTTP/1.1" 200 None 2026-04-02 01:12:37,788 - lava_callback - DEBUG - Processing node 69cdc15a23eb053795d423c0: baseline-arm64 2026-04-02 01:12:38,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:38,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:38,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-02 01:12:38,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:38,394 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15a23eb053795d423c0/log.txt.gz 2026-04-02 01:12:38,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:38,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:38,996 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15a23eb053795d423c0/lava-logs.yaml 2026-04-02 01:12:39,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:39,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:39,165 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15a23eb053795d423c0/lava_callback.json.gz 2026-04-02 01:12:39,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:39,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15a23eb053795d423c0 HTTP/1.1" 200 None 2026-04-02 01:12:39,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:39,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:12:39,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:39,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15a23eb053795d423c0 HTTP/1.1" 200 None 2026-04-02 01:12:39,371 - lava_callback - INFO - Completed processing callback for node 69cdc15a23eb053795d423c0 INFO: 172.20.0.1:58038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33146 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:12:58,023 - lava_callback - INFO - Received callback for node 69cdc16623eb053795d423fb INFO: 172.20.0.1:33160 - "POST /node/69cdc16623eb053795d423fb HTTP/1.0" 202 Accepted 2026-04-02 01:12:58,035 - lava_callback - INFO - Starting async job submit for node 69cdc16623eb053795d423fb 2026-04-02 01:12:58,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:58,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16623eb053795d423fb HTTP/1.1" 200 None 2026-04-02 01:12:58,136 - lava_callback - DEBUG - Processing node 69cdc16623eb053795d423fb: baseline-nfs-arm64 2026-04-02 01:12:58,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:58,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:58,828 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16623eb053795d423fb/log.txt.gz 2026-04-02 01:12:59,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:59,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:59,519 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16623eb053795d423fb/lava-logs.yaml 2026-04-02 01:12:59,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:12:59,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:12:59,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16623eb053795d423fb/lava_callback.json.gz 2026-04-02 01:12:59,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:59,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16623eb053795d423fb HTTP/1.1" 200 None 2026-04-02 01:12:59,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:59,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:12:59,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:12:59,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16623eb053795d423fb HTTP/1.1" 200 None 2026-04-02 01:12:59,860 - lava_callback - INFO - Completed processing callback for node 69cdc16623eb053795d423fb INFO: 172.20.0.1:47562 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:13:08,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:13:08,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53608 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:13:20,338 - lava_callback - INFO - Received callback for node 69cdc22823eb053795d42540 INFO: 172.20.0.1:37652 - "POST /node/69cdc22823eb053795d42540 HTTP/1.0" 202 Accepted 2026-04-02 01:13:20,346 - lava_callback - INFO - Starting async job submit for node 69cdc22823eb053795d42540 2026-04-02 01:13:20,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:13:20,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc22823eb053795d42540 HTTP/1.1" 200 None 2026-04-02 01:13:20,404 - lava_callback - DEBUG - Processing node 69cdc22823eb053795d42540: baseline-arm-pengutronix 2026-04-02 01:13:20,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:13:20,936 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:13:20,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc22823eb053795d42540/log.txt.gz 2026-04-02 01:13:21,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:13:21,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:13:21,461 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc22823eb053795d42540/lava-logs.yaml 2026-04-02 01:13:21,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:13:21,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:13:21,634 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc22823eb053795d42540/lava_callback.json.gz 2026-04-02 01:13:21,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:13:21,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc22823eb053795d42540 HTTP/1.1" 200 None 2026-04-02 01:13:21,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:13:21,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:13:21,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:13:21,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc22823eb053795d42540 HTTP/1.1" 200 None 2026-04-02 01:13:21,863 - lava_callback - INFO - Completed processing callback for node 69cdc22823eb053795d42540 INFO: 172.20.0.1:37658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54974 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:13:38,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:13:38,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55456 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:14:06,655 - lava_callback - INFO - Received callback for node 69cdc15523eb053795d423b4 INFO: 172.20.0.1:55462 - "POST /node/69cdc15523eb053795d423b4 HTTP/1.0" 202 Accepted 2026-04-02 01:14:06,665 - lava_callback - INFO - Starting async job submit for node 69cdc15523eb053795d423b4 2026-04-02 01:14:06,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:06,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15523eb053795d423b4 HTTP/1.1" 200 None 2026-04-02 01:14:06,730 - lava_callback - DEBUG - Processing node 69cdc15523eb053795d423b4: baseline-arm64 2026-04-02 01:14:07,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:14:07,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:14:07,967 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15523eb053795d423b4/log.txt.gz 2026-04-02 01:14:08,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:14:09,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:14:09,339 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15523eb053795d423b4/lava-logs.yaml 2026-04-02 01:14:09,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:14:10,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:14:10,367 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15523eb053795d423b4/lava_callback.json.gz 2026-04-02 01:14:10,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:10,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15523eb053795d423b4 HTTP/1.1" 200 None 2026-04-02 01:14:10,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:10,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:14:10,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:10,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15523eb053795d423b4 HTTP/1.1" 200 None 2026-04-02 01:14:10,500 - lava_callback - INFO - Completed processing callback for node 69cdc15523eb053795d423b4 2026-04-02 01:14:10,944 - lava_callback - INFO - Received callback for node 69cdbc5923eb053795d420f7 INFO: 172.20.0.1:34642 - "POST /node/69cdbc5923eb053795d420f7 HTTP/1.0" 202 Accepted 2026-04-02 01:14:10,996 - lava_callback - INFO - Starting async job submit for node 69cdbc5923eb053795d420f7 2026-04-02 01:14:11,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:11,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5923eb053795d420f7 HTTP/1.1" 200 None 2026-04-02 01:14:11,071 - lava_callback - DEBUG - Processing node 69cdbc5923eb053795d420f7: ltp-pty 2026-04-02 01:14:11,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:14:11,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:14:11,585 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69cdbc5923eb053795d420f7/log.txt.gz 2026-04-02 01:14:11,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:14:11,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:14:11,890 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69cdbc5923eb053795d420f7/lava-logs.yaml 2026-04-02 01:14:11,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:14:12,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:14:12,670 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69cdbc5923eb053795d420f7/lava_callback.json.gz 2026-04-02 01:14:12,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:12,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc5923eb053795d420f7 HTTP/1.1" 200 None 2026-04-02 01:14:12,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:12,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:14:12,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:12,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc5923eb053795d420f7 HTTP/1.1" 200 None 2026-04-02 01:14:12,962 - lava_callback - INFO - Completed processing callback for node 69cdbc5923eb053795d420f7 INFO: 172.20.0.1:34646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34654 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:14:21,028 - lava_callback - INFO - Received callback for node 69cdc16523eb053795d423f8 INFO: 172.20.0.1:57782 - "POST /node/69cdc16523eb053795d423f8 HTTP/1.0" 202 Accepted 2026-04-02 01:14:21,037 - lava_callback - INFO - Starting async job submit for node 69cdc16523eb053795d423f8 2026-04-02 01:14:21,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:21,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16523eb053795d423f8 HTTP/1.1" 200 None 2026-04-02 01:14:21,091 - lava_callback - DEBUG - Processing node 69cdc16523eb053795d423f8: baseline-nfs-arm64 2026-04-02 01:14:21,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:14:22,293 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:14:22,295 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16523eb053795d423f8/log.txt.gz 2026-04-02 01:14:22,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:14:23,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:14:23,358 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16523eb053795d423f8/lava-logs.yaml 2026-04-02 01:14:23,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:14:23,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:14:23,515 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16523eb053795d423f8/lava_callback.json.gz 2026-04-02 01:14:23,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:23,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16523eb053795d423f8 HTTP/1.1" 200 None 2026-04-02 01:14:23,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:23,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:14:23,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:57788 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:14:23,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16523eb053795d423f8 HTTP/1.1" 200 None 2026-04-02 01:14:23,642 - lava_callback - INFO - Completed processing callback for node 69cdc16523eb053795d423f8 INFO: 172.20.0.1:57798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55298 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:14:38,091 - lava_callback - INFO - Received callback for node 69cb1feb8c06c2799bf57c04 INFO: 172.20.0.1:55304 - "POST /node/69cb1feb8c06c2799bf57c04 HTTP/1.0" 202 Accepted 2026-04-02 01:14:38,101 - lava_callback - INFO - Starting async job submit for node 69cb1feb8c06c2799bf57c04 2026-04-02 01:14:38,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:38,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1feb8c06c2799bf57c04 HTTP/1.1" 200 None 2026-04-02 01:14:38,120 - lava_callback - DEBUG - Processing node 69cb1feb8c06c2799bf57c04: baseline-arm64-baylibre 2026-04-02 01:14:38,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb1feb8c06c2799bf57c04 2026-04-02 01:14:38,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:38,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb1feb8c06c2799bf57c04 HTTP/1.1" 200 None 2026-04-02 01:14:38,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:38,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 01:14:38,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:38,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb1feb8c06c2799bf57c04 HTTP/1.1" 200 None 2026-04-02 01:14:38,206 - lava_callback - ERROR - Error processing callback for node 69cb1feb8c06c2799bf57c04: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:14:38,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:14:38,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49908 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:16:48,254 - lava_callback - INFO - Received callback for node 69cdbf4f23eb053795d422e3 INFO: 172.20.0.1:49918 - "POST /node/69cdbf4f23eb053795d422e3 HTTP/1.0" 202 Accepted 2026-04-02 01:16:48,260 - lava_callback - INFO - Starting async job submit for node 69cdbf4f23eb053795d422e3 2026-04-02 01:16:48,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:16:48,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbf4f23eb053795d422e3 HTTP/1.1" 200 None 2026-04-02 01:16:48,333 - lava_callback - DEBUG - Processing node 69cdbf4f23eb053795d422e3: ltp-smoke 2026-04-02 01:16:48,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:16:48,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:16:48,924 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdbf4f23eb053795d422e3/log.txt.gz 2026-04-02 01:16:49,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:16:49,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:16:49,441 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdbf4f23eb053795d422e3/lava-logs.yaml 2026-04-02 01:16:49,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:16:49,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:16:49,614 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdbf4f23eb053795d422e3/lava_callback.json.gz 2026-04-02 01:16:49,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:16:49,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbf4f23eb053795d422e3 HTTP/1.1" 200 None 2026-04-02 01:16:49,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:16:49,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5bf23eb053795d41ebd HTTP/1.1" 200 None 2026-04-02 01:16:49,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:16:49,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbf4f23eb053795d422e3 HTTP/1.1" 200 None 2026-04-02 01:16:49,941 - lava_callback - INFO - Completed processing callback for node 69cdbf4f23eb053795d422e3 INFO: 172.20.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50238 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:17:08,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:08,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42658 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:17:25,920 - lava_callback - INFO - Received callback for node 69cdc39623eb053795d4266b INFO: 172.20.0.1:42664 - "POST /node/69cdc39623eb053795d4266b HTTP/1.0" 202 Accepted 2026-04-02 01:17:25,926 - lava_callback - INFO - Starting async job submit for node 69cdc39623eb053795d4266b 2026-04-02 01:17:25,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:25,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc39623eb053795d4266b HTTP/1.1" 200 None 2026-04-02 01:17:25,943 - lava_callback - DEBUG - Processing node 69cdc39623eb053795d4266b: baseline-nfs-arm64 2026-04-02 01:17:25,944 - lava_callback - WARNING - Failed to upload log 2026-04-02 01:17:25,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:17:26,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:17:26,103 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc39623eb053795d4266b/lava-logs.yaml 2026-04-02 01:17:26,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:17:26,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:17:26,229 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc39623eb053795d4266b/lava_callback.json.gz 2026-04-02 01:17:26,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:26,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc39623eb053795d4266b HTTP/1.1" 200 None 2026-04-02 01:17:26,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:26,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:17:26,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:26,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc39623eb053795d4266b HTTP/1.1" 200 None 2026-04-02 01:17:26,291 - lava_callback - INFO - Completed processing callback for node 69cdc39623eb053795d4266b INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:17:31,969 - lava_callback - INFO - Received callback for node 69cb209b8c06c2799bf57c5b INFO: 172.20.0.1:51514 - "POST /node/69cb209b8c06c2799bf57c5b HTTP/1.0" 202 Accepted 2026-04-02 01:17:31,974 - lava_callback - INFO - Starting async job submit for node 69cb209b8c06c2799bf57c5b 2026-04-02 01:17:31,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:31,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb209b8c06c2799bf57c5b HTTP/1.1" 200 None 2026-04-02 01:17:31,990 - lava_callback - DEBUG - Processing node 69cb209b8c06c2799bf57c5b: baseline-arm64-clabbe 2026-04-02 01:17:31,990 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb209b8c06c2799bf57c5b 2026-04-02 01:17:31,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:32,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb209b8c06c2799bf57c5b HTTP/1.1" 200 None 2026-04-02 01:17:32,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:32,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 01:17:32,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:32,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb209b8c06c2799bf57c5b HTTP/1.1" 200 None 2026-04-02 01:17:32,056 - lava_callback - ERROR - Error processing callback for node 69cb209b8c06c2799bf57c5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51516 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:17:38,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:38,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47500 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:17:53,881 - lava_callback - INFO - Received callback for node 69cdc16323eb053795d423d5 INFO: 172.20.0.1:43108 - "POST /node/69cdc16323eb053795d423d5 HTTP/1.0" 202 Accepted 2026-04-02 01:17:53,888 - lava_callback - INFO - Starting async job submit for node 69cdc16323eb053795d423d5 2026-04-02 01:17:53,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:53,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16323eb053795d423d5 HTTP/1.1" 200 None 2026-04-02 01:17:53,950 - lava_callback - DEBUG - Processing node 69cdc16323eb053795d423d5: baseline-arm64 2026-04-02 01:17:54,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:17:54,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:17:54,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc16323eb053795d423d5/log.txt.gz 2026-04-02 01:17:54,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:17:54,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:17:54,874 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc16323eb053795d423d5/lava-logs.yaml 2026-04-02 01:17:54,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:17:55,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:17:55,047 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc16323eb053795d423d5/lava_callback.json.gz DEBUG: Setup failed for 69cdc16323eb053795d423d5. Transitting job node result: incomplete -> fail 2026-04-02 01:17:55,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:55,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16323eb053795d423d5 HTTP/1.1" 200 None 2026-04-02 01:17:55,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:55,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:17:55,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:17:55,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16323eb053795d423d5 HTTP/1.1" 200 None 2026-04-02 01:17:55,266 - lava_callback - INFO - Completed processing callback for node 69cdc16323eb053795d423d5 INFO: 172.20.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36360 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:18:08,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:18:08,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60528 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:19:32,032 - lava_callback - INFO - Received callback for node 69cb21138c06c2799bf57c78 INFO: 172.20.0.1:39270 - "POST /node/69cb21138c06c2799bf57c78 HTTP/1.0" 202 Accepted 2026-04-02 01:19:32,038 - lava_callback - INFO - Starting async job submit for node 69cb21138c06c2799bf57c78 2026-04-02 01:19:32,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:19:32,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb21138c06c2799bf57c78 HTTP/1.1" 200 None 2026-04-02 01:19:32,055 - lava_callback - DEBUG - Processing node 69cb21138c06c2799bf57c78: baseline-arm64-clabbe 2026-04-02 01:19:32,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb21138c06c2799bf57c78 2026-04-02 01:19:32,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:19:32,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb21138c06c2799bf57c78 HTTP/1.1" 200 None 2026-04-02 01:19:32,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:19:32,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 01:19:32,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:19:32,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb21138c06c2799bf57c78 HTTP/1.1" 200 None 2026-04-02 01:19:32,131 - lava_callback - ERROR - Error processing callback for node 69cb21138c06c2799bf57c78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40938 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:20:52,140 - lava_callback - INFO - Received callback for node 69cb21628c06c2799bf57c98 INFO: 172.20.0.1:58374 - "POST /node/69cb21628c06c2799bf57c98 HTTP/1.0" 202 Accepted 2026-04-02 01:20:52,149 - lava_callback - INFO - Starting async job submit for node 69cb21628c06c2799bf57c98 2026-04-02 01:20:52,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:20:52,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb21628c06c2799bf57c98 HTTP/1.1" 200 None 2026-04-02 01:20:52,166 - lava_callback - DEBUG - Processing node 69cb21628c06c2799bf57c98: baseline-arm64-clabbe 2026-04-02 01:20:52,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb21628c06c2799bf57c98 2026-04-02 01:20:52,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:20:52,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb21628c06c2799bf57c98 HTTP/1.1" 200 None 2026-04-02 01:20:52,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:20:52,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 01:20:52,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:20:52,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb21628c06c2799bf57c98 HTTP/1.1" 200 None 2026-04-02 01:20:52,225 - lava_callback - ERROR - Error processing callback for node 69cb21628c06c2799bf57c98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36124 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:21:52,718 - lava_callback - INFO - Received callback for node 69cdc46e23eb053795d42698 INFO: 172.20.0.1:54992 - "POST /node/69cdc46e23eb053795d42698 HTTP/1.0" 202 Accepted 2026-04-02 01:21:52,725 - lava_callback - INFO - Starting async job submit for node 69cdc46e23eb053795d42698 2026-04-02 01:21:52,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:21:52,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc46e23eb053795d42698 HTTP/1.1" 200 None 2026-04-02 01:21:52,792 - lava_callback - DEBUG - Processing node 69cdc46e23eb053795d42698: baseline-arm64 2026-04-02 01:21:52,974 - lava_callback - WARNING - Failed to upload log 2026-04-02 01:21:53,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:21:53,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:21:53,227 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc46e23eb053795d42698/lava-logs.yaml 2026-04-02 01:21:53,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:21:53,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:21:53,351 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc46e23eb053795d42698/lava_callback.json.gz 2026-04-02 01:21:53,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:21:53,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc46e23eb053795d42698 HTTP/1.1" 200 None 2026-04-02 01:21:53,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:21:53,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:21:53,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:21:53,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc46e23eb053795d42698 HTTP/1.1" 200 None 2026-04-02 01:21:53,501 - lava_callback - INFO - Completed processing callback for node 69cdc46e23eb053795d42698 INFO: 172.20.0.1:55002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56652 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:22:08,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:22:08,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:22:28,877 - lava_callback - INFO - Received callback for node 69cdc42223eb053795d4268c INFO: 172.20.0.1:47856 - "POST /node/69cdc42223eb053795d4268c HTTP/1.0" 202 Accepted 2026-04-02 01:22:28,885 - lava_callback - INFO - Starting async job submit for node 69cdc42223eb053795d4268c 2026-04-02 01:22:28,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:22:28,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc42223eb053795d4268c HTTP/1.1" 200 None 2026-04-02 01:22:28,952 - lava_callback - DEBUG - Processing node 69cdc42223eb053795d4268c: baseline-arm64 2026-04-02 01:22:29,181 - lava_callback - WARNING - Failed to upload log 2026-04-02 01:22:29,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:22:29,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:22:29,538 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc42223eb053795d4268c/lava-logs.yaml 2026-04-02 01:22:29,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:22:29,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:22:29,689 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc42223eb053795d4268c/lava_callback.json.gz 2026-04-02 01:22:29,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:22:29,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc42223eb053795d4268c HTTP/1.1" 200 None 2026-04-02 01:22:29,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:22:29,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:22:29,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:22:29,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc42223eb053795d4268c HTTP/1.1" 200 None 2026-04-02 01:22:29,885 - lava_callback - INFO - Completed processing callback for node 69cdc42223eb053795d4268c INFO: 172.20.0.1:49594 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:22:38,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:22:38,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33900 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:24:55,426 - lava_callback - INFO - Received callback for node 69cdc51923eb053795d426b9 INFO: 172.20.0.1:39730 - "POST /node/69cdc51923eb053795d426b9 HTTP/1.0" 202 Accepted 2026-04-02 01:24:55,438 - lava_callback - INFO - Starting async job submit for node 69cdc51923eb053795d426b9 2026-04-02 01:24:55,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:24:55,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc51923eb053795d426b9 HTTP/1.1" 200 None 2026-04-02 01:24:55,498 - lava_callback - DEBUG - Processing node 69cdc51923eb053795d426b9: baseline-arm64-foundriesio 2026-04-02 01:24:56,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:24:56,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:24:56,290 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cdc51923eb053795d426b9/log.txt.gz INFO: 172.20.0.1:39744 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:24:56,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:24:56,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:24:56,792 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cdc51923eb053795d426b9/lava-logs.yaml 2026-04-02 01:24:56,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:24:57,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:24:57,158 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cdc51923eb053795d426b9/lava_callback.json.gz 2026-04-02 01:24:57,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:24:57,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc51923eb053795d426b9 HTTP/1.1" 200 None 2026-04-02 01:24:57,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:24:57,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:24:57,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:24:57,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc51923eb053795d426b9 HTTP/1.1" 200 None 2026-04-02 01:24:57,256 - lava_callback - INFO - Completed processing callback for node 69cdc51923eb053795d426b9 INFO: 172.20.0.1:60940 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:25:08,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:25:08,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42718 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:26:12,224 - lava_callback - INFO - Received callback for node 69cb22a78c06c2799bf57cd5 INFO: 172.20.0.1:50872 - "POST /node/69cb22a78c06c2799bf57cd5 HTTP/1.0" 202 Accepted 2026-04-02 01:26:12,229 - lava_callback - INFO - Starting async job submit for node 69cb22a78c06c2799bf57cd5 2026-04-02 01:26:12,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:26:12,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb22a78c06c2799bf57cd5 HTTP/1.1" 200 None 2026-04-02 01:26:12,246 - lava_callback - DEBUG - Processing node 69cb22a78c06c2799bf57cd5: baseline-arm64-clabbe 2026-04-02 01:26:12,246 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb22a78c06c2799bf57cd5 2026-04-02 01:26:12,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:26:12,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb22a78c06c2799bf57cd5 HTTP/1.1" 200 None 2026-04-02 01:26:12,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:26:12,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 01:26:12,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:26:12,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb22a78c06c2799bf57cd5 HTTP/1.1" 200 None 2026-04-02 01:26:12,309 - lava_callback - ERROR - Error processing callback for node 69cb22a78c06c2799bf57cd5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44492 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:28:52,045 - lava_callback - INFO - Received callback for node 69cdc15e23eb053795d423c9 INFO: 172.20.0.1:38104 - "POST /node/69cdc15e23eb053795d423c9 HTTP/1.0" 202 Accepted 2026-04-02 01:28:52,052 - lava_callback - INFO - Starting async job submit for node 69cdc15e23eb053795d423c9 2026-04-02 01:28:52,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:28:52,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15e23eb053795d423c9 HTTP/1.1" 200 None 2026-04-02 01:28:52,107 - lava_callback - DEBUG - Processing node 69cdc15e23eb053795d423c9: baseline-arm64 2026-04-02 01:28:52,412 - lava_callback - INFO - Received callback for node 69cb234a8c06c2799bf57cec INFO: 172.20.0.1:38112 - "POST /node/69cb234a8c06c2799bf57cec HTTP/1.0" 202 Accepted 2026-04-02 01:28:52,423 - lava_callback - INFO - Starting async job submit for node 69cb234a8c06c2799bf57cec 2026-04-02 01:28:52,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:28:52,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb234a8c06c2799bf57cec HTTP/1.1" 200 None 2026-04-02 01:28:52,533 - lava_callback - DEBUG - Processing node 69cb234a8c06c2799bf57cec: baseline-arm64-clabbe 2026-04-02 01:28:52,534 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb234a8c06c2799bf57cec 2026-04-02 01:28:52,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:28:52,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb234a8c06c2799bf57cec HTTP/1.1" 200 None 2026-04-02 01:28:52,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:28:52,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:28:52,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 01:28:52,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:28:52,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb234a8c06c2799bf57cec HTTP/1.1" 200 None 2026-04-02 01:28:52,696 - lava_callback - ERROR - Error processing callback for node 69cb234a8c06c2799bf57cec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 01:28:52,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:28:52,922 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15e23eb053795d423c9/log.txt.gz 2026-04-02 01:28:53,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:28:53,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:28:53,463 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15e23eb053795d423c9/lava-logs.yaml 2026-04-02 01:28:53,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:28:53,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:28:53,647 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc15e23eb053795d423c9/lava_callback.json.gz 2026-04-02 01:28:53,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:28:53,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc15e23eb053795d423c9 HTTP/1.1" 200 None 2026-04-02 01:28:53,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:28:53,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:28:53,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:28:53,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc15e23eb053795d423c9 HTTP/1.1" 200 None 2026-04-02 01:28:53,803 - lava_callback - INFO - Completed processing callback for node 69cdc15e23eb053795d423c9 INFO: 172.20.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41270 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:29:08,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:29:08,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56848 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:29:22,062 - lava_callback - INFO - Received callback for node 69cdc16e23eb053795d4240d INFO: 172.20.0.1:58082 - "POST /node/69cdc16e23eb053795d4240d HTTP/1.0" 202 Accepted 2026-04-02 01:29:22,069 - lava_callback - INFO - Starting async job submit for node 69cdc16e23eb053795d4240d 2026-04-02 01:29:22,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:29:22,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16e23eb053795d4240d HTTP/1.1" 200 None 2026-04-02 01:29:22,109 - lava_callback - DEBUG - Processing node 69cdc16e23eb053795d4240d: baseline-nfs-arm64 2026-04-02 01:29:22,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:29:22,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:29:22,797 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16e23eb053795d4240d/log.txt.gz 2026-04-02 01:29:23,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:29:23,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:29:23,305 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16e23eb053795d4240d/lava-logs.yaml 2026-04-02 01:29:23,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:29:23,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:29:23,466 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16e23eb053795d4240d/lava_callback.json.gz 2026-04-02 01:29:23,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:29:23,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16e23eb053795d4240d HTTP/1.1" 200 None 2026-04-02 01:29:23,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:29:23,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:29:23,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:29:23,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16e23eb053795d4240d HTTP/1.1" 200 None 2026-04-02 01:29:23,629 - lava_callback - INFO - Completed processing callback for node 69cdc16e23eb053795d4240d INFO: 172.20.0.1:58092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40056 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:29:36,684 - lava_callback - INFO - Received callback for node 69cdc2ab23eb053795d42616 INFO: 172.20.0.1:40068 - "POST /node/69cdc2ab23eb053795d42616 HTTP/1.0" 202 Accepted 2026-04-02 01:29:36,691 - lava_callback - INFO - Starting async job submit for node 69cdc2ab23eb053795d42616 2026-04-02 01:29:36,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:29:36,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc2ab23eb053795d42616 HTTP/1.1" 200 None 2026-04-02 01:29:36,769 - lava_callback - DEBUG - Processing node 69cdc2ab23eb053795d42616: baseline-arm 2026-04-02 01:29:37,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:29:37,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:29:37,513 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cdc2ab23eb053795d42616/log.txt.gz 2026-04-02 01:29:37,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:29:38,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:29:38,084 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdc2ab23eb053795d42616/lava-logs.yaml 2026-04-02 01:29:38,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:29:38,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:29:38,305 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdc2ab23eb053795d42616/lava_callback.json.gz DEBUG: Setup failed for 69cdc2ab23eb053795d42616. Transitting job node result: incomplete -> fail 2026-04-02 01:29:38,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:29:38,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc2ab23eb053795d42616 HTTP/1.1" 200 None 2026-04-02 01:29:38,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:29:38,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:29:38,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:29:38,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc2ab23eb053795d42616 HTTP/1.1" 200 None 2026-04-02 01:29:38,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:29:38,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 01:29:38,665 - lava_callback - INFO - Completed processing callback for node 69cdc2ab23eb053795d42616 INFO: 172.20.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51112 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:30:08,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:30:08,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55152 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:30:43,703 - lava_callback - INFO - Received callback for node 69cdc16823eb053795d423fe INFO: 172.20.0.1:54418 - "POST /node/69cdc16823eb053795d423fe HTTP/1.0" 202 Accepted 2026-04-02 01:30:43,712 - lava_callback - INFO - Starting async job submit for node 69cdc16823eb053795d423fe 2026-04-02 01:30:43,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:30:43,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16823eb053795d423fe HTTP/1.1" 200 None 2026-04-02 01:30:43,776 - lava_callback - DEBUG - Processing node 69cdc16823eb053795d423fe: baseline-nfs-arm64 2026-04-02 01:30:45,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:30:45,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:30:45,222 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16823eb053795d423fe/log.txt.gz 2026-04-02 01:30:45,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:30:46,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:30:46,099 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16823eb053795d423fe/lava-logs.yaml 2026-04-02 01:30:46,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:30:46,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:30:46,314 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc16823eb053795d423fe/lava_callback.json.gz DEBUG: Setup failed for 69cdc16823eb053795d423fe. Transitting job node result: incomplete -> fail 2026-04-02 01:30:46,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:30:46,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc16823eb053795d423fe HTTP/1.1" 200 None 2026-04-02 01:30:46,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:30:46,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:30:46,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:30:46,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc16823eb053795d423fe HTTP/1.1" 200 None INFO: 172.20.0.1:54426 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:30:46,544 - lava_callback - INFO - Completed processing callback for node 69cdc16823eb053795d423fe INFO: 172.20.0.1:40596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44162 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:31:08,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:31:08,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38808 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:32:12,529 - lava_callback - INFO - Received callback for node 69cb24188c06c2799bf57d22 INFO: 172.20.0.1:35534 - "POST /node/69cb24188c06c2799bf57d22 HTTP/1.0" 202 Accepted 2026-04-02 01:32:12,535 - lava_callback - INFO - Starting async job submit for node 69cb24188c06c2799bf57d22 2026-04-02 01:32:12,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:32:12,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb24188c06c2799bf57d22 HTTP/1.1" 200 None 2026-04-02 01:32:12,551 - lava_callback - DEBUG - Processing node 69cb24188c06c2799bf57d22: baseline-arm64-clabbe 2026-04-02 01:32:12,551 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb24188c06c2799bf57d22 2026-04-02 01:32:12,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:32:12,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb24188c06c2799bf57d22 HTTP/1.1" 200 None 2026-04-02 01:32:12,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:32:12,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 01:32:12,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:32:12,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb24188c06c2799bf57d22 HTTP/1.1" 200 None 2026-04-02 01:32:12,621 - lava_callback - ERROR - Error processing callback for node 69cb24188c06c2799bf57d22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58656 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:33:44,987 - lava_callback - INFO - Received callback for node 69cdc2da23eb053795d4262b INFO: 172.20.0.1:57628 - "POST /node/69cdc2da23eb053795d4262b HTTP/1.0" 202 Accepted 2026-04-02 01:33:45,033 - lava_callback - INFO - Starting async job submit for node 69cdc2da23eb053795d4262b 2026-04-02 01:33:45,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:33:45,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc2da23eb053795d4262b HTTP/1.1" 200 None 2026-04-02 01:33:45,073 - lava_callback - DEBUG - Processing node 69cdc2da23eb053795d4262b: baseline-arm-pengutronix 2026-04-02 01:33:45,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:33:45,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:33:45,313 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc2da23eb053795d4262b/log.txt.gz 2026-04-02 01:33:45,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:33:45,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:33:45,565 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc2da23eb053795d4262b/lava-logs.yaml 2026-04-02 01:33:45,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:33:45,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:33:45,712 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc2da23eb053795d4262b/lava_callback.json.gz DEBUG: Setup failed for 69cdc2da23eb053795d4262b. Transitting job node result: incomplete -> fail 2026-04-02 01:33:45,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:33:45,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc2da23eb053795d4262b HTTP/1.1" 200 None 2026-04-02 01:33:45,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:33:45,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:33:45,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:33:45,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc2da23eb053795d4262b HTTP/1.1" 200 None 2026-04-02 01:33:45,881 - lava_callback - INFO - Completed processing callback for node 69cdc2da23eb053795d4262b INFO: 172.20.0.1:57640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60880 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:34:08,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:34:08,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45124 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:40:40,831 - lava_callback - INFO - Received callback for node 69cdc89123eb053795d42ae3 INFO: 172.20.0.1:38660 - "POST /node/69cdc89123eb053795d42ae3 HTTP/1.0" 202 Accepted 2026-04-02 01:40:40,838 - lava_callback - INFO - Starting async job submit for node 69cdc89123eb053795d42ae3 2026-04-02 01:40:40,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:40:40,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc89123eb053795d42ae3 HTTP/1.1" 200 None 2026-04-02 01:40:40,878 - lava_callback - DEBUG - Processing node 69cdc89123eb053795d42ae3: baseline-arm64 2026-04-02 01:40:41,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:40:41,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:40:41,483 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc89123eb053795d42ae3/log.txt.gz 2026-04-02 01:40:41,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:40:41,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:40:41,962 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc89123eb053795d42ae3/lava-logs.yaml 2026-04-02 01:40:41,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:40:42,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:40:42,143 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdc89123eb053795d42ae3/lava_callback.json.gz 2026-04-02 01:40:42,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:40:42,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc89123eb053795d42ae3 HTTP/1.1" 200 None 2026-04-02 01:40:42,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:40:42,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:40:42,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:40:42,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc89123eb053795d42ae3 HTTP/1.1" 200 None 2026-04-02 01:40:42,309 - lava_callback - INFO - Completed processing callback for node 69cdc89123eb053795d42ae3 INFO: 172.20.0.1:38668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36266 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:41:08,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:41:08,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34078 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:41:39,661 - lava_callback - INFO - Received callback for node 69cb264a8c06c2799bf57d9e INFO: 172.20.0.1:34086 - "POST /node/69cb264a8c06c2799bf57d9e HTTP/1.0" 202 Accepted 2026-04-02 01:41:39,667 - lava_callback - INFO - Starting async job submit for node 69cb264a8c06c2799bf57d9e 2026-04-02 01:41:39,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:41:39,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb264a8c06c2799bf57d9e HTTP/1.1" 200 None 2026-04-02 01:41:39,686 - lava_callback - DEBUG - Processing node 69cb264a8c06c2799bf57d9e: baseline-arm64-baylibre 2026-04-02 01:41:39,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb264a8c06c2799bf57d9e 2026-04-02 01:41:39,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:41:39,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb264a8c06c2799bf57d9e HTTP/1.1" 200 None 2026-04-02 01:41:39,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:41:39,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 01:41:39,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:41:39,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb264a8c06c2799bf57d9e HTTP/1.1" 200 None 2026-04-02 01:41:39,738 - lava_callback - ERROR - Error processing callback for node 69cb264a8c06c2799bf57d9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58568 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:42:59,766 - lava_callback - INFO - Received callback for node 69cb269c8c06c2799bf57da6 INFO: 172.20.0.1:35636 - "POST /node/69cb269c8c06c2799bf57da6 HTTP/1.0" 202 Accepted 2026-04-02 01:42:59,772 - lava_callback - INFO - Starting async job submit for node 69cb269c8c06c2799bf57da6 2026-04-02 01:42:59,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:42:59,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb269c8c06c2799bf57da6 HTTP/1.1" 200 None 2026-04-02 01:42:59,791 - lava_callback - DEBUG - Processing node 69cb269c8c06c2799bf57da6: baseline-arm64-baylibre 2026-04-02 01:42:59,791 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb269c8c06c2799bf57da6 2026-04-02 01:42:59,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:42:59,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb269c8c06c2799bf57da6 HTTP/1.1" 200 None 2026-04-02 01:42:59,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:42:59,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 01:42:59,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:42:59,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb269c8c06c2799bf57da6 HTTP/1.1" 200 None 2026-04-02 01:42:59,845 - lava_callback - ERROR - Error processing callback for node 69cb269c8c06c2799bf57da6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55366 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:43:47,104 - lava_callback - INFO - Received callback for node 69cdc9c023eb053795d42b23 INFO: 172.20.0.1:55372 - "POST /node/69cdc9c023eb053795d42b23 HTTP/1.0" 202 Accepted 2026-04-02 01:43:47,110 - lava_callback - INFO - Starting async job submit for node 69cdc9c023eb053795d42b23 2026-04-02 01:43:47,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:43:47,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc9c023eb053795d42b23 HTTP/1.1" 200 None 2026-04-02 01:43:47,128 - lava_callback - DEBUG - Processing node 69cdc9c023eb053795d42b23: baseline-nfs-arm64 2026-04-02 01:43:47,130 - lava_callback - WARNING - Failed to upload log 2026-04-02 01:43:47,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:43:47,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:43:47,263 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc9c023eb053795d42b23/lava-logs.yaml 2026-04-02 01:43:47,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:43:47,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:43:47,386 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdc9c023eb053795d42b23/lava_callback.json.gz 2026-04-02 01:43:47,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:43:47,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc9c023eb053795d42b23 HTTP/1.1" 200 None 2026-04-02 01:43:47,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:43:47,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:43:47,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:43:47,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc9c023eb053795d42b23 HTTP/1.1" 200 None 2026-04-02 01:43:47,456 - lava_callback - INFO - Completed processing callback for node 69cdc9c023eb053795d42b23 INFO: 172.20.0.1:54322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57660 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:44:08,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:44:08,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48832 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:44:19,680 - lava_callback - INFO - Received callback for node 69cb26ee8c06c2799bf57db3 INFO: 172.20.0.1:48844 - "POST /node/69cb26ee8c06c2799bf57db3 HTTP/1.0" 202 Accepted 2026-04-02 01:44:19,686 - lava_callback - INFO - Starting async job submit for node 69cb26ee8c06c2799bf57db3 2026-04-02 01:44:19,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:44:19,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb26ee8c06c2799bf57db3 HTTP/1.1" 200 None 2026-04-02 01:44:19,702 - lava_callback - DEBUG - Processing node 69cb26ee8c06c2799bf57db3: baseline-arm64-baylibre 2026-04-02 01:44:19,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb26ee8c06c2799bf57db3 2026-04-02 01:44:19,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:44:19,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb26ee8c06c2799bf57db3 HTTP/1.1" 200 None 2026-04-02 01:44:19,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:44:19,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 01:44:19,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:44:19,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb26ee8c06c2799bf57db3 HTTP/1.1" 200 None 2026-04-02 01:44:19,756 - lava_callback - ERROR - Error processing callback for node 69cb26ee8c06c2799bf57db3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41740 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:44:52,490 - lava_callback - INFO - Received callback for node 69cdc91c23eb053795d42afe INFO: 172.20.0.1:46828 - "POST /node/69cdc91c23eb053795d42afe HTTP/1.0" 202 Accepted 2026-04-02 01:44:52,496 - lava_callback - INFO - Starting async job submit for node 69cdc91c23eb053795d42afe 2026-04-02 01:44:52,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:44:52,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc91c23eb053795d42afe HTTP/1.1" 200 None 2026-04-02 01:44:52,537 - lava_callback - DEBUG - Processing node 69cdc91c23eb053795d42afe: ltp-pty 2026-04-02 01:44:52,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:44:53,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:44:53,045 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69cdc91c23eb053795d42afe/log.txt.gz 2026-04-02 01:44:53,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:44:53,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:44:53,482 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69cdc91c23eb053795d42afe/lava-logs.yaml 2026-04-02 01:44:53,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:44:53,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:44:53,654 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69cdc91c23eb053795d42afe/lava_callback.json.gz 2026-04-02 01:44:53,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:44:53,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc91c23eb053795d42afe HTTP/1.1" 200 None 2026-04-02 01:44:53,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:44:53,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:44:53,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:44:53,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc91c23eb053795d42afe HTTP/1.1" 200 None 2026-04-02 01:44:53,757 - lava_callback - INFO - Completed processing callback for node 69cdc91c23eb053795d42afe INFO: 172.20.0.1:46836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58506 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:45:08,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:45:08,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37256 - "GET /node/69936aaecebe99dd8e0670dc%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-02 01:47:19,786 - lava_callback - INFO - Received callback for node 69cb27928c06c2799bf57dcb INFO: 172.20.0.1:47426 - "POST /node/69cb27928c06c2799bf57dcb HTTP/1.0" 202 Accepted 2026-04-02 01:47:19,792 - lava_callback - INFO - Starting async job submit for node 69cb27928c06c2799bf57dcb 2026-04-02 01:47:19,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:47:19,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb27928c06c2799bf57dcb HTTP/1.1" 200 None 2026-04-02 01:47:19,813 - lava_callback - DEBUG - Processing node 69cb27928c06c2799bf57dcb: baseline-arm64-baylibre 2026-04-02 01:47:19,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb27928c06c2799bf57dcb 2026-04-02 01:47:19,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:47:19,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb27928c06c2799bf57dcb HTTP/1.1" 200 None 2026-04-02 01:47:19,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:47:19,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 01:47:19,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:47:19,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb27928c06c2799bf57dcb HTTP/1.1" 200 None 2026-04-02 01:47:19,866 - lava_callback - ERROR - Error processing callback for node 69cb27928c06c2799bf57dcb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59524 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:47:46,157 - lava_callback - INFO - Received callback for node 69cdc8e323eb053795d42aed INFO: 172.20.0.1:34324 - "POST /node/69cdc8e323eb053795d42aed HTTP/1.0" 202 Accepted 2026-04-02 01:47:46,163 - lava_callback - INFO - Starting async job submit for node 69cdc8e323eb053795d42aed 2026-04-02 01:47:46,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:47:46,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc8e323eb053795d42aed HTTP/1.1" 200 None 2026-04-02 01:47:46,201 - lava_callback - DEBUG - Processing node 69cdc8e323eb053795d42aed: baseline-arm-pengutronix 2026-04-02 01:47:46,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:47:46,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:47:46,483 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc8e323eb053795d42aed/log.txt.gz INFO: 172.20.0.1:34332 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:47:46,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:47:46,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:47:46,785 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc8e323eb053795d42aed/lava-logs.yaml 2026-04-02 01:47:46,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:47:46,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:47:46,918 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdc8e323eb053795d42aed/lava_callback.json.gz 2026-04-02 01:47:46,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:47:46,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc8e323eb053795d42aed HTTP/1.1" 200 None 2026-04-02 01:47:46,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:47:46,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 01:47:46,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:47:47,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc8e323eb053795d42aed HTTP/1.1" 200 None 2026-04-02 01:47:47,020 - lava_callback - INFO - Completed processing callback for node 69cdc8e323eb053795d42aed INFO: 172.20.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51482 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:48:08,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:48:08,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33018 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:48:39,857 - lava_callback - INFO - Received callback for node 69cb27e58c06c2799bf57ddf INFO: 172.20.0.1:37880 - "POST /node/69cb27e58c06c2799bf57ddf HTTP/1.0" 202 Accepted 2026-04-02 01:48:39,863 - lava_callback - INFO - Starting async job submit for node 69cb27e58c06c2799bf57ddf 2026-04-02 01:48:39,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:48:39,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb27e58c06c2799bf57ddf HTTP/1.1" 200 None 2026-04-02 01:48:39,890 - lava_callback - DEBUG - Processing node 69cb27e58c06c2799bf57ddf: baseline-arm64-baylibre 2026-04-02 01:48:39,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb27e58c06c2799bf57ddf 2026-04-02 01:48:39,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:48:39,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb27e58c06c2799bf57ddf HTTP/1.1" 200 None 2026-04-02 01:48:39,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:48:39,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 01:48:39,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:48:39,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb27e58c06c2799bf57ddf HTTP/1.1" 200 None 2026-04-02 01:48:39,962 - lava_callback - ERROR - Error processing callback for node 69cb27e58c06c2799bf57ddf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54292 - "GET /node/69c24ed14a2c5046eb88002d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33918 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:49:59,981 - lava_callback - INFO - Received callback for node 69cb28368c06c2799bf57deb INFO: 172.20.0.1:34172 - "POST /node/69cb28368c06c2799bf57deb HTTP/1.0" 202 Accepted 2026-04-02 01:49:59,987 - lava_callback - INFO - Starting async job submit for node 69cb28368c06c2799bf57deb 2026-04-02 01:49:59,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:50:00,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb28368c06c2799bf57deb HTTP/1.1" 200 None 2026-04-02 01:50:00,002 - lava_callback - DEBUG - Processing node 69cb28368c06c2799bf57deb: baseline-arm64-baylibre 2026-04-02 01:50:00,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb28368c06c2799bf57deb 2026-04-02 01:50:00,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:50:00,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb28368c06c2799bf57deb HTTP/1.1" 200 None 2026-04-02 01:50:00,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:50:00,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 01:50:00,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:50:00,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb28368c06c2799bf57deb HTTP/1.1" 200 None 2026-04-02 01:50:00,061 - lava_callback - ERROR - Error processing callback for node 69cb28368c06c2799bf57deb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35920 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:50:42,798 - lava_callback - INFO - Received callback for node 69cdcb3d23eb053795d42b61 INFO: 172.20.0.1:45402 - "POST /node/69cdcb3d23eb053795d42b61 HTTP/1.0" 202 Accepted 2026-04-02 01:50:42,846 - lava_callback - INFO - Starting async job submit for node 69cdcb3d23eb053795d42b61 2026-04-02 01:50:42,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:50:42,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcb3d23eb053795d42b61 HTTP/1.1" 200 None 2026-04-02 01:50:42,894 - lava_callback - DEBUG - Processing node 69cdcb3d23eb053795d42b61: baseline-arm64 2026-04-02 01:50:43,018 - lava_callback - WARNING - Failed to upload log 2026-04-02 01:50:43,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:50:43,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:50:43,281 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcb3d23eb053795d42b61/lava-logs.yaml 2026-04-02 01:50:43,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:50:43,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:50:43,441 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcb3d23eb053795d42b61/lava_callback.json.gz 2026-04-02 01:50:43,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:50:43,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcb3d23eb053795d42b61 HTTP/1.1" 200 None 2026-04-02 01:50:43,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:50:43,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:50:43,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:50:43,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcb3d23eb053795d42b61 HTTP/1.1" 200 None 2026-04-02 01:50:43,618 - lava_callback - INFO - Completed processing callback for node 69cdcb3d23eb053795d42b61 INFO: 172.20.0.1:45406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58668 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:51:08,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:51:08,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54876 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:51:28,168 - lava_callback - INFO - Received callback for node 69cdca5423eb053795d42b3e INFO: 172.20.0.1:54884 - "POST /node/69cdca5423eb053795d42b3e HTTP/1.0" 202 Accepted 2026-04-02 01:51:28,175 - lava_callback - INFO - Starting async job submit for node 69cdca5423eb053795d42b3e 2026-04-02 01:51:28,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:51:28,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdca5423eb053795d42b3e HTTP/1.1" 200 None 2026-04-02 01:51:28,226 - lava_callback - DEBUG - Processing node 69cdca5423eb053795d42b3e: baseline-arm64 2026-04-02 01:51:28,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:51:28,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:51:28,756 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdca5423eb053795d42b3e/log.txt.gz 2026-04-02 01:51:28,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:51:29,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:51:29,189 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdca5423eb053795d42b3e/lava-logs.yaml 2026-04-02 01:51:29,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:51:29,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:51:29,365 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdca5423eb053795d42b3e/lava_callback.json.gz DEBUG: Setup failed for 69cdca5423eb053795d42b3e. Transitting job node result: incomplete -> fail 2026-04-02 01:51:29,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:51:29,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdca5423eb053795d42b3e HTTP/1.1" 200 None 2026-04-02 01:51:29,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:51:29,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:51:29,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:51:29,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdca5423eb053795d42b3e HTTP/1.1" 200 None 2026-04-02 01:51:29,563 - lava_callback - INFO - Completed processing callback for node 69cdca5423eb053795d42b3e INFO: 172.20.0.1:58366 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:51:38,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:51:38,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57118 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:52:06,056 - lava_callback - INFO - Received callback for node 69cdcb8923eb053795d42b6a INFO: 172.20.0.1:39140 - "POST /node/69cdcb8923eb053795d42b6a HTTP/1.0" 202 Accepted 2026-04-02 01:52:06,063 - lava_callback - INFO - Starting async job submit for node 69cdcb8923eb053795d42b6a 2026-04-02 01:52:06,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:52:06,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcb8923eb053795d42b6a HTTP/1.1" 200 None 2026-04-02 01:52:06,109 - lava_callback - DEBUG - Processing node 69cdcb8923eb053795d42b6a: baseline-arm64 2026-04-02 01:52:06,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:52:06,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:52:06,402 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcb8923eb053795d42b6a/log.txt.gz 2026-04-02 01:52:06,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39142 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:52:06,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:52:06,661 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcb8923eb053795d42b6a/lava-logs.yaml 2026-04-02 01:52:06,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 01:52:06,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 01:52:06,801 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcb8923eb053795d42b6a/lava_callback.json.gz 2026-04-02 01:52:06,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:52:06,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcb8923eb053795d42b6a HTTP/1.1" 200 None 2026-04-02 01:52:06,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:52:06,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 01:52:06,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:52:06,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcb8923eb053795d42b6a HTTP/1.1" 200 None 2026-04-02 01:52:06,988 - lava_callback - INFO - Completed processing callback for node 69cdcb8923eb053795d42b6a 2026-04-02 01:52:08,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:52:08,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44950 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:54:13,564 - lava_callback - INFO - Received callback for node 69cb29368c06c2799bf57e1a INFO: 172.20.0.1:54942 - "POST /node/69cb29368c06c2799bf57e1a HTTP/1.0" 202 Accepted 2026-04-02 01:54:13,572 - lava_callback - INFO - Starting async job submit for node 69cb29368c06c2799bf57e1a 2026-04-02 01:54:13,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:54:13,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb29368c06c2799bf57e1a HTTP/1.1" 200 None 2026-04-02 01:54:13,614 - lava_callback - DEBUG - Processing node 69cb29368c06c2799bf57e1a: baseline-arm64-clabbe 2026-04-02 01:54:13,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb29368c06c2799bf57e1a 2026-04-02 01:54:13,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:54:13,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb29368c06c2799bf57e1a HTTP/1.1" 200 None 2026-04-02 01:54:13,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:54:13,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 01:54:13,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:54:13,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb29368c06c2799bf57e1a HTTP/1.1" 200 None 2026-04-02 01:54:13,685 - lava_callback - ERROR - Error processing callback for node 69cb29368c06c2799bf57e1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43090 - "GET /node/6952f8d19912bd6bbcea60ff%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:43092 - "GET /node/698ee628f0e3ffbba67d2155%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:43108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43112 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:55:20,268 - lava_callback - INFO - Received callback for node 69cb29818c06c2799bf57e22 INFO: 172.20.0.1:60960 - "POST /node/69cb29818c06c2799bf57e22 HTTP/1.0" 202 Accepted 2026-04-02 01:55:20,276 - lava_callback - INFO - Starting async job submit for node 69cb29818c06c2799bf57e22 2026-04-02 01:55:20,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:55:20,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb29818c06c2799bf57e22 HTTP/1.1" 200 None 2026-04-02 01:55:20,291 - lava_callback - DEBUG - Processing node 69cb29818c06c2799bf57e22: baseline-arm64-baylibre 2026-04-02 01:55:20,292 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb29818c06c2799bf57e22 2026-04-02 01:55:20,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:55:20,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb29818c06c2799bf57e22 HTTP/1.1" 200 None 2026-04-02 01:55:20,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:55:20,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-02 01:55:20,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:55:20,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb29818c06c2799bf57e22 HTTP/1.1" 200 None 2026-04-02 01:55:20,345 - lava_callback - ERROR - Error processing callback for node 69cb29818c06c2799bf57e22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40424 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:56:53,627 - lava_callback - INFO - Received callback for node 69cb29dd8c06c2799bf57e2d INFO: 172.20.0.1:37864 - "POST /node/69cb29dd8c06c2799bf57e2d HTTP/1.0" 202 Accepted 2026-04-02 01:56:53,634 - lava_callback - INFO - Starting async job submit for node 69cb29dd8c06c2799bf57e2d 2026-04-02 01:56:53,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:56:53,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb29dd8c06c2799bf57e2d HTTP/1.1" 200 None 2026-04-02 01:56:53,650 - lava_callback - DEBUG - Processing node 69cb29dd8c06c2799bf57e2d: baseline-arm64-clabbe 2026-04-02 01:56:53,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb29dd8c06c2799bf57e2d 2026-04-02 01:56:53,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:56:53,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb29dd8c06c2799bf57e2d HTTP/1.1" 200 None 2026-04-02 01:56:53,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:56:53,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-02 01:56:53,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:56:53,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb29dd8c06c2799bf57e2d HTTP/1.1" 200 None 2026-04-02 01:56:53,724 - lava_callback - ERROR - Error processing callback for node 69cb29dd8c06c2799bf57e2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56034 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:58:13,721 - lava_callback - INFO - Received callback for node 69cb2a2d8c06c2799bf57e39 INFO: 172.20.0.1:52164 - "POST /node/69cb2a2d8c06c2799bf57e39 HTTP/1.0" 202 Accepted 2026-04-02 01:58:13,727 - lava_callback - INFO - Starting async job submit for node 69cb2a2d8c06c2799bf57e39 2026-04-02 01:58:13,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:58:13,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2a2d8c06c2799bf57e39 HTTP/1.1" 200 None 2026-04-02 01:58:13,743 - lava_callback - DEBUG - Processing node 69cb2a2d8c06c2799bf57e39: baseline-arm64-clabbe 2026-04-02 01:58:13,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb2a2d8c06c2799bf57e39 2026-04-02 01:58:13,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:58:13,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2a2d8c06c2799bf57e39 HTTP/1.1" 200 None 2026-04-02 01:58:13,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:58:13,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-02 01:58:13,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:58:13,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2a2d8c06c2799bf57e39 HTTP/1.1" 200 None 2026-04-02 01:58:13,803 - lava_callback - ERROR - Error processing callback for node 69cb2a2d8c06c2799bf57e39: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60718 - "GET / HTTP/1.0" 200 OK 2026-04-02 01:59:33,671 - lava_callback - INFO - Received callback for node 69cb2a808c06c2799bf57e45 INFO: 172.20.0.1:43468 - "POST /node/69cb2a808c06c2799bf57e45 HTTP/1.0" 202 Accepted 2026-04-02 01:59:33,677 - lava_callback - INFO - Starting async job submit for node 69cb2a808c06c2799bf57e45 2026-04-02 01:59:33,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:59:33,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2a808c06c2799bf57e45 HTTP/1.1" 200 None 2026-04-02 01:59:33,692 - lava_callback - DEBUG - Processing node 69cb2a808c06c2799bf57e45: baseline-arm64-clabbe 2026-04-02 01:59:33,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb2a808c06c2799bf57e45 2026-04-02 01:59:33,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:59:33,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2a808c06c2799bf57e45 HTTP/1.1" 200 None 2026-04-02 01:59:33,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:59:33,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-02 01:59:33,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 01:59:33,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2a808c06c2799bf57e45 HTTP/1.1" 200 None 2026-04-02 01:59:33,750 - lava_callback - ERROR - Error processing callback for node 69cb2a808c06c2799bf57e45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47044 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:01:12,146 - lava_callback - INFO - Received callback for node 69cdc9b523eb053795d42b1e INFO: 172.20.0.1:48324 - "POST /node/69cdc9b523eb053795d42b1e HTTP/1.0" 202 Accepted 2026-04-02 02:01:12,153 - lava_callback - INFO - Starting async job submit for node 69cdc9b523eb053795d42b1e 2026-04-02 02:01:12,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:01:12,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc9b523eb053795d42b1e HTTP/1.1" 200 None 2026-04-02 02:01:12,241 - lava_callback - DEBUG - Processing node 69cdc9b523eb053795d42b1e: ltp-smoke 2026-04-02 02:01:12,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:01:12,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:01:12,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdc9b523eb053795d42b1e/log.txt.gz 2026-04-02 02:01:13,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:01:13,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:01:13,633 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdc9b523eb053795d42b1e/lava-logs.yaml 2026-04-02 02:01:13,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:01:13,783 - lava_callback - INFO - Received callback for node 69cb2ad88c06c2799bf57e4f INFO: 172.20.0.1:48332 - "POST /node/69cb2ad88c06c2799bf57e4f HTTP/1.0" 202 Accepted 2026-04-02 02:01:13,789 - lava_callback - INFO - Starting async job submit for node 69cb2ad88c06c2799bf57e4f 2026-04-02 02:01:13,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:01:13,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2ad88c06c2799bf57e4f HTTP/1.1" 200 None 2026-04-02 02:01:13,805 - lava_callback - DEBUG - Processing node 69cb2ad88c06c2799bf57e4f: baseline-arm64-clabbe 2026-04-02 02:01:13,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb2ad88c06c2799bf57e4f 2026-04-02 02:01:13,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:01:13,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2ad88c06c2799bf57e4f HTTP/1.1" 200 None 2026-04-02 02:01:13,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:01:13,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-02 02:01:13,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:01:13,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2ad88c06c2799bf57e4f HTTP/1.1" 200 None 2026-04-02 02:01:13,863 - lava_callback - ERROR - Error processing callback for node 69cb2ad88c06c2799bf57e4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-02 02:01:13,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:01:13,873 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdc9b523eb053795d42b1e/lava_callback.json.gz 2026-04-02 02:01:14,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:01:14,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc9b523eb053795d42b1e HTTP/1.1" 200 None 2026-04-02 02:01:14,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:01:14,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5bf23eb053795d41ebd HTTP/1.1" 200 None 2026-04-02 02:01:14,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:01:14,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc9b523eb053795d42b1e HTTP/1.1" 200 None 2026-04-02 02:01:14,405 - lava_callback - INFO - Completed processing callback for node 69cdc9b523eb053795d42b1e INFO: 172.20.0.1:48338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42194 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:01:38,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:01:38,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37056 - "GET /node/693b36154b47c3f87266ce2c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37064 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:02:14,156 - lava_callback - INFO - Received callback for node 69cdcde923eb053795d42bc9 INFO: 172.20.0.1:57290 - "POST /node/69cdcde923eb053795d42bc9 HTTP/1.0" 202 Accepted 2026-04-02 02:02:14,163 - lava_callback - INFO - Starting async job submit for node 69cdcde923eb053795d42bc9 2026-04-02 02:02:14,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:02:14,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcde923eb053795d42bc9 HTTP/1.1" 200 None 2026-04-02 02:02:14,200 - lava_callback - DEBUG - Processing node 69cdcde923eb053795d42bc9: baseline-arm64 2026-04-02 02:02:14,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:02:14,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:02:14,764 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcde923eb053795d42bc9/log.txt.gz 2026-04-02 02:02:15,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:57298 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 02:02:15,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:02:15,266 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcde923eb053795d42bc9/lava-logs.yaml 2026-04-02 02:02:15,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:02:15,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:02:15,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcde923eb053795d42bc9/lava_callback.json.gz 2026-04-02 02:02:15,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:02:15,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcde923eb053795d42bc9 HTTP/1.1" 200 None 2026-04-02 02:02:15,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:02:15,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:02:15,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:02:15,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcde923eb053795d42bc9 HTTP/1.1" 200 None 2026-04-02 02:02:15,697 - lava_callback - INFO - Completed processing callback for node 69cdcde923eb053795d42bc9 INFO: 172.20.0.1:57308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40206 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:02:33,841 - lava_callback - INFO - Received callback for node 69cb2b298c06c2799bf57e55 INFO: 172.20.0.1:43020 - "POST /node/69cb2b298c06c2799bf57e55 HTTP/1.0" 202 Accepted 2026-04-02 02:02:33,847 - lava_callback - INFO - Starting async job submit for node 69cb2b298c06c2799bf57e55 2026-04-02 02:02:33,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:02:33,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2b298c06c2799bf57e55 HTTP/1.1" 200 None 2026-04-02 02:02:33,864 - lava_callback - DEBUG - Processing node 69cb2b298c06c2799bf57e55: baseline-arm64-clabbe 2026-04-02 02:02:33,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb2b298c06c2799bf57e55 2026-04-02 02:02:33,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:02:33,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2b298c06c2799bf57e55 HTTP/1.1" 200 None 2026-04-02 02:02:33,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:02:33,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 02:02:33,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:02:33,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2b298c06c2799bf57e55 HTTP/1.1" 200 None 2026-04-02 02:02:33,920 - lava_callback - ERROR - Error processing callback for node 69cb2b298c06c2799bf57e55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43024 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:02:38,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:02:38,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53446 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:03:54,179 - lava_callback - INFO - Received callback for node 69cb2b7c8c06c2799bf57e61 INFO: 172.20.0.1:53634 - "POST /node/69cb2b7c8c06c2799bf57e61 HTTP/1.0" 202 Accepted 2026-04-02 02:03:54,184 - lava_callback - INFO - Starting async job submit for node 69cb2b7c8c06c2799bf57e61 2026-04-02 02:03:54,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:03:54,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2b7c8c06c2799bf57e61 HTTP/1.1" 200 None 2026-04-02 02:03:54,205 - lava_callback - DEBUG - Processing node 69cb2b7c8c06c2799bf57e61: baseline-arm64-clabbe 2026-04-02 02:03:54,205 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb2b7c8c06c2799bf57e61 2026-04-02 02:03:54,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:03:54,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2b7c8c06c2799bf57e61 HTTP/1.1" 200 None 2026-04-02 02:03:54,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:03:54,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-02 02:03:54,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:03:54,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2b7c8c06c2799bf57e61 HTTP/1.1" 200 None 2026-04-02 02:03:54,256 - lava_callback - ERROR - Error processing callback for node 69cb2b7c8c06c2799bf57e61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51676 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:04:07,575 - lava_callback - INFO - Received callback for node 69cdce8423eb053795d42beb INFO: 172.20.0.1:51686 - "POST /node/69cdce8423eb053795d42beb HTTP/1.0" 202 Accepted 2026-04-02 02:04:07,580 - lava_callback - INFO - Starting async job submit for node 69cdce8423eb053795d42beb 2026-04-02 02:04:07,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:04:07,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdce8423eb053795d42beb HTTP/1.1" 200 None 2026-04-02 02:04:07,603 - lava_callback - DEBUG - Processing node 69cdce8423eb053795d42beb: baseline-nfs-arm64 2026-04-02 02:04:07,605 - lava_callback - WARNING - Failed to upload log 2026-04-02 02:04:07,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:04:07,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:04:07,762 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdce8423eb053795d42beb/lava-logs.yaml 2026-04-02 02:04:07,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:04:07,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:04:07,929 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdce8423eb053795d42beb/lava_callback.json.gz 2026-04-02 02:04:07,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:04:07,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdce8423eb053795d42beb HTTP/1.1" 200 None 2026-04-02 02:04:07,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:04:07,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:04:07,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:04:07,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdce8423eb053795d42beb HTTP/1.1" 200 None 2026-04-02 02:04:07,985 - lava_callback - INFO - Completed processing callback for node 69cdce8423eb053795d42beb 2026-04-02 02:04:08,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:04:08,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57042 - "GET /node/69935083523507f3806a6941%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-02 02:05:04,805 - lava_callback - INFO - Received callback for node 69cdcc6f23eb053795d42b8e INFO: 172.20.0.1:57046 - "POST /node/69cdcc6f23eb053795d42b8e HTTP/1.0" 202 Accepted 2026-04-02 02:05:04,812 - lava_callback - INFO - Starting async job submit for node 69cdcc6f23eb053795d42b8e 2026-04-02 02:05:04,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:04,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcc6f23eb053795d42b8e HTTP/1.1" 200 None 2026-04-02 02:05:04,859 - lava_callback - DEBUG - Processing node 69cdcc6f23eb053795d42b8e: baseline-arm64 2026-04-02 02:05:05,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:05:05,537 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:05:05,539 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcc6f23eb053795d42b8e/log.txt.gz 2026-04-02 02:05:05,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:05:06,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:05:06,028 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcc6f23eb053795d42b8e/lava-logs.yaml 2026-04-02 02:05:06,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:05:06,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:05:06,210 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcc6f23eb053795d42b8e/lava_callback.json.gz 2026-04-02 02:05:06,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:06,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcc6f23eb053795d42b8e HTTP/1.1" 200 None 2026-04-02 02:05:06,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:06,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:05:06,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:06,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcc6f23eb053795d42b8e HTTP/1.1" 200 None 2026-04-02 02:05:06,334 - lava_callback - INFO - Completed processing callback for node 69cdcc6f23eb053795d42b8e INFO: 172.20.0.1:57050 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:05:08,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:08,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-02 02:05:13,913 - lava_callback - INFO - Received callback for node 69cb2bce8c06c2799bf57e6d INFO: 172.20.0.1:40300 - "POST /node/69cb2bce8c06c2799bf57e6d HTTP/1.0" 202 Accepted 2026-04-02 02:05:13,919 - lava_callback - INFO - Starting async job submit for node 69cb2bce8c06c2799bf57e6d 2026-04-02 02:05:13,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:13,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2bce8c06c2799bf57e6d HTTP/1.1" 200 None 2026-04-02 02:05:13,935 - lava_callback - DEBUG - Processing node 69cb2bce8c06c2799bf57e6d: baseline-arm64-clabbe 2026-04-02 02:05:13,935 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb2bce8c06c2799bf57e6d 2026-04-02 02:05:13,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:13,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2bce8c06c2799bf57e6d HTTP/1.1" 200 None 2026-04-02 02:05:13,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:13,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-02 02:05:13,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:13,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2bce8c06c2799bf57e6d HTTP/1.1" 200 None 2026-04-02 02:05:14,000 - lava_callback - ERROR - Error processing callback for node 69cb2bce8c06c2799bf57e6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45906 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:05:43,439 - lava_callback - INFO - Received callback for node 69cdcec923eb053795d42bf8 INFO: 172.20.0.1:47748 - "POST /node/69cdcec923eb053795d42bf8 HTTP/1.0" 202 Accepted 2026-04-02 02:05:43,445 - lava_callback - INFO - Starting async job submit for node 69cdcec923eb053795d42bf8 2026-04-02 02:05:43,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:43,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcec923eb053795d42bf8 HTTP/1.1" 200 None 2026-04-02 02:05:43,489 - lava_callback - DEBUG - Processing node 69cdcec923eb053795d42bf8: baseline-arm64 2026-04-02 02:05:43,621 - lava_callback - WARNING - Failed to upload log 2026-04-02 02:05:43,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:05:43,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:05:43,903 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcec923eb053795d42bf8/lava-logs.yaml 2026-04-02 02:05:43,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:05:44,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:05:44,046 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcec923eb053795d42bf8/lava_callback.json.gz 2026-04-02 02:05:44,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:44,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcec923eb053795d42bf8 HTTP/1.1" 200 None 2026-04-02 02:05:44,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:44,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:05:44,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:05:44,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcec923eb053795d42bf8 HTTP/1.1" 200 None 2026-04-02 02:05:44,214 - lava_callback - INFO - Completed processing callback for node 69cdcec923eb053795d42bf8 INFO: 172.20.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55946 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:06:08,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:06:08,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56234 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:08:32,857 - lava_callback - INFO - Received callback for node 69cdcf6423eb053795d42c0e INFO: 172.20.0.1:44432 - "POST /node/69cdcf6423eb053795d42c0e HTTP/1.0" 202 Accepted 2026-04-02 02:08:32,864 - lava_callback - INFO - Starting async job submit for node 69cdcf6423eb053795d42c0e 2026-04-02 02:08:32,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:08:32,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcf6423eb053795d42c0e HTTP/1.1" 200 None 2026-04-02 02:08:32,908 - lava_callback - DEBUG - Processing node 69cdcf6423eb053795d42c0e: baseline-arm64 2026-04-02 02:08:33,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:08:33,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:08:33,210 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcf6423eb053795d42c0e/log.txt.gz 2026-04-02 02:08:33,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:08:33,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:08:33,465 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcf6423eb053795d42c0e/lava-logs.yaml 2026-04-02 02:08:33,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:08:33,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:08:33,590 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcf6423eb053795d42c0e/lava_callback.json.gz 2026-04-02 02:08:33,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:08:33,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcf6423eb053795d42c0e HTTP/1.1" 200 None 2026-04-02 02:08:33,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:08:33,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:08:33,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:08:33,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcf6423eb053795d42c0e HTTP/1.1" 200 None 2026-04-02 02:08:33,779 - lava_callback - INFO - Completed processing callback for node 69cdcf6423eb053795d42c0e INFO: 172.20.0.1:44434 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:08:38,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:08:39,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44836 - "GET /node/69ab7623efa461f55e59d452%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55980 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 02:10:15,202 - lava_callback - INFO - Received callback for node 69cdccbf23eb053795d42b97 INFO: 172.20.0.1:55988 - "POST /node/69cdccbf23eb053795d42b97 HTTP/1.0" 202 Accepted 2026-04-02 02:10:15,209 - lava_callback - INFO - Starting async job submit for node 69cdccbf23eb053795d42b97 2026-04-02 02:10:15,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:10:15,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdccbf23eb053795d42b97 HTTP/1.1" 200 None 2026-04-02 02:10:15,253 - lava_callback - DEBUG - Processing node 69cdccbf23eb053795d42b97: baseline-nfs-arm64 2026-04-02 02:10:15,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:10:15,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:10:15,978 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdccbf23eb053795d42b97/log.txt.gz 2026-04-02 02:10:16,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:56002 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:10:16,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:10:16,527 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdccbf23eb053795d42b97/lava-logs.yaml 2026-04-02 02:10:16,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:10:16,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:10:16,699 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdccbf23eb053795d42b97/lava_callback.json.gz 2026-04-02 02:10:16,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:10:16,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdccbf23eb053795d42b97 HTTP/1.1" 200 None 2026-04-02 02:10:16,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:10:16,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:10:16,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:10:16,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdccbf23eb053795d42b97 HTTP/1.1" 200 None 2026-04-02 02:10:16,836 - lava_callback - INFO - Completed processing callback for node 69cdccbf23eb053795d42b97 INFO: 172.20.0.1:45756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45758 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:10:34,105 - lava_callback - INFO - Received callback for node 69cb2d118c06c2799bf57e9a INFO: 172.20.0.1:56786 - "POST /node/69cb2d118c06c2799bf57e9a HTTP/1.0" 202 Accepted 2026-04-02 02:10:34,111 - lava_callback - INFO - Starting async job submit for node 69cb2d118c06c2799bf57e9a 2026-04-02 02:10:34,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:10:34,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2d118c06c2799bf57e9a HTTP/1.1" 200 None 2026-04-02 02:10:34,128 - lava_callback - DEBUG - Processing node 69cb2d118c06c2799bf57e9a: baseline-arm64-clabbe 2026-04-02 02:10:34,128 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb2d118c06c2799bf57e9a 2026-04-02 02:10:34,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:10:34,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2d118c06c2799bf57e9a HTTP/1.1" 200 None 2026-04-02 02:10:34,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:10:34,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 02:10:34,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:10:34,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2d118c06c2799bf57e9a HTTP/1.1" 200 None 2026-04-02 02:10:34,184 - lava_callback - ERROR - Error processing callback for node 69cb2d118c06c2799bf57e9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56790 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:10:39,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:10:39,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53786 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:13:14,308 - lava_callback - INFO - Received callback for node 69cb2daf8c06c2799bf57ebb INFO: 172.20.0.1:60380 - "POST /node/69cb2daf8c06c2799bf57ebb HTTP/1.0" 202 Accepted 2026-04-02 02:13:14,314 - lava_callback - INFO - Starting async job submit for node 69cb2daf8c06c2799bf57ebb 2026-04-02 02:13:14,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:14,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2daf8c06c2799bf57ebb HTTP/1.1" 200 None 2026-04-02 02:13:14,337 - lava_callback - DEBUG - Processing node 69cb2daf8c06c2799bf57ebb: baseline-arm64-clabbe 2026-04-02 02:13:14,338 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb2daf8c06c2799bf57ebb 2026-04-02 02:13:14,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:14,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2daf8c06c2799bf57ebb HTTP/1.1" 200 None 2026-04-02 02:13:14,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:14,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 02:13:14,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:14,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2daf8c06c2799bf57ebb HTTP/1.1" 200 None 2026-04-02 02:13:14,404 - lava_callback - ERROR - Error processing callback for node 69cb2daf8c06c2799bf57ebb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55364 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:13:40,996 - lava_callback - INFO - Received callback for node 69cdcd0123eb053795d42ba2 INFO: 172.20.0.1:45200 - "POST /node/69cdcd0123eb053795d42ba2 HTTP/1.0" 202 Accepted 2026-04-02 02:13:41,007 - lava_callback - INFO - Starting async job submit for node 69cdcd0123eb053795d42ba2 2026-04-02 02:13:41,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:41,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcd0123eb053795d42ba2 HTTP/1.1" 200 None 2026-04-02 02:13:41,116 - lava_callback - DEBUG - Processing node 69cdcd0123eb053795d42ba2: baseline-arm 2026-04-02 02:13:41,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:13:41,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:13:41,814 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cdcd0123eb053795d42ba2/log.txt.gz 2026-04-02 02:13:42,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:13:42,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:13:42,452 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdcd0123eb053795d42ba2/lava-logs.yaml 2026-04-02 02:13:42,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:13:42,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:13:42,638 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdcd0123eb053795d42ba2/lava_callback.json.gz DEBUG: Setup failed for 69cdcd0123eb053795d42ba2. Transitting job node result: incomplete -> fail 2026-04-02 02:13:42,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:42,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcd0123eb053795d42ba2 HTTP/1.1" 200 None 2026-04-02 02:13:42,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:42,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 02:13:42,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:42,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcd0123eb053795d42ba2 HTTP/1.1" 200 None 2026-04-02 02:13:42,959 - lava_callback - INFO - Completed processing callback for node 69cdcd0123eb053795d42ba2 INFO: 172.20.0.1:45210 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:13:50,742 - lava_callback - INFO - Received callback for node 69cdcf1e23eb053795d42c08 INFO: 172.20.0.1:43630 - "POST /node/69cdcf1e23eb053795d42c08 HTTP/1.0" 202 Accepted 2026-04-02 02:13:50,750 - lava_callback - INFO - Starting async job submit for node 69cdcf1e23eb053795d42c08 2026-04-02 02:13:50,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:50,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcf1e23eb053795d42c08 HTTP/1.1" 200 None 2026-04-02 02:13:50,804 - lava_callback - DEBUG - Processing node 69cdcf1e23eb053795d42c08: baseline-arm64 2026-04-02 02:13:51,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:13:51,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:13:51,391 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcf1e23eb053795d42c08/log.txt.gz 2026-04-02 02:13:51,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:13:51,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:13:51,845 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcf1e23eb053795d42c08/lava-logs.yaml 2026-04-02 02:13:51,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:13:52,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:13:52,015 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdcf1e23eb053795d42c08/lava_callback.json.gz DEBUG: Setup failed for 69cdcf1e23eb053795d42c08. Transitting job node result: incomplete -> fail 2026-04-02 02:13:52,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:52,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcf1e23eb053795d42c08 HTTP/1.1" 200 None 2026-04-02 02:13:52,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:52,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:13:52,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:13:52,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcf1e23eb053795d42c08 HTTP/1.1" 200 None 2026-04-02 02:13:52,224 - lava_callback - INFO - Completed processing callback for node 69cdcf1e23eb053795d42c08 INFO: 172.20.0.1:43644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47130 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:14:09,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:14:09,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57102 - "GET /node/6919928740db568ee36c47dc%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-02 02:14:34,347 - lava_callback - INFO - Received callback for node 69cb2dff8c06c2799bf57ec0 INFO: 172.20.0.1:57110 - "POST /node/69cb2dff8c06c2799bf57ec0 HTTP/1.0" 202 Accepted 2026-04-02 02:14:34,353 - lava_callback - INFO - Starting async job submit for node 69cb2dff8c06c2799bf57ec0 2026-04-02 02:14:34,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:14:34,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2dff8c06c2799bf57ec0 HTTP/1.1" 200 None 2026-04-02 02:14:34,375 - lava_callback - DEBUG - Processing node 69cb2dff8c06c2799bf57ec0: baseline-arm64-clabbe 2026-04-02 02:14:34,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb2dff8c06c2799bf57ec0 2026-04-02 02:14:34,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:14:34,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2dff8c06c2799bf57ec0 HTTP/1.1" 200 None 2026-04-02 02:14:34,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:14:34,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 02:14:34,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:14:34,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2dff8c06c2799bf57ec0 HTTP/1.1" 200 None 2026-04-02 02:14:34,451 - lava_callback - ERROR - Error processing callback for node 69cb2dff8c06c2799bf57ec0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34978 - "GET /node/6988b7399fc35f969e1eca25%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50208 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:15:54,367 - lava_callback - INFO - Received callback for node 69cb2e508c06c2799bf57ed2 INFO: 172.20.0.1:55756 - "POST /node/69cb2e508c06c2799bf57ed2 HTTP/1.0" 202 Accepted 2026-04-02 02:15:54,372 - lava_callback - INFO - Starting async job submit for node 69cb2e508c06c2799bf57ed2 2026-04-02 02:15:54,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:15:54,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2e508c06c2799bf57ed2 HTTP/1.1" 200 None 2026-04-02 02:15:54,388 - lava_callback - DEBUG - Processing node 69cb2e508c06c2799bf57ed2: baseline-arm64-clabbe 2026-04-02 02:15:54,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb2e508c06c2799bf57ed2 2026-04-02 02:15:54,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:15:54,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb2e508c06c2799bf57ed2 HTTP/1.1" 200 None 2026-04-02 02:15:54,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:15:54,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-04-02 02:15:54,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:15:54,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb2e508c06c2799bf57ed2 HTTP/1.1" 200 None 2026-04-02 02:15:54,446 - lava_callback - ERROR - Error processing callback for node 69cb2e508c06c2799bf57ed2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54828 - "GET /node/69934d14523507f3806a5be5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54844 - "GET /node/69830e14409235c29c0de5b4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34232 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:18:08,344 - lava_callback - INFO - Received callback for node 69cdcd3023eb053795d42bab INFO: 172.20.0.1:34240 - "POST /node/69cdcd3023eb053795d42bab HTTP/1.0" 202 Accepted 2026-04-02 02:18:08,353 - lava_callback - INFO - Starting async job submit for node 69cdcd3023eb053795d42bab 2026-04-02 02:18:08,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:18:08,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcd3023eb053795d42bab HTTP/1.1" 200 None 2026-04-02 02:18:08,412 - lava_callback - DEBUG - Processing node 69cdcd3023eb053795d42bab: baseline-nfs-arm64 2026-04-02 02:18:09,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:18:09,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:18:09,728 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdcd3023eb053795d42bab/log.txt.gz 2026-04-02 02:18:10,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:18:10,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:18:10,828 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdcd3023eb053795d42bab/lava-logs.yaml 2026-04-02 02:18:10,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:18:11,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:18:11,283 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdcd3023eb053795d42bab/lava_callback.json.gz DEBUG: Setup failed for 69cdcd3023eb053795d42bab. Transitting job node result: incomplete -> fail 2026-04-02 02:18:11,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:18:11,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcd3023eb053795d42bab HTTP/1.1" 200 None 2026-04-02 02:18:11,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:18:11,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:18:11,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:18:11,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcd3023eb053795d42bab HTTP/1.1" 200 None 2026-04-02 02:18:11,525 - lava_callback - INFO - Completed processing callback for node 69cdcd3023eb053795d42bab INFO: 172.20.0.1:48554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:18:39,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:18:39,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60828 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:20:05,151 - lava_callback - INFO - Received callback for node 69cdcdc923eb053795d42bbf INFO: 172.20.0.1:43616 - "POST /node/69cdcdc923eb053795d42bbf HTTP/1.0" 202 Accepted 2026-04-02 02:20:05,197 - lava_callback - INFO - Starting async job submit for node 69cdcdc923eb053795d42bbf 2026-04-02 02:20:05,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:20:05,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcdc923eb053795d42bbf HTTP/1.1" 200 None 2026-04-02 02:20:05,237 - lava_callback - DEBUG - Processing node 69cdcdc923eb053795d42bbf: baseline-arm-pengutronix 2026-04-02 02:20:05,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:20:05,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:20:05,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdcdc923eb053795d42bbf/log.txt.gz 2026-04-02 02:20:05,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:20:05,775 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:20:05,777 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdcdc923eb053795d42bbf/lava-logs.yaml 2026-04-02 02:20:05,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:20:05,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:20:05,925 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdcdc923eb053795d42bbf/lava_callback.json.gz DEBUG: Setup failed for 69cdcdc923eb053795d42bbf. Transitting job node result: incomplete -> fail 2026-04-02 02:20:05,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:20:05,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcdc923eb053795d42bbf HTTP/1.1" 200 None 2026-04-02 02:20:05,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:20:06,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 02:20:06,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:20:06,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcdc923eb053795d42bbf HTTP/1.1" 200 None 2026-04-02 02:20:06,108 - lava_callback - INFO - Completed processing callback for node 69cdcdc923eb053795d42bbf INFO: 172.20.0.1:43622 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:20:09,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:20:09,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32942 - "GET /node/691a72578dbc6bccc0a0bfdf%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52448 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:26:29,997 - lava_callback - INFO - Received callback for node 69cdd28d23eb053795d42c8f INFO: 172.20.0.1:57772 - "POST /node/69cdd28d23eb053795d42c8f HTTP/1.0" 202 Accepted 2026-04-02 02:26:30,004 - lava_callback - INFO - Starting async job submit for node 69cdd28d23eb053795d42c8f 2026-04-02 02:26:30,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:26:30,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd28d23eb053795d42c8f HTTP/1.1" 200 None 2026-04-02 02:26:30,061 - lava_callback - DEBUG - Processing node 69cdd28d23eb053795d42c8f: baseline-arm64 2026-04-02 02:26:30,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:26:30,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:26:30,597 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdd28d23eb053795d42c8f/log.txt.gz 2026-04-02 02:26:30,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:26:31,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:26:31,087 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdd28d23eb053795d42c8f/lava-logs.yaml 2026-04-02 02:26:31,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:26:31,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:26:31,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdd28d23eb053795d42c8f/lava_callback.json.gz DEBUG: Setup failed for 69cdd28d23eb053795d42c8f. Transitting job node result: incomplete -> fail 2026-04-02 02:26:31,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:26:31,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd28d23eb053795d42c8f HTTP/1.1" 200 None 2026-04-02 02:26:31,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:26:31,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:26:31,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:26:31,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd28d23eb053795d42c8f HTTP/1.1" 200 None 2026-04-02 02:26:31,492 - lava_callback - INFO - Completed processing callback for node 69cdd28d23eb053795d42c8f INFO: 172.20.0.1:57784 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:26:39,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:26:39,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54584 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:27:03,456 - lava_callback - INFO - Received callback for node 69cdd18023eb053795d42c62 INFO: 172.20.0.1:54254 - "POST /node/69cdd18023eb053795d42c62 HTTP/1.0" 202 Accepted 2026-04-02 02:27:03,466 - lava_callback - INFO - Starting async job submit for node 69cdd18023eb053795d42c62 2026-04-02 02:27:03,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:27:03,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd18023eb053795d42c62 HTTP/1.1" 200 None 2026-04-02 02:27:03,508 - lava_callback - DEBUG - Processing node 69cdd18023eb053795d42c62: baseline-arm64 2026-04-02 02:27:04,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:27:04,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:27:04,183 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdd18023eb053795d42c62/log.txt.gz 2026-04-02 02:27:04,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:27:04,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:27:04,724 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdd18023eb053795d42c62/lava-logs.yaml 2026-04-02 02:27:04,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:27:04,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:27:04,921 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdd18023eb053795d42c62/lava_callback.json.gz 2026-04-02 02:27:04,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:27:04,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd18023eb053795d42c62 HTTP/1.1" 200 None 2026-04-02 02:27:04,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:27:04,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:27:04,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:27:04,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd18023eb053795d42c62 HTTP/1.1" 200 None 2026-04-02 02:27:05,066 - lava_callback - INFO - Completed processing callback for node 69cdd18023eb053795d42c62 INFO: 172.20.0.1:54260 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:27:09,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:27:09,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44750 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:29:37,744 - lava_callback - INFO - Received callback for node 69cdd21f23eb053795d42c77 INFO: 172.20.0.1:44764 - "POST /node/69cdd21f23eb053795d42c77 HTTP/1.0" 202 Accepted 2026-04-02 02:29:37,751 - lava_callback - INFO - Starting async job submit for node 69cdd21f23eb053795d42c77 2026-04-02 02:29:37,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:29:37,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd21f23eb053795d42c77 HTTP/1.1" 200 None 2026-04-02 02:29:37,817 - lava_callback - DEBUG - Processing node 69cdd21f23eb053795d42c77: baseline-nfs-arm64 2026-04-02 02:29:38,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:29:38,622 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:29:38,624 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd21f23eb053795d42c77/log.txt.gz 2026-04-02 02:29:38,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:29:39,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:29:39,149 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd21f23eb053795d42c77/lava-logs.yaml 2026-04-02 02:29:39,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:29:39,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:29:39,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd21f23eb053795d42c77/lava_callback.json.gz 2026-04-02 02:29:39,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:29:39,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd21f23eb053795d42c77 HTTP/1.1" 200 None 2026-04-02 02:29:39,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:29:39,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:29:39,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:29:39,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd21f23eb053795d42c77 HTTP/1.1" 200 None 2026-04-02 02:29:39,481 - lava_callback - INFO - Completed processing callback for node 69cdd21f23eb053795d42c77 INFO: 172.20.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48380 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:30:09,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:30:09,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48338 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:32:31,164 - lava_callback - INFO - Received callback for node 69cdd12d23eb053795d42c54 INFO: 172.20.0.1:58022 - "POST /node/69cdd12d23eb053795d42c54 HTTP/1.0" 202 Accepted 2026-04-02 02:32:31,170 - lava_callback - INFO - Starting async job submit for node 69cdd12d23eb053795d42c54 2026-04-02 02:32:31,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:32:31,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd12d23eb053795d42c54 HTTP/1.1" 200 None 2026-04-02 02:32:31,251 - lava_callback - DEBUG - Processing node 69cdd12d23eb053795d42c54: ltp-smoke 2026-04-02 02:32:31,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:32:31,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:32:31,895 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdd12d23eb053795d42c54/log.txt.gz 2026-04-02 02:32:32,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:32:32,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:32:32,412 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdd12d23eb053795d42c54/lava-logs.yaml 2026-04-02 02:32:32,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:32:32,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:32:32,581 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cdd12d23eb053795d42c54/lava_callback.json.gz 2026-04-02 02:32:32,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:32:32,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd12d23eb053795d42c54 HTTP/1.1" 200 None 2026-04-02 02:32:32,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:32:32,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5bf23eb053795d41ebd HTTP/1.1" 200 None 2026-04-02 02:32:32,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:32:32,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd12d23eb053795d42c54 HTTP/1.1" 200 None 2026-04-02 02:32:32,909 - lava_callback - INFO - Completed processing callback for node 69cdd12d23eb053795d42c54 INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:32:39,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:32:39,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54712 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:37:54,055 - lava_callback - INFO - Received callback for node 69cdd3ff23eb053795d42cb0 INFO: 172.20.0.1:52598 - "POST /node/69cdd3ff23eb053795d42cb0 HTTP/1.0" 202 Accepted 2026-04-02 02:37:54,062 - lava_callback - INFO - Starting async job submit for node 69cdd3ff23eb053795d42cb0 2026-04-02 02:37:54,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:37:54,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd3ff23eb053795d42cb0 HTTP/1.1" 200 None 2026-04-02 02:37:54,108 - lava_callback - DEBUG - Processing node 69cdd3ff23eb053795d42cb0: baseline-arm64 2026-04-02 02:37:54,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:37:54,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:37:54,786 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdd3ff23eb053795d42cb0/log.txt.gz 2026-04-02 02:37:55,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:37:55,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:37:55,484 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdd3ff23eb053795d42cb0/lava-logs.yaml 2026-04-02 02:37:55,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:37:55,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:37:55,672 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cdd3ff23eb053795d42cb0/lava_callback.json.gz 2026-04-02 02:37:55,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:37:55,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd3ff23eb053795d42cb0 HTTP/1.1" 200 None 2026-04-02 02:37:55,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:37:55,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:37:55,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:37:55,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd3ff23eb053795d42cb0 HTTP/1.1" 200 None 2026-04-02 02:37:55,843 - lava_callback - INFO - Completed processing callback for node 69cdd3ff23eb053795d42cb0 2026-04-02 02:37:56,174 - lava_callback - INFO - Received callback for node 69cdd26423eb053795d42c85 INFO: 172.20.0.1:52608 - "POST /node/69cdd26423eb053795d42c85 HTTP/1.0" 202 Accepted 2026-04-02 02:37:56,181 - lava_callback - INFO - Starting async job submit for node 69cdd26423eb053795d42c85 2026-04-02 02:37:56,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:37:56,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd26423eb053795d42c85 HTTP/1.1" 200 None 2026-04-02 02:37:56,301 - lava_callback - DEBUG - Processing node 69cdd26423eb053795d42c85: baseline-arm INFO: 172.20.0.1:52612 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:37:56,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:37:57,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:37:57,025 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cdd26423eb053795d42c85/log.txt.gz 2026-04-02 02:37:57,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:37:57,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:37:57,674 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cdd26423eb053795d42c85/lava-logs.yaml 2026-04-02 02:37:57,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:37:57,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:37:57,892 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cdd26423eb053795d42c85/lava_callback.json.gz DEBUG: Setup failed for 69cdd26423eb053795d42c85. Transitting job node result: incomplete -> fail 2026-04-02 02:37:58,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:37:58,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd26423eb053795d42c85 HTTP/1.1" 200 None 2026-04-02 02:37:58,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:37:58,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 02:37:58,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:37:58,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd26423eb053795d42c85 HTTP/1.1" 200 None 2026-04-02 02:37:58,246 - lava_callback - INFO - Completed processing callback for node 69cdd26423eb053795d42c85 INFO: 172.20.0.1:48776 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:38:09,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:38:09,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57412 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:40:36,415 - lava_callback - INFO - Received callback for node 69cdd49d23eb053795d42cba INFO: 172.20.0.1:49160 - "POST /node/69cdd49d23eb053795d42cba HTTP/1.0" 202 Accepted 2026-04-02 02:40:36,422 - lava_callback - INFO - Starting async job submit for node 69cdd49d23eb053795d42cba 2026-04-02 02:40:36,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:40:36,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd49d23eb053795d42cba HTTP/1.1" 200 None 2026-04-02 02:40:36,463 - lava_callback - DEBUG - Processing node 69cdd49d23eb053795d42cba: baseline-nfs-arm64 INFO: 172.20.0.1:49168 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:40:36,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:40:37,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:40:37,137 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd49d23eb053795d42cba/log.txt.gz 2026-04-02 02:40:37,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:40:37,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:40:37,736 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd49d23eb053795d42cba/lava-logs.yaml 2026-04-02 02:40:37,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:40:38,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:40:38,197 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd49d23eb053795d42cba/lava_callback.json.gz 2026-04-02 02:40:38,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:40:38,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd49d23eb053795d42cba HTTP/1.1" 200 None 2026-04-02 02:40:38,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:40:38,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:40:38,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:40:38,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd49d23eb053795d42cba HTTP/1.1" 200 None 2026-04-02 02:40:38,352 - lava_callback - INFO - Completed processing callback for node 69cdd49d23eb053795d42cba 2026-04-02 02:40:39,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:40:39,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34212 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:42:09,091 - lava_callback - INFO - Received callback for node 69cdd2dc23eb053795d42c99 INFO: 172.20.0.1:34216 - "POST /node/69cdd2dc23eb053795d42c99 HTTP/1.0" 202 Accepted 2026-04-02 02:42:09,099 - lava_callback - INFO - Starting async job submit for node 69cdd2dc23eb053795d42c99 2026-04-02 02:42:09,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:42:09,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd2dc23eb053795d42c99 HTTP/1.1" 200 None 2026-04-02 02:42:09,163 - lava_callback - DEBUG - Processing node 69cdd2dc23eb053795d42c99: baseline-nfs-arm64 2026-04-02 02:42:10,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:42:10,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:42:10,785 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd2dc23eb053795d42c99/log.txt.gz 2026-04-02 02:42:11,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:42:11,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:42:11,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd2dc23eb053795d42c99/lava-logs.yaml 2026-04-02 02:42:11,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:42:11,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:42:11,999 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd2dc23eb053795d42c99/lava_callback.json.gz DEBUG: Setup failed for 69cdd2dc23eb053795d42c99. Transitting job node result: incomplete -> fail 2026-04-02 02:42:12,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:42:12,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd2dc23eb053795d42c99 HTTP/1.1" 200 None 2026-04-02 02:42:12,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:42:12,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 02:42:12,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:42:12,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd2dc23eb053795d42c99 HTTP/1.1" 200 None 2026-04-02 02:42:12,266 - lava_callback - INFO - Completed processing callback for node 69cdd2dc23eb053795d42c99 INFO: 172.20.0.1:32776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57354 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:42:39,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:42:39,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58200 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:43:06,102 - lava_callback - INFO - Received callback for node 69cdd32a23eb053795d42c9f INFO: 172.20.0.1:40130 - "POST /node/69cdd32a23eb053795d42c9f HTTP/1.0" 202 Accepted 2026-04-02 02:43:06,149 - lava_callback - INFO - Starting async job submit for node 69cdd32a23eb053795d42c9f 2026-04-02 02:43:06,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:43:06,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd32a23eb053795d42c9f HTTP/1.1" 200 None 2026-04-02 02:43:06,189 - lava_callback - DEBUG - Processing node 69cdd32a23eb053795d42c9f: baseline-arm-pengutronix 2026-04-02 02:43:06,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:43:06,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:43:06,439 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdd32a23eb053795d42c9f/log.txt.gz INFO: 172.20.0.1:40136 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:43:06,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:43:06,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:43:06,666 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdd32a23eb053795d42c9f/lava-logs.yaml 2026-04-02 02:43:06,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 02:43:06,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 02:43:06,791 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cdd32a23eb053795d42c9f/lava_callback.json.gz DEBUG: Setup failed for 69cdd32a23eb053795d42c9f. Transitting job node result: incomplete -> fail 2026-04-02 02:43:06,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:43:06,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd32a23eb053795d42c9f HTTP/1.1" 200 None 2026-04-02 02:43:06,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:43:06,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-02 02:43:06,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:43:06,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd32a23eb053795d42c9f HTTP/1.1" 200 None 2026-04-02 02:43:06,977 - lava_callback - INFO - Completed processing callback for node 69cdd32a23eb053795d42c9f 2026-04-02 02:43:09,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:43:09,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45370 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 02:53:16,195 - lava_callback - INFO - Received callback for node 69cb370e8c06c2799bf58018 INFO: 172.20.0.1:45382 - "POST /node/69cb370e8c06c2799bf58018 HTTP/1.0" 202 Accepted 2026-04-02 02:53:16,200 - lava_callback - INFO - Starting async job submit for node 69cb370e8c06c2799bf58018 2026-04-02 02:53:16,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:53:16,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb370e8c06c2799bf58018 HTTP/1.1" 200 None 2026-04-02 02:53:16,217 - lava_callback - DEBUG - Processing node 69cb370e8c06c2799bf58018: baseline-arm-clabbe 2026-04-02 02:53:16,217 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb370e8c06c2799bf58018 2026-04-02 02:53:16,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:53:16,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb370e8c06c2799bf58018 HTTP/1.1" 200 None 2026-04-02 02:53:16,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:53:16,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-02 02:53:16,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:53:16,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb370e8c06c2799bf58018 HTTP/1.1" 200 None 2026-04-02 02:53:16,277 - lava_callback - ERROR - Error processing callback for node 69cb370e8c06c2799bf58018: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42122 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:53:55,919 - lava_callback - INFO - Received callback for node 69cb37358c06c2799bf58021 INFO: 172.20.0.1:33118 - "POST /node/69cb37358c06c2799bf58021 HTTP/1.0" 202 Accepted 2026-04-02 02:53:55,924 - lava_callback - INFO - Starting async job submit for node 69cb37358c06c2799bf58021 2026-04-02 02:53:55,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:53:55,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb37358c06c2799bf58021 HTTP/1.1" 200 None 2026-04-02 02:53:55,941 - lava_callback - DEBUG - Processing node 69cb37358c06c2799bf58021: baseline-arm-clabbe 2026-04-02 02:53:55,941 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb37358c06c2799bf58021 2026-04-02 02:53:55,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:53:55,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb37358c06c2799bf58021 HTTP/1.1" 200 None 2026-04-02 02:53:55,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:53:55,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-02 02:53:55,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:53:55,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb37358c06c2799bf58021 HTTP/1.1" 200 None 2026-04-02 02:53:56,002 - lava_callback - ERROR - Error processing callback for node 69cb37358c06c2799bf58021: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59260 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:54:35,981 - lava_callback - INFO - Received callback for node 69cb375c8c06c2799bf5802b INFO: 172.20.0.1:44214 - "POST /node/69cb375c8c06c2799bf5802b HTTP/1.0" 202 Accepted 2026-04-02 02:54:35,986 - lava_callback - INFO - Starting async job submit for node 69cb375c8c06c2799bf5802b 2026-04-02 02:54:35,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:54:36,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb375c8c06c2799bf5802b HTTP/1.1" 200 None 2026-04-02 02:54:36,003 - lava_callback - DEBUG - Processing node 69cb375c8c06c2799bf5802b: baseline-arm-clabbe 2026-04-02 02:54:36,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb375c8c06c2799bf5802b 2026-04-02 02:54:36,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:54:36,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb375c8c06c2799bf5802b HTTP/1.1" 200 None 2026-04-02 02:54:36,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:54:36,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-02 02:54:36,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:54:36,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb375c8c06c2799bf5802b HTTP/1.1" 200 None 2026-04-02 02:54:36,059 - lava_callback - ERROR - Error processing callback for node 69cb375c8c06c2799bf5802b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40362 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 02:55:15,950 - lava_callback - INFO - Received callback for node 69cb37838c06c2799bf5803b INFO: 172.20.0.1:40366 - "POST /node/69cb37838c06c2799bf5803b HTTP/1.0" 202 Accepted 2026-04-02 02:55:15,955 - lava_callback - INFO - Starting async job submit for node 69cb37838c06c2799bf5803b 2026-04-02 02:55:15,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:55:15,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb37838c06c2799bf5803b HTTP/1.1" 200 None 2026-04-02 02:55:15,971 - lava_callback - DEBUG - Processing node 69cb37838c06c2799bf5803b: baseline-arm-clabbe 2026-04-02 02:55:15,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb37838c06c2799bf5803b 2026-04-02 02:55:15,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:55:15,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb37838c06c2799bf5803b HTTP/1.1" 200 None 2026-04-02 02:55:15,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:55:16,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-02 02:55:16,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:55:16,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb37838c06c2799bf5803b HTTP/1.1" 200 None 2026-04-02 02:55:16,038 - lava_callback - ERROR - Error processing callback for node 69cb37838c06c2799bf5803b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57576 - "GET / HTTP/1.0" 200 OK 2026-04-02 02:55:55,995 - lava_callback - INFO - Received callback for node 69cb37aa8c06c2799bf58044 INFO: 172.20.0.1:55738 - "POST /node/69cb37aa8c06c2799bf58044 HTTP/1.0" 202 Accepted 2026-04-02 02:55:56,001 - lava_callback - INFO - Starting async job submit for node 69cb37aa8c06c2799bf58044 2026-04-02 02:55:56,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:55:56,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb37aa8c06c2799bf58044 HTTP/1.1" 200 None 2026-04-02 02:55:56,017 - lava_callback - DEBUG - Processing node 69cb37aa8c06c2799bf58044: baseline-arm-clabbe 2026-04-02 02:55:56,017 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb37aa8c06c2799bf58044 2026-04-02 02:55:56,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:55:56,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb37aa8c06c2799bf58044 HTTP/1.1" 200 None 2026-04-02 02:55:56,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:55:56,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-02 02:55:56,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 02:55:56,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb37aa8c06c2799bf58044 HTTP/1.1" 200 None 2026-04-02 02:55:56,103 - lava_callback - ERROR - Error processing callback for node 69cb37aa8c06c2799bf58044: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50354 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:02:55,076 - lava_callback - INFO - Received callback for node 69cdd7ad23eb053795d42d10 INFO: 172.20.0.1:47246 - "POST /node/69cdd7ad23eb053795d42d10 HTTP/1.0" 202 Accepted 2026-04-02 03:02:55,084 - lava_callback - INFO - Starting async job submit for node 69cdd7ad23eb053795d42d10 2026-04-02 03:02:55,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:02:55,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd7ad23eb053795d42d10 HTTP/1.1" 200 None 2026-04-02 03:02:55,143 - lava_callback - DEBUG - Processing node 69cdd7ad23eb053795d42d10: baseline-nfs-arm64 2026-04-02 03:02:56,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:47258 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:02:56,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 03:02:56,515 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd7ad23eb053795d42d10/log.txt.gz 2026-04-02 03:02:57,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 03:02:57,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 03:02:57,634 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd7ad23eb053795d42d10/lava-logs.yaml 2026-04-02 03:02:57,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-02 03:02:57,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-02 03:02:57,842 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cdd7ad23eb053795d42d10/lava_callback.json.gz DEBUG: Setup failed for 69cdd7ad23eb053795d42d10. Transitting job node result: incomplete -> fail 2026-04-02 03:02:57,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:02:57,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd7ad23eb053795d42d10 HTTP/1.1" 200 None 2026-04-02 03:02:57,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:02:57,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-02 03:02:57,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:02:57,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd7ad23eb053795d42d10 HTTP/1.1" 200 None 2026-04-02 03:02:58,074 - lava_callback - INFO - Completed processing callback for node 69cdd7ad23eb053795d42d10 INFO: 172.20.0.1:36214 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:03:09,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:03:09,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49118 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:07:24,138 - lava_callback - INFO - Received callback for node 69cb3a5b8c06c2799bf58093 INFO: 172.20.0.1:53556 - "POST /node/69cb3a5b8c06c2799bf58093 HTTP/1.0" 202 Accepted 2026-04-02 03:07:24,143 - lava_callback - INFO - Starting async job submit for node 69cb3a5b8c06c2799bf58093 2026-04-02 03:07:24,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:07:24,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3a5b8c06c2799bf58093 HTTP/1.1" 200 None 2026-04-02 03:07:24,162 - lava_callback - DEBUG - Processing node 69cb3a5b8c06c2799bf58093: baseline-arm64-baylibre 2026-04-02 03:07:24,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3a5b8c06c2799bf58093 2026-04-02 03:07:24,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:07:24,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3a5b8c06c2799bf58093 HTTP/1.1" 200 None 2026-04-02 03:07:24,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:07:24,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:07:24,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:07:24,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3a5b8c06c2799bf58093 HTTP/1.1" 200 None 2026-04-02 03:07:24,218 - lava_callback - ERROR - Error processing callback for node 69cb3a5b8c06c2799bf58093: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40048 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:09:24,250 - lava_callback - INFO - Received callback for node 69cb3ad68c06c2799bf580a7 INFO: 172.20.0.1:51288 - "POST /node/69cb3ad68c06c2799bf580a7 HTTP/1.0" 202 Accepted 2026-04-02 03:09:24,259 - lava_callback - INFO - Starting async job submit for node 69cb3ad68c06c2799bf580a7 2026-04-02 03:09:24,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:09:24,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3ad68c06c2799bf580a7 HTTP/1.1" 200 None 2026-04-02 03:09:24,276 - lava_callback - DEBUG - Processing node 69cb3ad68c06c2799bf580a7: baseline-arm64-baylibre 2026-04-02 03:09:24,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3ad68c06c2799bf580a7 2026-04-02 03:09:24,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:09:24,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3ad68c06c2799bf580a7 HTTP/1.1" 200 None 2026-04-02 03:09:24,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:09:24,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:09:24,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:09:24,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3ad68c06c2799bf580a7 HTTP/1.1" 200 None 2026-04-02 03:09:24,335 - lava_callback - ERROR - Error processing callback for node 69cb3ad68c06c2799bf580a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53878 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:10:44,363 - lava_callback - INFO - Received callback for node 69cb3b258c06c2799bf580bb INFO: 172.20.0.1:50606 - "POST /node/69cb3b258c06c2799bf580bb HTTP/1.0" 202 Accepted 2026-04-02 03:10:44,372 - lava_callback - INFO - Starting async job submit for node 69cb3b258c06c2799bf580bb 2026-04-02 03:10:44,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:10:44,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3b258c06c2799bf580bb HTTP/1.1" 200 None 2026-04-02 03:10:44,410 - lava_callback - DEBUG - Processing node 69cb3b258c06c2799bf580bb: baseline-arm64-baylibre 2026-04-02 03:10:44,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3b258c06c2799bf580bb 2026-04-02 03:10:44,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:10:44,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3b258c06c2799bf580bb HTTP/1.1" 200 None 2026-04-02 03:10:44,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:10:44,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:10:44,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:10:44,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3b258c06c2799bf580bb HTTP/1.1" 200 None 2026-04-02 03:10:44,473 - lava_callback - ERROR - Error processing callback for node 69cb3b258c06c2799bf580bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47954 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:12:04,445 - lava_callback - INFO - Received callback for node 69cb3b758c06c2799bf580cc INFO: 172.20.0.1:53852 - "POST /node/69cb3b758c06c2799bf580cc HTTP/1.0" 202 Accepted 2026-04-02 03:12:04,451 - lava_callback - INFO - Starting async job submit for node 69cb3b758c06c2799bf580cc 2026-04-02 03:12:04,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:12:04,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3b758c06c2799bf580cc HTTP/1.1" 200 None 2026-04-02 03:12:04,467 - lava_callback - DEBUG - Processing node 69cb3b758c06c2799bf580cc: baseline-arm64-baylibre 2026-04-02 03:12:04,467 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3b758c06c2799bf580cc 2026-04-02 03:12:04,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:12:04,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3b758c06c2799bf580cc HTTP/1.1" 200 None 2026-04-02 03:12:04,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:12:04,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:12:04,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:12:04,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3b758c06c2799bf580cc HTTP/1.1" 200 None 2026-04-02 03:12:04,532 - lava_callback - ERROR - Error processing callback for node 69cb3b758c06c2799bf580cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51262 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:15:04,546 - lava_callback - INFO - Received callback for node 69cb3c358c06c2799bf580df INFO: 172.20.0.1:45332 - "POST /node/69cb3c358c06c2799bf580df HTTP/1.0" 202 Accepted 2026-04-02 03:15:04,552 - lava_callback - INFO - Starting async job submit for node 69cb3c358c06c2799bf580df 2026-04-02 03:15:04,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:15:04,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3c358c06c2799bf580df HTTP/1.1" 200 None 2026-04-02 03:15:04,568 - lava_callback - DEBUG - Processing node 69cb3c358c06c2799bf580df: baseline-arm64-baylibre 2026-04-02 03:15:04,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3c358c06c2799bf580df 2026-04-02 03:15:04,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:15:04,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3c358c06c2799bf580df HTTP/1.1" 200 None 2026-04-02 03:15:04,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:15:04,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:15:04,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:15:04,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3c358c06c2799bf580df HTTP/1.1" 200 None 2026-04-02 03:15:04,641 - lava_callback - ERROR - Error processing callback for node 69cb3c358c06c2799bf580df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35678 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:18:17,001 - lava_callback - INFO - Received callback for node 69cb3ce58c06c2799bf580ec INFO: 172.20.0.1:35684 - "POST /node/69cb3ce58c06c2799bf580ec HTTP/1.0" 202 Accepted 2026-04-02 03:18:17,007 - lava_callback - INFO - Starting async job submit for node 69cb3ce58c06c2799bf580ec 2026-04-02 03:18:17,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:18:17,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3ce58c06c2799bf580ec HTTP/1.1" 200 None 2026-04-02 03:18:17,024 - lava_callback - DEBUG - Processing node 69cb3ce58c06c2799bf580ec: baseline-arm64-clabbe 2026-04-02 03:18:17,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3ce58c06c2799bf580ec 2026-04-02 03:18:17,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:18:17,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3ce58c06c2799bf580ec HTTP/1.1" 200 None 2026-04-02 03:18:17,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:18:17,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:18:17,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:18:17,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3ce58c06c2799bf580ec HTTP/1.1" 200 None 2026-04-02 03:18:17,090 - lava_callback - ERROR - Error processing callback for node 69cb3ce58c06c2799bf580ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50336 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:20:17,129 - lava_callback - INFO - Received callback for node 69cb3d5e8c06c2799bf580fe INFO: 172.20.0.1:50350 - "POST /node/69cb3d5e8c06c2799bf580fe HTTP/1.0" 202 Accepted 2026-04-02 03:20:17,134 - lava_callback - INFO - Starting async job submit for node 69cb3d5e8c06c2799bf580fe 2026-04-02 03:20:17,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:20:17,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3d5e8c06c2799bf580fe HTTP/1.1" 200 None 2026-04-02 03:20:17,151 - lava_callback - DEBUG - Processing node 69cb3d5e8c06c2799bf580fe: baseline-arm64-clabbe 2026-04-02 03:20:17,152 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3d5e8c06c2799bf580fe 2026-04-02 03:20:17,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:20:17,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3d5e8c06c2799bf580fe HTTP/1.1" 200 None 2026-04-02 03:20:17,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:20:17,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:20:17,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:20:17,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3d5e8c06c2799bf580fe HTTP/1.1" 200 None 2026-04-02 03:20:17,222 - lava_callback - ERROR - Error processing callback for node 69cb3d5e8c06c2799bf580fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43480 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:23:17,532 - lava_callback - INFO - Received callback for node 69cb3e148c06c2799bf5810c INFO: 172.20.0.1:43486 - "POST /node/69cb3e148c06c2799bf5810c HTTP/1.0" 202 Accepted 2026-04-02 03:23:17,537 - lava_callback - INFO - Starting async job submit for node 69cb3e148c06c2799bf5810c 2026-04-02 03:23:17,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:23:17,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3e148c06c2799bf5810c HTTP/1.1" 200 None 2026-04-02 03:23:17,553 - lava_callback - DEBUG - Processing node 69cb3e148c06c2799bf5810c: baseline-arm64-clabbe 2026-04-02 03:23:17,553 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3e148c06c2799bf5810c 2026-04-02 03:23:17,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:23:17,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3e148c06c2799bf5810c HTTP/1.1" 200 None 2026-04-02 03:23:17,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:23:17,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:23:17,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:23:17,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3e148c06c2799bf5810c HTTP/1.1" 200 None 2026-04-02 03:23:17,610 - lava_callback - ERROR - Error processing callback for node 69cb3e148c06c2799bf5810c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41650 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:26:37,419 - lava_callback - INFO - Received callback for node 69cb3ee68c06c2799bf58124 INFO: 172.20.0.1:41664 - "POST /node/69cb3ee68c06c2799bf58124 HTTP/1.0" 202 Accepted 2026-04-02 03:26:37,424 - lava_callback - INFO - Starting async job submit for node 69cb3ee68c06c2799bf58124 2026-04-02 03:26:37,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:26:37,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3ee68c06c2799bf58124 HTTP/1.1" 200 None 2026-04-02 03:26:37,449 - lava_callback - DEBUG - Processing node 69cb3ee68c06c2799bf58124: baseline-arm64-clabbe 2026-04-02 03:26:37,450 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3ee68c06c2799bf58124 2026-04-02 03:26:37,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:26:37,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3ee68c06c2799bf58124 HTTP/1.1" 200 None 2026-04-02 03:26:37,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:26:37,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:26:37,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:26:37,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3ee68c06c2799bf58124 HTTP/1.1" 200 None 2026-04-02 03:26:37,685 - lava_callback - ERROR - Error processing callback for node 69cb3ee68c06c2799bf58124: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42376 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:28:17,486 - lava_callback - INFO - Received callback for node 69cb3f3e8c06c2799bf58131 INFO: 172.20.0.1:42384 - "POST /node/69cb3f3e8c06c2799bf58131 HTTP/1.0" 202 Accepted 2026-04-02 03:28:17,491 - lava_callback - INFO - Starting async job submit for node 69cb3f3e8c06c2799bf58131 2026-04-02 03:28:17,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:28:17,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3f3e8c06c2799bf58131 HTTP/1.1" 200 None 2026-04-02 03:28:17,509 - lava_callback - DEBUG - Processing node 69cb3f3e8c06c2799bf58131: baseline-arm64-clabbe 2026-04-02 03:28:17,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3f3e8c06c2799bf58131 2026-04-02 03:28:17,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:28:17,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3f3e8c06c2799bf58131 HTTP/1.1" 200 None 2026-04-02 03:28:17,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:28:17,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:28:17,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:28:17,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3f3e8c06c2799bf58131 HTTP/1.1" 200 None 2026-04-02 03:28:17,571 - lava_callback - ERROR - Error processing callback for node 69cb3f3e8c06c2799bf58131: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47854 - "GET / HTTP/1.0" 200 OK 2026-04-02 03:31:17,560 - lava_callback - INFO - Received callback for node 69cb3ff38c06c2799bf58143 INFO: 172.20.0.1:47870 - "POST /node/69cb3ff38c06c2799bf58143 HTTP/1.0" 202 Accepted 2026-04-02 03:31:17,567 - lava_callback - INFO - Starting async job submit for node 69cb3ff38c06c2799bf58143 2026-04-02 03:31:17,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:31:17,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3ff38c06c2799bf58143 HTTP/1.1" 200 None 2026-04-02 03:31:17,582 - lava_callback - DEBUG - Processing node 69cb3ff38c06c2799bf58143: baseline-arm64-clabbe 2026-04-02 03:31:17,582 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb3ff38c06c2799bf58143 2026-04-02 03:31:17,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:31:17,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb3ff38c06c2799bf58143 HTTP/1.1" 200 None 2026-04-02 03:31:17,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:31:17,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-02 03:31:17,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 03:31:17,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb3ff38c06c2799bf58143 HTTP/1.1" 200 None 2026-04-02 03:31:17,639 - lava_callback - ERROR - Error processing callback for node 69cb3ff38c06c2799bf58143: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45564 - "GET /node/69937a63cebe99dd8e06b9d4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53706 - "GET /node/694aadff83c67c43d1ede727%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56998 - "GET /node/6983ed38969566dcb0d29a7a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57002 - "GET /node/698fcf2176d4b6342e9d7f7d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47558 - "GET /node/6990a98ac45febfe445215b5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51878 - "GET /node/69a159962b7cf147caaf0428%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51880 - "GET /node/6985d90971989fb14149a88f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47736 - "GET /node/698b8f220a16aea5a0dc8ed2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49996 - "GET /node/698458b53a8748f2820850fb%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50010 - "GET /node/694aae3b83c67c43d1ede73a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46286 - "GET /node/69903bc6985e4efca431dea1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:46294 - "GET /node/69c17ed92b2b51e9ad97a999%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48546 - "GET /node/69bd7b0c0e93e2314723e7bd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36000 - "GET /node/69be6378448e074b3ffe21f5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:36014 - "GET /node/69bdea83d54b5122738c6626%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:33164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35534 - "GET /node/698a8343d9a1cfa777a23a5d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50058 - "GET /node/69836a64409235c29c0e0eb6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50074 - "GET /node/69c16cc92b2b51e9ad97a0f5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46456 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:44:00,618 - lava_callback - INFO - Received callback for node 69cb51068c06c2799bf58189 INFO: 172.20.0.1:60712 - "POST /node/69cb51068c06c2799bf58189 HTTP/1.0" 202 Accepted 2026-04-02 04:44:00,624 - lava_callback - INFO - Starting async job submit for node 69cb51068c06c2799bf58189 2026-04-02 04:44:00,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:44:00,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb51068c06c2799bf58189 HTTP/1.1" 200 None 2026-04-02 04:44:00,647 - lava_callback - DEBUG - Processing node 69cb51068c06c2799bf58189: baseline-arm-clabbe 2026-04-02 04:44:00,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb51068c06c2799bf58189 2026-04-02 04:44:00,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:44:00,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb51068c06c2799bf58189 HTTP/1.1" 200 None 2026-04-02 04:44:00,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:44:00,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:44:00,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:44:00,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb51068c06c2799bf58189 HTTP/1.1" 200 None 2026-04-02 04:44:00,718 - lava_callback - ERROR - Error processing callback for node 69cb51068c06c2799bf58189: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54238 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:44:40,582 - lava_callback - INFO - Received callback for node 69cb512d8c06c2799bf58196 INFO: 172.20.0.1:54996 - "POST /node/69cb512d8c06c2799bf58196 HTTP/1.0" 202 Accepted 2026-04-02 04:44:40,595 - lava_callback - INFO - Starting async job submit for node 69cb512d8c06c2799bf58196 2026-04-02 04:44:40,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:44:40,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb512d8c06c2799bf58196 HTTP/1.1" 200 None 2026-04-02 04:44:40,646 - lava_callback - DEBUG - Processing node 69cb512d8c06c2799bf58196: baseline-arm-clabbe 2026-04-02 04:44:40,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb512d8c06c2799bf58196 2026-04-02 04:44:40,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:44:40,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb512d8c06c2799bf58196 HTTP/1.1" 200 None 2026-04-02 04:44:40,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:44:40,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:44:40,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:44:40,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb512d8c06c2799bf58196 HTTP/1.1" 200 None 2026-04-02 04:44:40,749 - lava_callback - ERROR - Error processing callback for node 69cb512d8c06c2799bf58196: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54580 - "GET /node/698f5dda3013abef95cf904e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:43064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43074 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:45:20,715 - lava_callback - INFO - Received callback for node 69cb51528c06c2799bf581a0 INFO: 172.20.0.1:55862 - "POST /node/69cb51528c06c2799bf581a0 HTTP/1.0" 202 Accepted 2026-04-02 04:45:20,721 - lava_callback - INFO - Starting async job submit for node 69cb51528c06c2799bf581a0 2026-04-02 04:45:20,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:45:20,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb51528c06c2799bf581a0 HTTP/1.1" 200 None 2026-04-02 04:45:20,738 - lava_callback - DEBUG - Processing node 69cb51528c06c2799bf581a0: baseline-arm-clabbe 2026-04-02 04:45:20,739 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb51528c06c2799bf581a0 2026-04-02 04:45:20,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:45:20,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb51528c06c2799bf581a0 HTTP/1.1" 200 None 2026-04-02 04:45:20,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:45:20,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:45:20,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:45:20,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb51528c06c2799bf581a0 HTTP/1.1" 200 None 2026-04-02 04:45:20,809 - lava_callback - ERROR - Error processing callback for node 69cb51528c06c2799bf581a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45674 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:46:00,628 - lava_callback - INFO - Received callback for node 69cb51778c06c2799bf581af INFO: 172.20.0.1:38396 - "POST /node/69cb51778c06c2799bf581af HTTP/1.0" 202 Accepted 2026-04-02 04:46:00,634 - lava_callback - INFO - Starting async job submit for node 69cb51778c06c2799bf581af 2026-04-02 04:46:00,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:46:00,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb51778c06c2799bf581af HTTP/1.1" 200 None 2026-04-02 04:46:00,651 - lava_callback - DEBUG - Processing node 69cb51778c06c2799bf581af: baseline-arm-clabbe 2026-04-02 04:46:00,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb51778c06c2799bf581af 2026-04-02 04:46:00,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:46:00,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb51778c06c2799bf581af HTTP/1.1" 200 None 2026-04-02 04:46:00,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:46:00,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:46:00,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:46:00,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb51778c06c2799bf581af HTTP/1.1" 200 None 2026-04-02 04:46:00,733 - lava_callback - ERROR - Error processing callback for node 69cb51778c06c2799bf581af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48280 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:46:40,687 - lava_callback - INFO - Received callback for node 69cb519c8c06c2799bf581bc INFO: 172.20.0.1:39400 - "POST /node/69cb519c8c06c2799bf581bc HTTP/1.0" 202 Accepted 2026-04-02 04:46:40,694 - lava_callback - INFO - Starting async job submit for node 69cb519c8c06c2799bf581bc 2026-04-02 04:46:40,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:46:40,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb519c8c06c2799bf581bc HTTP/1.1" 200 None 2026-04-02 04:46:40,721 - lava_callback - DEBUG - Processing node 69cb519c8c06c2799bf581bc: baseline-arm-clabbe 2026-04-02 04:46:40,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb519c8c06c2799bf581bc 2026-04-02 04:46:40,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:46:40,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb519c8c06c2799bf581bc HTTP/1.1" 200 None 2026-04-02 04:46:40,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:46:40,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:46:40,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:46:40,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb519c8c06c2799bf581bc HTTP/1.1" 200 None 2026-04-02 04:46:40,809 - lava_callback - ERROR - Error processing callback for node 69cb519c8c06c2799bf581bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53318 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:47:00,644 - lava_callback - INFO - Received callback for node 69cb51c18c06c2799bf581ca INFO: 172.20.0.1:59756 - "POST /node/69cb51c18c06c2799bf581ca HTTP/1.0" 202 Accepted 2026-04-02 04:47:00,653 - lava_callback - INFO - Starting async job submit for node 69cb51c18c06c2799bf581ca 2026-04-02 04:47:00,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:47:00,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb51c18c06c2799bf581ca HTTP/1.1" 200 None 2026-04-02 04:47:00,671 - lava_callback - DEBUG - Processing node 69cb51c18c06c2799bf581ca: baseline-arm-clabbe 2026-04-02 04:47:00,671 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb51c18c06c2799bf581ca 2026-04-02 04:47:00,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:47:00,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb51c18c06c2799bf581ca HTTP/1.1" 200 None 2026-04-02 04:47:00,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:47:00,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:47:00,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:47:00,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb51c18c06c2799bf581ca HTTP/1.1" 200 None 2026-04-02 04:47:00,722 - lava_callback - ERROR - Error processing callback for node 69cb51c18c06c2799bf581ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34934 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:47:40,731 - lava_callback - INFO - Received callback for node 69cb51e68c06c2799bf581d7 INFO: 172.20.0.1:55070 - "POST /node/69cb51e68c06c2799bf581d7 HTTP/1.0" 202 Accepted 2026-04-02 04:47:40,737 - lava_callback - INFO - Starting async job submit for node 69cb51e68c06c2799bf581d7 2026-04-02 04:47:40,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:47:40,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb51e68c06c2799bf581d7 HTTP/1.1" 200 None 2026-04-02 04:47:40,761 - lava_callback - DEBUG - Processing node 69cb51e68c06c2799bf581d7: baseline-arm-clabbe 2026-04-02 04:47:40,761 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb51e68c06c2799bf581d7 2026-04-02 04:47:40,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:47:40,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb51e68c06c2799bf581d7 HTTP/1.1" 200 None 2026-04-02 04:47:40,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:47:40,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:47:40,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:47:40,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb51e68c06c2799bf581d7 HTTP/1.1" 200 None 2026-04-02 04:47:40,832 - lava_callback - ERROR - Error processing callback for node 69cb51e68c06c2799bf581d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40810 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:48:20,695 - lava_callback - INFO - Received callback for node 69cb520a8c06c2799bf581e6 INFO: 172.20.0.1:58568 - "POST /node/69cb520a8c06c2799bf581e6 HTTP/1.0" 202 Accepted 2026-04-02 04:48:20,701 - lava_callback - INFO - Starting async job submit for node 69cb520a8c06c2799bf581e6 2026-04-02 04:48:20,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:48:20,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb520a8c06c2799bf581e6 HTTP/1.1" 200 None 2026-04-02 04:48:20,716 - lava_callback - DEBUG - Processing node 69cb520a8c06c2799bf581e6: baseline-arm-clabbe 2026-04-02 04:48:20,716 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb520a8c06c2799bf581e6 2026-04-02 04:48:20,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:48:20,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb520a8c06c2799bf581e6 HTTP/1.1" 200 None 2026-04-02 04:48:20,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:48:20,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:48:20,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:48:20,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb520a8c06c2799bf581e6 HTTP/1.1" 200 None 2026-04-02 04:48:20,774 - lava_callback - ERROR - Error processing callback for node 69cb520a8c06c2799bf581e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54976 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:49:00,796 - lava_callback - INFO - Received callback for node 69cb522f8c06c2799bf581f1 INFO: 172.20.0.1:53244 - "POST /node/69cb522f8c06c2799bf581f1 HTTP/1.0" 202 Accepted 2026-04-02 04:49:00,802 - lava_callback - INFO - Starting async job submit for node 69cb522f8c06c2799bf581f1 2026-04-02 04:49:00,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:49:00,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb522f8c06c2799bf581f1 HTTP/1.1" 200 None 2026-04-02 04:49:00,818 - lava_callback - DEBUG - Processing node 69cb522f8c06c2799bf581f1: baseline-arm-clabbe 2026-04-02 04:49:00,818 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb522f8c06c2799bf581f1 2026-04-02 04:49:00,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:49:00,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb522f8c06c2799bf581f1 HTTP/1.1" 200 None 2026-04-02 04:49:00,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:49:00,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:49:00,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:49:00,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb522f8c06c2799bf581f1 HTTP/1.1" 200 None 2026-04-02 04:49:00,909 - lava_callback - ERROR - Error processing callback for node 69cb522f8c06c2799bf581f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48142 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:49:40,775 - lava_callback - INFO - Received callback for node 69cb52558c06c2799bf581fb INFO: 172.20.0.1:38314 - "POST /node/69cb52558c06c2799bf581fb HTTP/1.0" 202 Accepted 2026-04-02 04:49:40,781 - lava_callback - INFO - Starting async job submit for node 69cb52558c06c2799bf581fb 2026-04-02 04:49:40,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:49:40,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb52558c06c2799bf581fb HTTP/1.1" 200 None 2026-04-02 04:49:40,796 - lava_callback - DEBUG - Processing node 69cb52558c06c2799bf581fb: baseline-arm-clabbe 2026-04-02 04:49:40,796 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb52558c06c2799bf581fb 2026-04-02 04:49:40,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:49:40,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb52558c06c2799bf581fb HTTP/1.1" 200 None 2026-04-02 04:49:40,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:49:40,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:49:40,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:49:40,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb52558c06c2799bf581fb HTTP/1.1" 200 None 2026-04-02 04:49:40,850 - lava_callback - ERROR - Error processing callback for node 69cb52558c06c2799bf581fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46910 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:50:20,810 - lava_callback - INFO - Received callback for node 69cb52798c06c2799bf58209 INFO: 172.20.0.1:47978 - "POST /node/69cb52798c06c2799bf58209 HTTP/1.0" 202 Accepted 2026-04-02 04:50:20,816 - lava_callback - INFO - Starting async job submit for node 69cb52798c06c2799bf58209 2026-04-02 04:50:20,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:50:20,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb52798c06c2799bf58209 HTTP/1.1" 200 None 2026-04-02 04:50:20,832 - lava_callback - DEBUG - Processing node 69cb52798c06c2799bf58209: baseline-arm-clabbe 2026-04-02 04:50:20,832 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb52798c06c2799bf58209 2026-04-02 04:50:20,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:50:20,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb52798c06c2799bf58209 HTTP/1.1" 200 None 2026-04-02 04:50:20,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:50:20,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:50:20,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:50:20,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb52798c06c2799bf58209 HTTP/1.1" 200 None 2026-04-02 04:50:20,887 - lava_callback - ERROR - Error processing callback for node 69cb52798c06c2799bf58209: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55836 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:51:00,799 - lava_callback - INFO - Received callback for node 69cb529e8c06c2799bf58210 INFO: 172.20.0.1:45244 - "POST /node/69cb529e8c06c2799bf58210 HTTP/1.0" 202 Accepted 2026-04-02 04:51:00,804 - lava_callback - INFO - Starting async job submit for node 69cb529e8c06c2799bf58210 2026-04-02 04:51:00,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:51:00,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb529e8c06c2799bf58210 HTTP/1.1" 200 None 2026-04-02 04:51:00,821 - lava_callback - DEBUG - Processing node 69cb529e8c06c2799bf58210: baseline-arm-clabbe 2026-04-02 04:51:00,822 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb529e8c06c2799bf58210 2026-04-02 04:51:00,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:51:00,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb529e8c06c2799bf58210 HTTP/1.1" 200 None 2026-04-02 04:51:00,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:51:00,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:51:00,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:51:00,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb529e8c06c2799bf58210 HTTP/1.1" 200 None 2026-04-02 04:51:00,880 - lava_callback - ERROR - Error processing callback for node 69cb529e8c06c2799bf58210: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57910 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:51:20,858 - lava_callback - INFO - Received callback for node 69cb52c48c06c2799bf58219 INFO: 172.20.0.1:48744 - "POST /node/69cb52c48c06c2799bf58219 HTTP/1.0" 202 Accepted 2026-04-02 04:51:20,868 - lava_callback - INFO - Starting async job submit for node 69cb52c48c06c2799bf58219 2026-04-02 04:51:20,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:51:20,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb52c48c06c2799bf58219 HTTP/1.1" 200 None 2026-04-02 04:51:20,891 - lava_callback - DEBUG - Processing node 69cb52c48c06c2799bf58219: baseline-arm-clabbe 2026-04-02 04:51:20,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb52c48c06c2799bf58219 2026-04-02 04:51:20,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:51:20,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb52c48c06c2799bf58219 HTTP/1.1" 200 None 2026-04-02 04:51:20,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:51:20,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:51:20,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:51:20,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb52c48c06c2799bf58219 HTTP/1.1" 200 None 2026-04-02 04:51:20,948 - lava_callback - ERROR - Error processing callback for node 69cb52c48c06c2799bf58219: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49858 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:52:00,857 - lava_callback - INFO - Received callback for node 69cb52e98c06c2799bf58222 INFO: 172.20.0.1:50564 - "POST /node/69cb52e98c06c2799bf58222 HTTP/1.0" 202 Accepted 2026-04-02 04:52:00,863 - lava_callback - INFO - Starting async job submit for node 69cb52e98c06c2799bf58222 2026-04-02 04:52:00,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:52:00,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb52e98c06c2799bf58222 HTTP/1.1" 200 None 2026-04-02 04:52:00,906 - lava_callback - DEBUG - Processing node 69cb52e98c06c2799bf58222: baseline-arm-clabbe 2026-04-02 04:52:00,906 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb52e98c06c2799bf58222 2026-04-02 04:52:00,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:52:00,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb52e98c06c2799bf58222 HTTP/1.1" 200 None 2026-04-02 04:52:00,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:52:00,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:52:00,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:52:00,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb52e98c06c2799bf58222 HTTP/1.1" 200 None 2026-04-02 04:52:00,964 - lava_callback - ERROR - Error processing callback for node 69cb52e98c06c2799bf58222: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44894 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:52:40,896 - lava_callback - INFO - Received callback for node 69cb530e8c06c2799bf5822d INFO: 172.20.0.1:39252 - "POST /node/69cb530e8c06c2799bf5822d HTTP/1.0" 202 Accepted 2026-04-02 04:52:40,902 - lava_callback - INFO - Starting async job submit for node 69cb530e8c06c2799bf5822d 2026-04-02 04:52:40,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:52:40,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb530e8c06c2799bf5822d HTTP/1.1" 200 None 2026-04-02 04:52:40,917 - lava_callback - DEBUG - Processing node 69cb530e8c06c2799bf5822d: baseline-arm-clabbe 2026-04-02 04:52:40,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb530e8c06c2799bf5822d 2026-04-02 04:52:40,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:52:40,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb530e8c06c2799bf5822d HTTP/1.1" 200 None 2026-04-02 04:52:40,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:52:40,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff43123b47fd2376d74b HTTP/1.1" 200 None 2026-04-02 04:52:40,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:52:40,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb530e8c06c2799bf5822d HTTP/1.1" 200 None 2026-04-02 04:52:40,973 - lava_callback - ERROR - Error processing callback for node 69cb530e8c06c2799bf5822d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53408 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:56:10,135 - lava_callback - INFO - Received callback for node 69cb53de8c06c2799bf58237 INFO: 172.20.0.1:53660 - "POST /node/69cb53de8c06c2799bf58237 HTTP/1.0" 202 Accepted 2026-04-02 04:56:10,140 - lava_callback - INFO - Starting async job submit for node 69cb53de8c06c2799bf58237 2026-04-02 04:56:10,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:56:10,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb53de8c06c2799bf58237 HTTP/1.1" 200 None 2026-04-02 04:56:10,169 - lava_callback - DEBUG - Processing node 69cb53de8c06c2799bf58237: baseline-arm64-baylibre 2026-04-02 04:56:10,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb53de8c06c2799bf58237 2026-04-02 04:56:10,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:56:10,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb53de8c06c2799bf58237 HTTP/1.1" 200 None 2026-04-02 04:56:10,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:56:10,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 04:56:10,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:56:10,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb53de8c06c2799bf58237 HTTP/1.1" 200 None 2026-04-02 04:56:10,228 - lava_callback - ERROR - Error processing callback for node 69cb53de8c06c2799bf58237: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37648 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:57:30,081 - lava_callback - INFO - Received callback for node 69cb54388c06c2799bf58241 INFO: 172.20.0.1:36664 - "POST /node/69cb54388c06c2799bf58241 HTTP/1.0" 202 Accepted 2026-04-02 04:57:30,087 - lava_callback - INFO - Starting async job submit for node 69cb54388c06c2799bf58241 2026-04-02 04:57:30,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:57:30,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb54388c06c2799bf58241 HTTP/1.1" 200 None 2026-04-02 04:57:30,109 - lava_callback - DEBUG - Processing node 69cb54388c06c2799bf58241: baseline-arm64-baylibre 2026-04-02 04:57:30,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb54388c06c2799bf58241 2026-04-02 04:57:30,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:57:30,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb54388c06c2799bf58241 HTTP/1.1" 200 None 2026-04-02 04:57:30,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:57:30,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 04:57:30,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:57:30,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb54388c06c2799bf58241 HTTP/1.1" 200 None 2026-04-02 04:57:30,180 - lava_callback - ERROR - Error processing callback for node 69cb54388c06c2799bf58241: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33952 - "GET / HTTP/1.0" 200 OK 2026-04-02 04:58:50,151 - lava_callback - INFO - Received callback for node 69cb54848c06c2799bf5824d INFO: 172.20.0.1:57050 - "POST /node/69cb54848c06c2799bf5824d HTTP/1.0" 202 Accepted 2026-04-02 04:58:50,157 - lava_callback - INFO - Starting async job submit for node 69cb54848c06c2799bf5824d 2026-04-02 04:58:50,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:58:50,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb54848c06c2799bf5824d HTTP/1.1" 200 None 2026-04-02 04:58:50,173 - lava_callback - DEBUG - Processing node 69cb54848c06c2799bf5824d: baseline-arm64-baylibre 2026-04-02 04:58:50,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb54848c06c2799bf5824d 2026-04-02 04:58:50,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:58:50,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb54848c06c2799bf5824d HTTP/1.1" 200 None 2026-04-02 04:58:50,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:58:50,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 04:58:50,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 04:58:50,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb54848c06c2799bf5824d HTTP/1.1" 200 None 2026-04-02 04:58:50,225 - lava_callback - ERROR - Error processing callback for node 69cb54848c06c2799bf5824d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46188 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:00:10,245 - lava_callback - INFO - Received callback for node 69cb54d48c06c2799bf58260 INFO: 172.20.0.1:39674 - "POST /node/69cb54d48c06c2799bf58260 HTTP/1.0" 202 Accepted 2026-04-02 05:00:10,250 - lava_callback - INFO - Starting async job submit for node 69cb54d48c06c2799bf58260 2026-04-02 05:00:10,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:00:10,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb54d48c06c2799bf58260 HTTP/1.1" 200 None 2026-04-02 05:00:10,266 - lava_callback - DEBUG - Processing node 69cb54d48c06c2799bf58260: baseline-arm64-baylibre 2026-04-02 05:00:10,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb54d48c06c2799bf58260 2026-04-02 05:00:10,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:00:10,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb54d48c06c2799bf58260 HTTP/1.1" 200 None 2026-04-02 05:00:10,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:00:10,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:00:10,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:00:10,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb54d48c06c2799bf58260 HTTP/1.1" 200 None 2026-04-02 05:00:10,317 - lava_callback - ERROR - Error processing callback for node 69cb54d48c06c2799bf58260: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57338 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:01:30,346 - lava_callback - INFO - Received callback for node 69cb55248c06c2799bf5826a INFO: 172.20.0.1:59272 - "POST /node/69cb55248c06c2799bf5826a HTTP/1.0" 202 Accepted 2026-04-02 05:01:30,352 - lava_callback - INFO - Starting async job submit for node 69cb55248c06c2799bf5826a 2026-04-02 05:01:30,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:01:30,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb55248c06c2799bf5826a HTTP/1.1" 200 None 2026-04-02 05:01:30,367 - lava_callback - DEBUG - Processing node 69cb55248c06c2799bf5826a: baseline-arm64-baylibre 2026-04-02 05:01:30,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb55248c06c2799bf5826a 2026-04-02 05:01:30,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:01:30,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb55248c06c2799bf5826a HTTP/1.1" 200 None 2026-04-02 05:01:30,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:01:30,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:01:30,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:01:30,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb55248c06c2799bf5826a HTTP/1.1" 200 None 2026-04-02 05:01:30,460 - lava_callback - ERROR - Error processing callback for node 69cb55248c06c2799bf5826a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44196 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:03:50,452 - lava_callback - INFO - Received callback for node 69cb55ad8c06c2799bf5828f INFO: 172.20.0.1:54182 - "POST /node/69cb55ad8c06c2799bf5828f HTTP/1.0" 202 Accepted 2026-04-02 05:03:50,458 - lava_callback - INFO - Starting async job submit for node 69cb55ad8c06c2799bf5828f 2026-04-02 05:03:50,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:03:50,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb55ad8c06c2799bf5828f HTTP/1.1" 200 None 2026-04-02 05:03:50,473 - lava_callback - DEBUG - Processing node 69cb55ad8c06c2799bf5828f: baseline-arm-baylibre 2026-04-02 05:03:50,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb55ad8c06c2799bf5828f 2026-04-02 05:03:50,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:03:50,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb55ad8c06c2799bf5828f HTTP/1.1" 200 None 2026-04-02 05:03:50,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:03:50,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:03:50,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:03:50,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb55ad8c06c2799bf5828f HTTP/1.1" 200 None 2026-04-02 05:03:50,531 - lava_callback - ERROR - Error processing callback for node 69cb55ad8c06c2799bf5828f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37556 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:04:50,489 - lava_callback - INFO - Received callback for node 69cb55dd8c06c2799bf582a2 INFO: 172.20.0.1:56422 - "POST /node/69cb55dd8c06c2799bf582a2 HTTP/1.0" 202 Accepted 2026-04-02 05:04:50,495 - lava_callback - INFO - Starting async job submit for node 69cb55dd8c06c2799bf582a2 2026-04-02 05:04:50,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:04:50,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb55dd8c06c2799bf582a2 HTTP/1.1" 200 None 2026-04-02 05:04:50,512 - lava_callback - DEBUG - Processing node 69cb55dd8c06c2799bf582a2: baseline-arm64-baylibre 2026-04-02 05:04:50,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb55dd8c06c2799bf582a2 2026-04-02 05:04:50,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:04:50,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb55dd8c06c2799bf582a2 HTTP/1.1" 200 None 2026-04-02 05:04:50,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:04:50,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:04:50,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:04:50,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb55dd8c06c2799bf582a2 HTTP/1.1" 200 None 2026-04-02 05:04:50,573 - lava_callback - ERROR - Error processing callback for node 69cb55dd8c06c2799bf582a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52428 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:05:50,567 - lava_callback - INFO - Received callback for node 69cb561f8c06c2799bf582a7 INFO: 172.20.0.1:49560 - "POST /node/69cb561f8c06c2799bf582a7 HTTP/1.0" 202 Accepted 2026-04-02 05:05:50,573 - lava_callback - INFO - Starting async job submit for node 69cb561f8c06c2799bf582a7 2026-04-02 05:05:50,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:05:50,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb561f8c06c2799bf582a7 HTTP/1.1" 200 None 2026-04-02 05:05:50,589 - lava_callback - DEBUG - Processing node 69cb561f8c06c2799bf582a7: baseline-arm-baylibre 2026-04-02 05:05:50,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb561f8c06c2799bf582a7 2026-04-02 05:05:50,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:05:50,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb561f8c06c2799bf582a7 HTTP/1.1" 200 None 2026-04-02 05:05:50,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:05:50,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:05:50,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:05:50,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb561f8c06c2799bf582a7 HTTP/1.1" 200 None 2026-04-02 05:05:50,646 - lava_callback - ERROR - Error processing callback for node 69cb561f8c06c2799bf582a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59988 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:06:30,559 - lava_callback - INFO - Received callback for node 69cb56458c06c2799bf582b6 INFO: 172.20.0.1:46972 - "POST /node/69cb56458c06c2799bf582b6 HTTP/1.0" 202 Accepted 2026-04-02 05:06:30,565 - lava_callback - INFO - Starting async job submit for node 69cb56458c06c2799bf582b6 2026-04-02 05:06:30,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:06:30,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb56458c06c2799bf582b6 HTTP/1.1" 200 None 2026-04-02 05:06:30,582 - lava_callback - DEBUG - Processing node 69cb56458c06c2799bf582b6: baseline-arm-baylibre 2026-04-02 05:06:30,582 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb56458c06c2799bf582b6 2026-04-02 05:06:30,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:06:30,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb56458c06c2799bf582b6 HTTP/1.1" 200 None 2026-04-02 05:06:30,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:06:30,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:06:30,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:06:30,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb56458c06c2799bf582b6 HTTP/1.1" 200 None 2026-04-02 05:06:30,642 - lava_callback - ERROR - Error processing callback for node 69cb56458c06c2799bf582b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40058 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:07:10,605 - lava_callback - INFO - Received callback for node 69cb566a8c06c2799bf582c2 INFO: 172.20.0.1:34012 - "POST /node/69cb566a8c06c2799bf582c2 HTTP/1.0" 202 Accepted 2026-04-02 05:07:10,611 - lava_callback - INFO - Starting async job submit for node 69cb566a8c06c2799bf582c2 2026-04-02 05:07:10,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:07:10,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb566a8c06c2799bf582c2 HTTP/1.1" 200 None 2026-04-02 05:07:10,627 - lava_callback - DEBUG - Processing node 69cb566a8c06c2799bf582c2: baseline-arm-baylibre 2026-04-02 05:07:10,627 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb566a8c06c2799bf582c2 2026-04-02 05:07:10,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:07:10,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb566a8c06c2799bf582c2 HTTP/1.1" 200 None 2026-04-02 05:07:10,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:07:10,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:07:10,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:07:10,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb566a8c06c2799bf582c2 HTTP/1.1" 200 None 2026-04-02 05:07:10,697 - lava_callback - ERROR - Error processing callback for node 69cb566a8c06c2799bf582c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36366 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:07:30,520 - lava_callback - INFO - Received callback for node 69cb56908c06c2799bf582cd INFO: 172.20.0.1:39770 - "POST /node/69cb56908c06c2799bf582cd HTTP/1.0" 202 Accepted 2026-04-02 05:07:30,525 - lava_callback - INFO - Starting async job submit for node 69cb56908c06c2799bf582cd 2026-04-02 05:07:30,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:07:30,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb56908c06c2799bf582cd HTTP/1.1" 200 None 2026-04-02 05:07:30,542 - lava_callback - DEBUG - Processing node 69cb56908c06c2799bf582cd: baseline-arm-baylibre 2026-04-02 05:07:30,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb56908c06c2799bf582cd 2026-04-02 05:07:30,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:07:30,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb56908c06c2799bf582cd HTTP/1.1" 200 None 2026-04-02 05:07:30,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:07:30,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:07:30,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:07:30,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb56908c06c2799bf582cd HTTP/1.1" 200 None 2026-04-02 05:07:30,606 - lava_callback - ERROR - Error processing callback for node 69cb56908c06c2799bf582cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48082 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:08:30,649 - lava_callback - INFO - Received callback for node 69cb56bb8c06c2799bf582df INFO: 172.20.0.1:33670 - "POST /node/69cb56bb8c06c2799bf582df HTTP/1.0" 202 Accepted 2026-04-02 05:08:30,654 - lava_callback - INFO - Starting async job submit for node 69cb56bb8c06c2799bf582df 2026-04-02 05:08:30,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:08:30,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb56bb8c06c2799bf582df HTTP/1.1" 200 None 2026-04-02 05:08:30,691 - lava_callback - DEBUG - Processing node 69cb56bb8c06c2799bf582df: baseline-arm64-baylibre 2026-04-02 05:08:30,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb56bb8c06c2799bf582df 2026-04-02 05:08:30,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:08:30,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb56bb8c06c2799bf582df HTTP/1.1" 200 None 2026-04-02 05:08:30,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:08:30,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:08:30,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:08:30,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb56bb8c06c2799bf582df HTTP/1.1" 200 None 2026-04-02 05:08:30,760 - lava_callback - ERROR - Error processing callback for node 69cb56bb8c06c2799bf582df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38066 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:09:30,648 - lava_callback - INFO - Received callback for node 69cb57028c06c2799bf582eb INFO: 172.20.0.1:50520 - "POST /node/69cb57028c06c2799bf582eb HTTP/1.0" 202 Accepted 2026-04-02 05:09:30,653 - lava_callback - INFO - Starting async job submit for node 69cb57028c06c2799bf582eb 2026-04-02 05:09:30,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:09:30,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb57028c06c2799bf582eb HTTP/1.1" 200 None 2026-04-02 05:09:30,669 - lava_callback - DEBUG - Processing node 69cb57028c06c2799bf582eb: baseline-arm-baylibre 2026-04-02 05:09:30,669 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb57028c06c2799bf582eb 2026-04-02 05:09:30,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:09:30,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb57028c06c2799bf582eb HTTP/1.1" 200 None 2026-04-02 05:09:30,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:09:30,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:09:30,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:09:30,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb57028c06c2799bf582eb HTTP/1.1" 200 None 2026-04-02 05:09:30,723 - lava_callback - ERROR - Error processing callback for node 69cb57028c06c2799bf582eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57796 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:10:10,737 - lava_callback - INFO - Received callback for node 69cb572f8c06c2799bf582ff INFO: 172.20.0.1:39646 - "POST /node/69cb572f8c06c2799bf582ff HTTP/1.0" 202 Accepted 2026-04-02 05:10:10,743 - lava_callback - INFO - Starting async job submit for node 69cb572f8c06c2799bf582ff 2026-04-02 05:10:10,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:10:10,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb572f8c06c2799bf582ff HTTP/1.1" 200 None 2026-04-02 05:10:10,760 - lava_callback - DEBUG - Processing node 69cb572f8c06c2799bf582ff: baseline-arm64-baylibre 2026-04-02 05:10:10,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb572f8c06c2799bf582ff 2026-04-02 05:10:10,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:10:10,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb572f8c06c2799bf582ff HTTP/1.1" 200 None 2026-04-02 05:10:10,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:10:10,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:10:10,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:10:10,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb572f8c06c2799bf582ff HTTP/1.1" 200 None 2026-04-02 05:10:10,815 - lava_callback - ERROR - Error processing callback for node 69cb572f8c06c2799bf582ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41590 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:11:30,766 - lava_callback - INFO - Received callback for node 69cb577c8c06c2799bf58309 INFO: 172.20.0.1:36424 - "POST /node/69cb577c8c06c2799bf58309 HTTP/1.0" 202 Accepted 2026-04-02 05:11:30,771 - lava_callback - INFO - Starting async job submit for node 69cb577c8c06c2799bf58309 2026-04-02 05:11:30,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:11:30,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb577c8c06c2799bf58309 HTTP/1.1" 200 None 2026-04-02 05:11:30,787 - lava_callback - DEBUG - Processing node 69cb577c8c06c2799bf58309: baseline-arm64-baylibre 2026-04-02 05:11:30,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb577c8c06c2799bf58309 2026-04-02 05:11:30,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:11:30,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb577c8c06c2799bf58309 HTTP/1.1" 200 None 2026-04-02 05:11:30,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:11:30,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:11:30,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:11:30,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb577c8c06c2799bf58309 HTTP/1.1" 200 None 2026-04-02 05:11:30,857 - lava_callback - ERROR - Error processing callback for node 69cb577c8c06c2799bf58309: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36302 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:14:01,785 - lava_callback - INFO - Received callback for node 69cb580d8c06c2799bf58335 INFO: 172.20.0.1:34248 - "POST /node/69cb580d8c06c2799bf58335 HTTP/1.0" 202 Accepted 2026-04-02 05:14:01,791 - lava_callback - INFO - Starting async job submit for node 69cb580d8c06c2799bf58335 2026-04-02 05:14:01,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:14:01,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb580d8c06c2799bf58335 HTTP/1.1" 200 None 2026-04-02 05:14:01,807 - lava_callback - DEBUG - Processing node 69cb580d8c06c2799bf58335: baseline-arm-clabbe 2026-04-02 05:14:01,807 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb580d8c06c2799bf58335 2026-04-02 05:14:01,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:14:01,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb580d8c06c2799bf58335 HTTP/1.1" 200 None 2026-04-02 05:14:01,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:14:01,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:14:01,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:14:01,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb580d8c06c2799bf58335 HTTP/1.1" 200 None 2026-04-02 05:14:01,858 - lava_callback - ERROR - Error processing callback for node 69cb580d8c06c2799bf58335: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56232 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:14:51,083 - lava_callback - INFO - Received callback for node 69cb58388c06c2799bf58346 INFO: 172.20.0.1:45492 - "POST /node/69cb58388c06c2799bf58346 HTTP/1.0" 202 Accepted 2026-04-02 05:14:51,088 - lava_callback - INFO - Starting async job submit for node 69cb58388c06c2799bf58346 2026-04-02 05:14:51,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:14:51,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb58388c06c2799bf58346 HTTP/1.1" 200 None 2026-04-02 05:14:51,104 - lava_callback - DEBUG - Processing node 69cb58388c06c2799bf58346: baseline-arm64-baylibre 2026-04-02 05:14:51,104 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb58388c06c2799bf58346 2026-04-02 05:14:51,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:14:51,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb58388c06c2799bf58346 HTTP/1.1" 200 None 2026-04-02 05:14:51,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:14:51,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:14:51,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:14:51,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb58388c06c2799bf58346 HTTP/1.1" 200 None 2026-04-02 05:14:51,169 - lava_callback - ERROR - Error processing callback for node 69cb58388c06c2799bf58346: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45246 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:16:01,744 - lava_callback - INFO - Received callback for node 69cb58808c06c2799bf58352 INFO: 172.20.0.1:40610 - "POST /node/69cb58808c06c2799bf58352 HTTP/1.0" 202 Accepted 2026-04-02 05:16:01,750 - lava_callback - INFO - Starting async job submit for node 69cb58808c06c2799bf58352 2026-04-02 05:16:01,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:16:01,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb58808c06c2799bf58352 HTTP/1.1" 200 None 2026-04-02 05:16:01,793 - lava_callback - DEBUG - Processing node 69cb58808c06c2799bf58352: baseline-arm-clabbe 2026-04-02 05:16:01,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb58808c06c2799bf58352 2026-04-02 05:16:01,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:16:01,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb58808c06c2799bf58352 HTTP/1.1" 200 None 2026-04-02 05:16:01,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:16:01,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:16:01,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:16:01,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb58808c06c2799bf58352 HTTP/1.1" 200 None 2026-04-02 05:16:01,885 - lava_callback - ERROR - Error processing callback for node 69cb58808c06c2799bf58352: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59752 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:16:41,754 - lava_callback - INFO - Received callback for node 69cb58a58c06c2799bf5835f INFO: 172.20.0.1:34254 - "POST /node/69cb58a58c06c2799bf5835f HTTP/1.0" 202 Accepted 2026-04-02 05:16:41,760 - lava_callback - INFO - Starting async job submit for node 69cb58a58c06c2799bf5835f 2026-04-02 05:16:41,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:16:41,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb58a58c06c2799bf5835f HTTP/1.1" 200 None 2026-04-02 05:16:41,776 - lava_callback - DEBUG - Processing node 69cb58a58c06c2799bf5835f: baseline-arm-clabbe 2026-04-02 05:16:41,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb58a58c06c2799bf5835f 2026-04-02 05:16:41,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:16:41,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb58a58c06c2799bf5835f HTTP/1.1" 200 None 2026-04-02 05:16:41,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:16:41,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:16:41,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:16:41,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb58a58c06c2799bf5835f HTTP/1.1" 200 None 2026-04-02 05:16:41,859 - lava_callback - ERROR - Error processing callback for node 69cb58a58c06c2799bf5835f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48930 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:17:21,817 - lava_callback - INFO - Received callback for node 69cb58d98c06c2799bf5836f INFO: 172.20.0.1:47536 - "POST /node/69cb58d98c06c2799bf5836f HTTP/1.0" 202 Accepted 2026-04-02 05:17:21,822 - lava_callback - INFO - Starting async job submit for node 69cb58d98c06c2799bf5836f 2026-04-02 05:17:21,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:17:21,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb58d98c06c2799bf5836f HTTP/1.1" 200 None 2026-04-02 05:17:21,838 - lava_callback - DEBUG - Processing node 69cb58d98c06c2799bf5836f: baseline-arm64-clabbe 2026-04-02 05:17:21,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb58d98c06c2799bf5836f 2026-04-02 05:17:21,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:17:21,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb58d98c06c2799bf5836f HTTP/1.1" 200 None 2026-04-02 05:17:21,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:17:21,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:17:21,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:17:21,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb58d98c06c2799bf5836f HTTP/1.1" 200 None 2026-04-02 05:17:21,891 - lava_callback - ERROR - Error processing callback for node 69cb58d98c06c2799bf5836f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42806 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:18:21,863 - lava_callback - INFO - Received callback for node 69cb59188c06c2799bf5837d INFO: 172.20.0.1:43470 - "POST /node/69cb59188c06c2799bf5837d HTTP/1.0" 202 Accepted 2026-04-02 05:18:21,869 - lava_callback - INFO - Starting async job submit for node 69cb59188c06c2799bf5837d 2026-04-02 05:18:21,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:18:21,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb59188c06c2799bf5837d HTTP/1.1" 200 None 2026-04-02 05:18:21,884 - lava_callback - DEBUG - Processing node 69cb59188c06c2799bf5837d: baseline-arm-clabbe 2026-04-02 05:18:21,884 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb59188c06c2799bf5837d 2026-04-02 05:18:21,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:18:21,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb59188c06c2799bf5837d HTTP/1.1" 200 None 2026-04-02 05:18:21,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:18:21,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:18:21,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:18:21,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb59188c06c2799bf5837d HTTP/1.1" 200 None 2026-04-02 05:18:21,963 - lava_callback - ERROR - Error processing callback for node 69cb59188c06c2799bf5837d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52920 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:19:21,955 - lava_callback - INFO - Received callback for node 69cb594c8c06c2799bf58392 INFO: 172.20.0.1:45310 - "POST /node/69cb594c8c06c2799bf58392 HTTP/1.0" 202 Accepted 2026-04-02 05:19:21,961 - lava_callback - INFO - Starting async job submit for node 69cb594c8c06c2799bf58392 2026-04-02 05:19:21,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:19:21,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb594c8c06c2799bf58392 HTTP/1.1" 200 None 2026-04-02 05:19:21,976 - lava_callback - DEBUG - Processing node 69cb594c8c06c2799bf58392: baseline-arm64-clabbe 2026-04-02 05:19:21,976 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb594c8c06c2799bf58392 2026-04-02 05:19:21,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:19:22,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb594c8c06c2799bf58392 HTTP/1.1" 200 None 2026-04-02 05:19:22,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:19:22,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:19:22,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:19:22,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb594c8c06c2799bf58392 HTTP/1.1" 200 None 2026-04-02 05:19:22,047 - lava_callback - ERROR - Error processing callback for node 69cb594c8c06c2799bf58392: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60666 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:20:21,985 - lava_callback - INFO - Received callback for node 69cb598d8c06c2799bf58399 INFO: 172.20.0.1:52900 - "POST /node/69cb598d8c06c2799bf58399 HTTP/1.0" 202 Accepted 2026-04-02 05:20:21,990 - lava_callback - INFO - Starting async job submit for node 69cb598d8c06c2799bf58399 2026-04-02 05:20:21,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:20:22,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb598d8c06c2799bf58399 HTTP/1.1" 200 None 2026-04-02 05:20:22,009 - lava_callback - DEBUG - Processing node 69cb598d8c06c2799bf58399: baseline-arm-clabbe 2026-04-02 05:20:22,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb598d8c06c2799bf58399 2026-04-02 05:20:22,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:20:22,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb598d8c06c2799bf58399 HTTP/1.1" 200 None 2026-04-02 05:20:22,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:20:22,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-02 05:20:22,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:20:22,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb598d8c06c2799bf58399 HTTP/1.1" 200 None 2026-04-02 05:20:22,066 - lava_callback - ERROR - Error processing callback for node 69cb598d8c06c2799bf58399: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56526 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:21:22,032 - lava_callback - INFO - Received callback for node 69cb59c18c06c2799bf583af INFO: 172.20.0.1:37598 - "POST /node/69cb59c18c06c2799bf583af HTTP/1.0" 202 Accepted 2026-04-02 05:21:22,038 - lava_callback - INFO - Starting async job submit for node 69cb59c18c06c2799bf583af 2026-04-02 05:21:22,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:21:22,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb59c18c06c2799bf583af HTTP/1.1" 200 None 2026-04-02 05:21:22,054 - lava_callback - DEBUG - Processing node 69cb59c18c06c2799bf583af: baseline-arm64-clabbe 2026-04-02 05:21:22,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb59c18c06c2799bf583af 2026-04-02 05:21:22,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:21:22,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb59c18c06c2799bf583af HTTP/1.1" 200 None 2026-04-02 05:21:22,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:21:22,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:21:22,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:21:22,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb59c18c06c2799bf583af HTTP/1.1" 200 None 2026-04-02 05:21:22,104 - lava_callback - ERROR - Error processing callback for node 69cb59c18c06c2799bf583af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47364 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:22:42,064 - lava_callback - INFO - Received callback for node 69cb5a0f8c06c2799bf583b9 INFO: 172.20.0.1:52836 - "POST /node/69cb5a0f8c06c2799bf583b9 HTTP/1.0" 202 Accepted 2026-04-02 05:22:42,070 - lava_callback - INFO - Starting async job submit for node 69cb5a0f8c06c2799bf583b9 2026-04-02 05:22:42,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:22:42,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5a0f8c06c2799bf583b9 HTTP/1.1" 200 None 2026-04-02 05:22:42,085 - lava_callback - DEBUG - Processing node 69cb5a0f8c06c2799bf583b9: baseline-arm64-clabbe 2026-04-02 05:22:42,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5a0f8c06c2799bf583b9 2026-04-02 05:22:42,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:22:42,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5a0f8c06c2799bf583b9 HTTP/1.1" 200 None 2026-04-02 05:22:42,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:22:42,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:22:42,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:22:42,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5a0f8c06c2799bf583b9 HTTP/1.1" 200 None 2026-04-02 05:22:42,148 - lava_callback - ERROR - Error processing callback for node 69cb5a0f8c06c2799bf583b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50236 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:23:51,451 - lava_callback - INFO - Received callback for node 69cb5a588c06c2799bf583cc INFO: 172.20.0.1:59896 - "POST /node/69cb5a588c06c2799bf583cc HTTP/1.0" 202 Accepted 2026-04-02 05:23:51,457 - lava_callback - INFO - Starting async job submit for node 69cb5a588c06c2799bf583cc 2026-04-02 05:23:51,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:23:51,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5a588c06c2799bf583cc HTTP/1.1" 200 None 2026-04-02 05:23:51,472 - lava_callback - DEBUG - Processing node 69cb5a588c06c2799bf583cc: baseline-arm64-baylibre 2026-04-02 05:23:51,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5a588c06c2799bf583cc 2026-04-02 05:23:51,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:23:51,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5a588c06c2799bf583cc HTTP/1.1" 200 None 2026-04-02 05:23:51,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:23:51,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:23:51,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:23:51,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5a588c06c2799bf583cc HTTP/1.1" 200 None 2026-04-02 05:23:51,544 - lava_callback - ERROR - Error processing callback for node 69cb5a588c06c2799bf583cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58596 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:25:11,467 - lava_callback - INFO - Received callback for node 69cb5aa78c06c2799bf583d9 INFO: 172.20.0.1:52018 - "POST /node/69cb5aa78c06c2799bf583d9 HTTP/1.0" 202 Accepted 2026-04-02 05:25:11,473 - lava_callback - INFO - Starting async job submit for node 69cb5aa78c06c2799bf583d9 2026-04-02 05:25:11,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:25:11,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5aa78c06c2799bf583d9 HTTP/1.1" 200 None 2026-04-02 05:25:11,497 - lava_callback - DEBUG - Processing node 69cb5aa78c06c2799bf583d9: baseline-arm64-baylibre 2026-04-02 05:25:11,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5aa78c06c2799bf583d9 2026-04-02 05:25:11,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:25:11,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5aa78c06c2799bf583d9 HTTP/1.1" 200 None 2026-04-02 05:25:11,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:25:11,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:25:11,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:25:11,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5aa78c06c2799bf583d9 HTTP/1.1" 200 None 2026-04-02 05:25:11,562 - lava_callback - ERROR - Error processing callback for node 69cb5aa78c06c2799bf583d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43532 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:26:22,204 - lava_callback - INFO - Received callback for node 69cb5afb8c06c2799bf583ea INFO: 172.20.0.1:38340 - "POST /node/69cb5afb8c06c2799bf583ea HTTP/1.0" 202 Accepted 2026-04-02 05:26:22,209 - lava_callback - INFO - Starting async job submit for node 69cb5afb8c06c2799bf583ea 2026-04-02 05:26:22,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:26:22,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5afb8c06c2799bf583ea HTTP/1.1" 200 None 2026-04-02 05:26:22,231 - lava_callback - DEBUG - Processing node 69cb5afb8c06c2799bf583ea: baseline-arm64-clabbe 2026-04-02 05:26:22,232 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5afb8c06c2799bf583ea 2026-04-02 05:26:22,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:26:22,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5afb8c06c2799bf583ea HTTP/1.1" 200 None 2026-04-02 05:26:22,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:26:22,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:26:22,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:26:22,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5afb8c06c2799bf583ea HTTP/1.1" 200 None 2026-04-02 05:26:22,287 - lava_callback - ERROR - Error processing callback for node 69cb5afb8c06c2799bf583ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42964 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:27:42,263 - lava_callback - INFO - Received callback for node 69cb5b498c06c2799bf583fe INFO: 172.20.0.1:53072 - "POST /node/69cb5b498c06c2799bf583fe HTTP/1.0" 202 Accepted 2026-04-02 05:27:42,270 - lava_callback - INFO - Starting async job submit for node 69cb5b498c06c2799bf583fe 2026-04-02 05:27:42,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:27:42,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5b498c06c2799bf583fe HTTP/1.1" 200 None 2026-04-02 05:27:42,294 - lava_callback - DEBUG - Processing node 69cb5b498c06c2799bf583fe: baseline-arm64-clabbe 2026-04-02 05:27:42,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5b498c06c2799bf583fe 2026-04-02 05:27:42,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:27:42,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5b498c06c2799bf583fe HTTP/1.1" 200 None 2026-04-02 05:27:42,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:27:42,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:27:42,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:27:42,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5b498c06c2799bf583fe HTTP/1.1" 200 None 2026-04-02 05:27:42,392 - lava_callback - ERROR - Error processing callback for node 69cb5b498c06c2799bf583fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39808 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:29:02,370 - lava_callback - INFO - Received callback for node 69cb5b998c06c2799bf58409 INFO: 172.20.0.1:58520 - "POST /node/69cb5b998c06c2799bf58409 HTTP/1.0" 202 Accepted 2026-04-02 05:29:02,376 - lava_callback - INFO - Starting async job submit for node 69cb5b998c06c2799bf58409 2026-04-02 05:29:02,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:29:02,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5b998c06c2799bf58409 HTTP/1.1" 200 None 2026-04-02 05:29:02,393 - lava_callback - DEBUG - Processing node 69cb5b998c06c2799bf58409: baseline-arm64-clabbe 2026-04-02 05:29:02,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5b998c06c2799bf58409 2026-04-02 05:29:02,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:29:02,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5b998c06c2799bf58409 HTTP/1.1" 200 None 2026-04-02 05:29:02,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:29:02,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:29:02,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:29:02,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5b998c06c2799bf58409 HTTP/1.1" 200 None 2026-04-02 05:29:02,449 - lava_callback - ERROR - Error processing callback for node 69cb5b998c06c2799bf58409: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54734 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:31:42,508 - lava_callback - INFO - Received callback for node 69cb5c398c06c2799bf5842a INFO: 172.20.0.1:47136 - "POST /node/69cb5c398c06c2799bf5842a HTTP/1.0" 202 Accepted 2026-04-02 05:31:42,514 - lava_callback - INFO - Starting async job submit for node 69cb5c398c06c2799bf5842a 2026-04-02 05:31:42,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:31:42,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5c398c06c2799bf5842a HTTP/1.1" 200 None 2026-04-02 05:31:42,529 - lava_callback - DEBUG - Processing node 69cb5c398c06c2799bf5842a: baseline-arm64-clabbe 2026-04-02 05:31:42,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5c398c06c2799bf5842a 2026-04-02 05:31:42,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:31:42,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5c398c06c2799bf5842a HTTP/1.1" 200 None 2026-04-02 05:31:42,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:31:42,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:31:42,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:31:42,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5c398c06c2799bf5842a HTTP/1.1" 200 None 2026-04-02 05:31:42,579 - lava_callback - ERROR - Error processing callback for node 69cb5c398c06c2799bf5842a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41426 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:33:02,553 - lava_callback - INFO - Received callback for node 69cb5c888c06c2799bf58436 INFO: 172.20.0.1:55254 - "POST /node/69cb5c888c06c2799bf58436 HTTP/1.0" 202 Accepted 2026-04-02 05:33:02,559 - lava_callback - INFO - Starting async job submit for node 69cb5c888c06c2799bf58436 2026-04-02 05:33:02,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:33:02,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5c888c06c2799bf58436 HTTP/1.1" 200 None 2026-04-02 05:33:02,574 - lava_callback - DEBUG - Processing node 69cb5c888c06c2799bf58436: baseline-arm64-clabbe 2026-04-02 05:33:02,575 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5c888c06c2799bf58436 2026-04-02 05:33:02,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:33:02,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5c888c06c2799bf58436 HTTP/1.1" 200 None 2026-04-02 05:33:02,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:33:02,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:33:02,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:33:02,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5c888c06c2799bf58436 HTTP/1.1" 200 None 2026-04-02 05:33:02,625 - lava_callback - ERROR - Error processing callback for node 69cb5c888c06c2799bf58436: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45546 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:34:22,651 - lava_callback - INFO - Received callback for node 69cb5cd98c06c2799bf58448 INFO: 172.20.0.1:59108 - "POST /node/69cb5cd98c06c2799bf58448 HTTP/1.0" 202 Accepted 2026-04-02 05:34:22,657 - lava_callback - INFO - Starting async job submit for node 69cb5cd98c06c2799bf58448 2026-04-02 05:34:22,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:34:22,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5cd98c06c2799bf58448 HTTP/1.1" 200 None 2026-04-02 05:34:22,672 - lava_callback - DEBUG - Processing node 69cb5cd98c06c2799bf58448: baseline-arm64-clabbe 2026-04-02 05:34:22,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5cd98c06c2799bf58448 2026-04-02 05:34:22,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:34:22,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5cd98c06c2799bf58448 HTTP/1.1" 200 None 2026-04-02 05:34:22,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:34:22,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:34:22,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:34:22,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5cd98c06c2799bf58448 HTTP/1.1" 200 None 2026-04-02 05:34:22,724 - lava_callback - ERROR - Error processing callback for node 69cb5cd98c06c2799bf58448: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52310 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:35:42,705 - lava_callback - INFO - Received callback for node 69cb5d298c06c2799bf58458 INFO: 172.20.0.1:49162 - "POST /node/69cb5d298c06c2799bf58458 HTTP/1.0" 202 Accepted 2026-04-02 05:35:42,710 - lava_callback - INFO - Starting async job submit for node 69cb5d298c06c2799bf58458 2026-04-02 05:35:42,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:35:42,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5d298c06c2799bf58458 HTTP/1.1" 200 None 2026-04-02 05:35:42,727 - lava_callback - DEBUG - Processing node 69cb5d298c06c2799bf58458: baseline-arm64-clabbe 2026-04-02 05:35:42,727 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5d298c06c2799bf58458 2026-04-02 05:35:42,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:35:42,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5d298c06c2799bf58458 HTTP/1.1" 200 None 2026-04-02 05:35:42,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:35:42,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:35:42,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:35:42,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5d298c06c2799bf58458 HTTP/1.1" 200 None 2026-04-02 05:35:42,792 - lava_callback - ERROR - Error processing callback for node 69cb5d298c06c2799bf58458: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33926 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:37:02,789 - lava_callback - INFO - Received callback for node 69cb5d788c06c2799bf58466 INFO: 172.20.0.1:47850 - "POST /node/69cb5d788c06c2799bf58466 HTTP/1.0" 202 Accepted 2026-04-02 05:37:02,795 - lava_callback - INFO - Starting async job submit for node 69cb5d788c06c2799bf58466 2026-04-02 05:37:02,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:37:02,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5d788c06c2799bf58466 HTTP/1.1" 200 None 2026-04-02 05:37:02,811 - lava_callback - DEBUG - Processing node 69cb5d788c06c2799bf58466: baseline-arm64-clabbe 2026-04-02 05:37:02,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5d788c06c2799bf58466 2026-04-02 05:37:02,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:37:02,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5d788c06c2799bf58466 HTTP/1.1" 200 None 2026-04-02 05:37:02,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:37:02,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:37:02,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:37:02,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5d788c06c2799bf58466 HTTP/1.1" 200 None 2026-04-02 05:37:02,875 - lava_callback - ERROR - Error processing callback for node 69cb5d788c06c2799bf58466: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43074 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:38:22,771 - lava_callback - INFO - Received callback for node 69cb5dc68c06c2799bf58478 INFO: 172.20.0.1:44392 - "POST /node/69cb5dc68c06c2799bf58478 HTTP/1.0" 202 Accepted 2026-04-02 05:38:22,777 - lava_callback - INFO - Starting async job submit for node 69cb5dc68c06c2799bf58478 2026-04-02 05:38:22,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:38:22,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5dc68c06c2799bf58478 HTTP/1.1" 200 None 2026-04-02 05:38:22,799 - lava_callback - DEBUG - Processing node 69cb5dc68c06c2799bf58478: baseline-arm64-clabbe 2026-04-02 05:38:22,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5dc68c06c2799bf58478 2026-04-02 05:38:22,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:38:22,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5dc68c06c2799bf58478 HTTP/1.1" 200 None 2026-04-02 05:38:22,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:38:22,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:38:22,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:38:22,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5dc68c06c2799bf58478 HTTP/1.1" 200 None 2026-04-02 05:38:22,883 - lava_callback - ERROR - Error processing callback for node 69cb5dc68c06c2799bf58478: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60860 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:39:43,051 - lava_callback - INFO - Received callback for node 69cb5e168c06c2799bf58489 INFO: 172.20.0.1:59578 - "POST /node/69cb5e168c06c2799bf58489 HTTP/1.0" 202 Accepted 2026-04-02 05:39:43,057 - lava_callback - INFO - Starting async job submit for node 69cb5e168c06c2799bf58489 2026-04-02 05:39:43,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:39:43,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5e168c06c2799bf58489 HTTP/1.1" 200 None 2026-04-02 05:39:43,073 - lava_callback - DEBUG - Processing node 69cb5e168c06c2799bf58489: baseline-arm64-clabbe 2026-04-02 05:39:43,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5e168c06c2799bf58489 2026-04-02 05:39:43,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:39:43,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5e168c06c2799bf58489 HTTP/1.1" 200 None 2026-04-02 05:39:43,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:39:43,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:39:43,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:39:43,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5e168c06c2799bf58489 HTTP/1.1" 200 None 2026-04-02 05:39:43,124 - lava_callback - ERROR - Error processing callback for node 69cb5e168c06c2799bf58489: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36308 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:41:02,878 - lava_callback - INFO - Received callback for node 69cb5e658c06c2799bf58497 INFO: 172.20.0.1:51616 - "POST /node/69cb5e658c06c2799bf58497 HTTP/1.0" 202 Accepted 2026-04-02 05:41:02,883 - lava_callback - INFO - Starting async job submit for node 69cb5e658c06c2799bf58497 2026-04-02 05:41:02,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:41:02,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5e658c06c2799bf58497 HTTP/1.1" 200 None 2026-04-02 05:41:02,899 - lava_callback - DEBUG - Processing node 69cb5e658c06c2799bf58497: baseline-arm64-clabbe 2026-04-02 05:41:02,899 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5e658c06c2799bf58497 2026-04-02 05:41:02,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:41:03,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5e658c06c2799bf58497 HTTP/1.1" 200 None 2026-04-02 05:41:03,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:41:03,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:41:03,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:41:03,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5e658c06c2799bf58497 HTTP/1.1" 200 None 2026-04-02 05:41:03,137 - lava_callback - ERROR - Error processing callback for node 69cb5e658c06c2799bf58497: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53552 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:43:43,054 - lava_callback - INFO - Received callback for node 69cb5f038c06c2799bf584ba INFO: 172.20.0.1:39698 - "POST /node/69cb5f038c06c2799bf584ba HTTP/1.0" 202 Accepted 2026-04-02 05:43:43,060 - lava_callback - INFO - Starting async job submit for node 69cb5f038c06c2799bf584ba 2026-04-02 05:43:43,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:43:43,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5f038c06c2799bf584ba HTTP/1.1" 200 None 2026-04-02 05:43:43,076 - lava_callback - DEBUG - Processing node 69cb5f038c06c2799bf584ba: baseline-arm64-clabbe 2026-04-02 05:43:43,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5f038c06c2799bf584ba 2026-04-02 05:43:43,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:43:43,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5f038c06c2799bf584ba HTTP/1.1" 200 None 2026-04-02 05:43:43,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:43:43,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:43:43,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:43:43,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5f038c06c2799bf584ba HTTP/1.1" 200 None 2026-04-02 05:43:43,141 - lava_callback - ERROR - Error processing callback for node 69cb5f038c06c2799bf584ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57968 - "GET /node/698f868b3013abef95d0217d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59224 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:46:12,619 - lava_callback - INFO - Received callback for node 69cb5f9a8c06c2799bf584db INFO: 172.20.0.1:44954 - "POST /node/69cb5f9a8c06c2799bf584db HTTP/1.0" 202 Accepted 2026-04-02 05:46:12,624 - lava_callback - INFO - Starting async job submit for node 69cb5f9a8c06c2799bf584db 2026-04-02 05:46:12,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:46:12,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5f9a8c06c2799bf584db HTTP/1.1" 200 None 2026-04-02 05:46:12,641 - lava_callback - DEBUG - Processing node 69cb5f9a8c06c2799bf584db: baseline-arm64-baylibre 2026-04-02 05:46:12,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5f9a8c06c2799bf584db 2026-04-02 05:46:12,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:46:12,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5f9a8c06c2799bf584db HTTP/1.1" 200 None 2026-04-02 05:46:12,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:46:12,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:46:12,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:46:12,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5f9a8c06c2799bf584db HTTP/1.1" 200 None 2026-04-02 05:46:12,707 - lava_callback - ERROR - Error processing callback for node 69cb5f9a8c06c2799bf584db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52982 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:47:43,214 - lava_callback - INFO - Received callback for node 69cb5fee8c06c2799bf584ea INFO: 172.20.0.1:34892 - "POST /node/69cb5fee8c06c2799bf584ea HTTP/1.0" 202 Accepted 2026-04-02 05:47:43,220 - lava_callback - INFO - Starting async job submit for node 69cb5fee8c06c2799bf584ea 2026-04-02 05:47:43,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:47:43,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5fee8c06c2799bf584ea HTTP/1.1" 200 None 2026-04-02 05:47:43,237 - lava_callback - DEBUG - Processing node 69cb5fee8c06c2799bf584ea: baseline-arm64-clabbe 2026-04-02 05:47:43,237 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb5fee8c06c2799bf584ea 2026-04-02 05:47:43,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:47:43,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb5fee8c06c2799bf584ea HTTP/1.1" 200 None 2026-04-02 05:47:43,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:47:43,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:47:43,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:47:43,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb5fee8c06c2799bf584ea HTTP/1.1" 200 None 2026-04-02 05:47:43,295 - lava_callback - ERROR - Error processing callback for node 69cb5fee8c06c2799bf584ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37500 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:51:23,325 - lava_callback - INFO - Received callback for node 69cb60d68c06c2799bf5851d INFO: 172.20.0.1:53594 - "POST /node/69cb60d68c06c2799bf5851d HTTP/1.0" 202 Accepted 2026-04-02 05:51:23,331 - lava_callback - INFO - Starting async job submit for node 69cb60d68c06c2799bf5851d 2026-04-02 05:51:23,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:51:23,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb60d68c06c2799bf5851d HTTP/1.1" 200 None 2026-04-02 05:51:23,347 - lava_callback - DEBUG - Processing node 69cb60d68c06c2799bf5851d: baseline-arm64-clabbe 2026-04-02 05:51:23,347 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb60d68c06c2799bf5851d 2026-04-02 05:51:23,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:51:23,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb60d68c06c2799bf5851d HTTP/1.1" 200 None 2026-04-02 05:51:23,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:51:23,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:51:23,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:51:23,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb60d68c06c2799bf5851d HTTP/1.1" 200 None 2026-04-02 05:51:23,402 - lava_callback - ERROR - Error processing callback for node 69cb60d68c06c2799bf5851d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45820 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:52:43,428 - lava_callback - INFO - Received callback for node 69cb61238c06c2799bf58532 INFO: 172.20.0.1:55806 - "POST /node/69cb61238c06c2799bf58532 HTTP/1.0" 202 Accepted 2026-04-02 05:52:43,433 - lava_callback - INFO - Starting async job submit for node 69cb61238c06c2799bf58532 2026-04-02 05:52:43,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:52:43,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb61238c06c2799bf58532 HTTP/1.1" 200 None 2026-04-02 05:52:43,449 - lava_callback - DEBUG - Processing node 69cb61238c06c2799bf58532: baseline-arm64-clabbe 2026-04-02 05:52:43,449 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb61238c06c2799bf58532 2026-04-02 05:52:43,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:52:43,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb61238c06c2799bf58532 HTTP/1.1" 200 None 2026-04-02 05:52:43,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:52:43,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:52:43,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:52:43,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb61238c06c2799bf58532 HTTP/1.1" 200 None 2026-04-02 05:52:43,510 - lava_callback - ERROR - Error processing callback for node 69cb61238c06c2799bf58532: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40222 - "GET /node/698af6973b3beb19e76171c6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40238 - "GET /node/6994339d5ae459b9df3d32fc%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41936 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:53:52,975 - lava_callback - INFO - Received callback for node 69cb61698c06c2799bf5853c INFO: 172.20.0.1:47532 - "POST /node/69cb61698c06c2799bf5853c HTTP/1.0" 202 Accepted 2026-04-02 05:53:52,981 - lava_callback - INFO - Starting async job submit for node 69cb61698c06c2799bf5853c 2026-04-02 05:53:52,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:53:53,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb61698c06c2799bf5853c HTTP/1.1" 200 None 2026-04-02 05:53:53,007 - lava_callback - DEBUG - Processing node 69cb61698c06c2799bf5853c: baseline-arm64-baylibre 2026-04-02 05:53:53,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb61698c06c2799bf5853c 2026-04-02 05:53:53,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:53:53,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb61698c06c2799bf5853c HTTP/1.1" 200 None 2026-04-02 05:53:53,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:53:53,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-02 05:53:53,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:53:53,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb61698c06c2799bf5853c HTTP/1.1" 200 None 2026-04-02 05:53:53,086 - lava_callback - ERROR - Error processing callback for node 69cb61698c06c2799bf5853c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60538 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:55:23,536 - lava_callback - INFO - Received callback for node 69cb61be8c06c2799bf58551 INFO: 172.20.0.1:34224 - "POST /node/69cb61be8c06c2799bf58551 HTTP/1.0" 202 Accepted 2026-04-02 05:55:23,542 - lava_callback - INFO - Starting async job submit for node 69cb61be8c06c2799bf58551 2026-04-02 05:55:23,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:55:23,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb61be8c06c2799bf58551 HTTP/1.1" 200 None 2026-04-02 05:55:23,565 - lava_callback - DEBUG - Processing node 69cb61be8c06c2799bf58551: baseline-arm64-clabbe 2026-04-02 05:55:23,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb61be8c06c2799bf58551 2026-04-02 05:55:23,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:55:23,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb61be8c06c2799bf58551 HTTP/1.1" 200 None 2026-04-02 05:55:23,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:55:23,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 05:55:23,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:55:23,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb61be8c06c2799bf58551 HTTP/1.1" 200 None 2026-04-02 05:55:23,627 - lava_callback - ERROR - Error processing callback for node 69cb61be8c06c2799bf58551: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39516 - "GET /node/69402e61575bc060ce155fdb%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:56:33,179 - lava_callback - INFO - Received callback for node 69cb62068c06c2799bf5855b INFO: 172.20.0.1:44578 - "POST /node/69cb62068c06c2799bf5855b HTTP/1.0" 202 Accepted 2026-04-02 05:56:33,185 - lava_callback - INFO - Starting async job submit for node 69cb62068c06c2799bf5855b 2026-04-02 05:56:33,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:56:33,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb62068c06c2799bf5855b HTTP/1.1" 200 None 2026-04-02 05:56:33,201 - lava_callback - DEBUG - Processing node 69cb62068c06c2799bf5855b: baseline-arm64-baylibre 2026-04-02 05:56:33,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb62068c06c2799bf5855b 2026-04-02 05:56:33,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:56:33,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb62068c06c2799bf5855b HTTP/1.1" 200 None 2026-04-02 05:56:33,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:56:33,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-02 05:56:33,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:56:33,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb62068c06c2799bf5855b HTTP/1.1" 200 None 2026-04-02 05:56:33,256 - lava_callback - ERROR - Error processing callback for node 69cb62068c06c2799bf5855b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48280 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:58:03,588 - lava_callback - INFO - Received callback for node 69cb62598c06c2799bf5856b INFO: 172.20.0.1:54092 - "POST /node/69cb62598c06c2799bf5856b HTTP/1.0" 202 Accepted 2026-04-02 05:58:03,598 - lava_callback - INFO - Starting async job submit for node 69cb62598c06c2799bf5856b 2026-04-02 05:58:03,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:58:03,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb62598c06c2799bf5856b HTTP/1.1" 200 None 2026-04-02 05:58:03,615 - lava_callback - DEBUG - Processing node 69cb62598c06c2799bf5856b: baseline-arm64-clabbe 2026-04-02 05:58:03,616 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb62598c06c2799bf5856b 2026-04-02 05:58:03,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:58:03,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb62598c06c2799bf5856b HTTP/1.1" 200 None 2026-04-02 05:58:03,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:58:03,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 05:58:03,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:58:03,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb62598c06c2799bf5856b HTTP/1.1" 200 None 2026-04-02 05:58:03,670 - lava_callback - ERROR - Error processing callback for node 69cb62598c06c2799bf5856b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60558 - "GET /node/693fda62575bc060ce14da3a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33414 - "GET / HTTP/1.0" 200 OK 2026-04-02 05:59:13,234 - lava_callback - INFO - Received callback for node 69cb62a38c06c2799bf5857a INFO: 172.20.0.1:40618 - "POST /node/69cb62a38c06c2799bf5857a HTTP/1.0" 202 Accepted 2026-04-02 05:59:13,240 - lava_callback - INFO - Starting async job submit for node 69cb62a38c06c2799bf5857a 2026-04-02 05:59:13,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:59:13,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb62a38c06c2799bf5857a HTTP/1.1" 200 None 2026-04-02 05:59:13,257 - lava_callback - DEBUG - Processing node 69cb62a38c06c2799bf5857a: baseline-arm64-baylibre 2026-04-02 05:59:13,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb62a38c06c2799bf5857a 2026-04-02 05:59:13,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:59:13,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb62a38c06c2799bf5857a HTTP/1.1" 200 None 2026-04-02 05:59:13,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:59:13,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-02 05:59:13,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 05:59:13,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb62a38c06c2799bf5857a HTTP/1.1" 200 None 2026-04-02 05:59:13,325 - lava_callback - ERROR - Error processing callback for node 69cb62a38c06c2799bf5857a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49156 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:00:43,701 - lava_callback - INFO - Received callback for node 69cb62fb8c06c2799bf58587 INFO: 172.20.0.1:56442 - "POST /node/69cb62fb8c06c2799bf58587 HTTP/1.0" 202 Accepted 2026-04-02 06:00:43,706 - lava_callback - INFO - Starting async job submit for node 69cb62fb8c06c2799bf58587 2026-04-02 06:00:43,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:00:43,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb62fb8c06c2799bf58587 HTTP/1.1" 200 None 2026-04-02 06:00:43,725 - lava_callback - DEBUG - Processing node 69cb62fb8c06c2799bf58587: baseline-arm64-clabbe 2026-04-02 06:00:43,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb62fb8c06c2799bf58587 2026-04-02 06:00:43,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:00:43,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb62fb8c06c2799bf58587 HTTP/1.1" 200 None 2026-04-02 06:00:43,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:00:43,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 06:00:43,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:00:43,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb62fb8c06c2799bf58587 HTTP/1.1" 200 None 2026-04-02 06:00:43,840 - lava_callback - ERROR - Error processing callback for node 69cb62fb8c06c2799bf58587: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53146 - "GET /node/69c1739c2b2b51e9ad97a578%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53156 - "GET /node/698b8d600a16aea5a0dc8dd4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40602 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:02:03,780 - lava_callback - INFO - Received callback for node 69cb634c8c06c2799bf5859a INFO: 172.20.0.1:48644 - "POST /node/69cb634c8c06c2799bf5859a HTTP/1.0" 202 Accepted 2026-04-02 06:02:03,785 - lava_callback - INFO - Starting async job submit for node 69cb634c8c06c2799bf5859a 2026-04-02 06:02:03,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:02:03,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb634c8c06c2799bf5859a HTTP/1.1" 200 None 2026-04-02 06:02:03,801 - lava_callback - DEBUG - Processing node 69cb634c8c06c2799bf5859a: baseline-arm64-clabbe 2026-04-02 06:02:03,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb634c8c06c2799bf5859a 2026-04-02 06:02:03,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:02:03,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb634c8c06c2799bf5859a HTTP/1.1" 200 None 2026-04-02 06:02:03,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:02:03,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-02 06:02:03,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:02:03,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb634c8c06c2799bf5859a HTTP/1.1" 200 None 2026-04-02 06:02:03,863 - lava_callback - ERROR - Error processing callback for node 69cb634c8c06c2799bf5859a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42974 - "GET /node/694afdf06c96ed51d3172c3e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42980 - "GET /node/69935d90523507f3806aa8b8%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-02 06:03:23,788 - lava_callback - INFO - Received callback for node 69cb639c8c06c2799bf585ab INFO: 172.20.0.1:36406 - "POST /node/69cb639c8c06c2799bf585ab HTTP/1.0" 202 Accepted 2026-04-02 06:03:23,794 - lava_callback - INFO - Starting async job submit for node 69cb639c8c06c2799bf585ab 2026-04-02 06:03:23,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:03:23,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb639c8c06c2799bf585ab HTTP/1.1" 200 None 2026-04-02 06:03:23,809 - lava_callback - DEBUG - Processing node 69cb639c8c06c2799bf585ab: baseline-arm64-clabbe 2026-04-02 06:03:23,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb639c8c06c2799bf585ab 2026-04-02 06:03:23,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:03:23,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb639c8c06c2799bf585ab HTTP/1.1" 200 None 2026-04-02 06:03:23,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:03:23,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-02 06:03:23,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:03:23,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb639c8c06c2799bf585ab HTTP/1.1" 200 None 2026-04-02 06:03:23,878 - lava_callback - ERROR - Error processing callback for node 69cb639c8c06c2799bf585ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37032 - "GET /node/69846a673a8748f282086ce4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37034 - "GET /node/69af3cf2ce46d8ec186ad412%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46134 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:04:44,123 - lava_callback - INFO - Received callback for node 69cb63ea8c06c2799bf585b4 INFO: 172.20.0.1:58568 - "POST /node/69cb63ea8c06c2799bf585b4 HTTP/1.0" 202 Accepted 2026-04-02 06:04:44,129 - lava_callback - INFO - Starting async job submit for node 69cb63ea8c06c2799bf585b4 2026-04-02 06:04:44,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:04:44,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb63ea8c06c2799bf585b4 HTTP/1.1" 200 None 2026-04-02 06:04:44,146 - lava_callback - DEBUG - Processing node 69cb63ea8c06c2799bf585b4: baseline-arm64-clabbe 2026-04-02 06:04:44,146 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb63ea8c06c2799bf585b4 2026-04-02 06:04:44,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:04:44,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb63ea8c06c2799bf585b4 HTTP/1.1" 200 None 2026-04-02 06:04:44,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:04:44,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 06:04:44,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:04:44,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb63ea8c06c2799bf585b4 HTTP/1.1" 200 None 2026-04-02 06:04:44,213 - lava_callback - ERROR - Error processing callback for node 69cb63ea8c06c2799bf585b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58588 - "GET /node/693b318ced677bab79b2cdc7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58596 - "GET /node/698af0b73b3beb19e76164c7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39910 - "GET /node/6991170e85f550420785e85c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39920 - "GET /node/6987035e6e077e4126a7e32e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46322 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:06:03,976 - lava_callback - INFO - Received callback for node 69cb643a8c06c2799bf585c5 INFO: 172.20.0.1:44356 - "POST /node/69cb643a8c06c2799bf585c5 HTTP/1.0" 202 Accepted 2026-04-02 06:06:03,982 - lava_callback - INFO - Starting async job submit for node 69cb643a8c06c2799bf585c5 2026-04-02 06:06:03,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:06:04,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb643a8c06c2799bf585c5 HTTP/1.1" 200 None 2026-04-02 06:06:04,004 - lava_callback - DEBUG - Processing node 69cb643a8c06c2799bf585c5: baseline-arm64-clabbe 2026-04-02 06:06:04,004 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb643a8c06c2799bf585c5 2026-04-02 06:06:04,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:06:04,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb643a8c06c2799bf585c5 HTTP/1.1" 200 None 2026-04-02 06:06:04,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:06:04,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 06:06:04,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:06:04,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb643a8c06c2799bf585c5 HTTP/1.1" 200 None 2026-04-02 06:06:04,070 - lava_callback - ERROR - Error processing callback for node 69cb643a8c06c2799bf585c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48546 - "GET /node/69a46c74a327b9815668ffba%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57098 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:07:24,179 - lava_callback - INFO - Received callback for node 69cb64898c06c2799bf585dc INFO: 172.20.0.1:42730 - "POST /node/69cb64898c06c2799bf585dc HTTP/1.0" 202 Accepted 2026-04-02 06:07:24,185 - lava_callback - INFO - Starting async job submit for node 69cb64898c06c2799bf585dc 2026-04-02 06:07:24,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:07:24,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb64898c06c2799bf585dc HTTP/1.1" 200 None 2026-04-02 06:07:24,200 - lava_callback - DEBUG - Processing node 69cb64898c06c2799bf585dc: baseline-arm64-clabbe 2026-04-02 06:07:24,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb64898c06c2799bf585dc 2026-04-02 06:07:24,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:07:24,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb64898c06c2799bf585dc HTTP/1.1" 200 None 2026-04-02 06:07:24,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:07:24,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-02 06:07:24,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:07:24,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb64898c06c2799bf585dc HTTP/1.1" 200 None 2026-04-02 06:07:24,252 - lava_callback - ERROR - Error processing callback for node 69cb64898c06c2799bf585dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48862 - "GET /node/69937e54cebe99dd8e06cec1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49926 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:08:44,039 - lava_callback - INFO - Received callback for node 69cb64d98c06c2799bf585e3 INFO: 172.20.0.1:59160 - "POST /node/69cb64d98c06c2799bf585e3 HTTP/1.0" 202 Accepted 2026-04-02 06:08:44,045 - lava_callback - INFO - Starting async job submit for node 69cb64d98c06c2799bf585e3 2026-04-02 06:08:44,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:08:44,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb64d98c06c2799bf585e3 HTTP/1.1" 200 None 2026-04-02 06:08:44,062 - lava_callback - DEBUG - Processing node 69cb64d98c06c2799bf585e3: baseline-arm64-clabbe 2026-04-02 06:08:44,062 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb64d98c06c2799bf585e3 2026-04-02 06:08:44,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:08:44,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb64d98c06c2799bf585e3 HTTP/1.1" 200 None 2026-04-02 06:08:44,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:08:44,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-02 06:08:44,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:08:44,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb64d98c06c2799bf585e3 HTTP/1.1" 200 None 2026-04-02 06:08:44,128 - lava_callback - ERROR - Error processing callback for node 69cb64d98c06c2799bf585e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39854 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:10:04,235 - lava_callback - INFO - Received callback for node 69cb65278c06c2799bf585f6 INFO: 172.20.0.1:59984 - "POST /node/69cb65278c06c2799bf585f6 HTTP/1.0" 202 Accepted 2026-04-02 06:10:04,241 - lava_callback - INFO - Starting async job submit for node 69cb65278c06c2799bf585f6 2026-04-02 06:10:04,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:10:04,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb65278c06c2799bf585f6 HTTP/1.1" 200 None 2026-04-02 06:10:04,256 - lava_callback - DEBUG - Processing node 69cb65278c06c2799bf585f6: baseline-arm64-clabbe 2026-04-02 06:10:04,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb65278c06c2799bf585f6 2026-04-02 06:10:04,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:10:04,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb65278c06c2799bf585f6 HTTP/1.1" 200 None 2026-04-02 06:10:04,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:10:04,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-02 06:10:04,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:10:04,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb65278c06c2799bf585f6 HTTP/1.1" 200 None 2026-04-02 06:10:04,316 - lava_callback - ERROR - Error processing callback for node 69cb65278c06c2799bf585f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53928 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:11:24,137 - lava_callback - INFO - Received callback for node 69cb65768c06c2799bf5860c INFO: 172.20.0.1:49612 - "POST /node/69cb65768c06c2799bf5860c HTTP/1.0" 202 Accepted 2026-04-02 06:11:24,143 - lava_callback - INFO - Starting async job submit for node 69cb65768c06c2799bf5860c 2026-04-02 06:11:24,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:11:24,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb65768c06c2799bf5860c HTTP/1.1" 200 None 2026-04-02 06:11:24,165 - lava_callback - DEBUG - Processing node 69cb65768c06c2799bf5860c: baseline-arm64-clabbe 2026-04-02 06:11:24,165 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb65768c06c2799bf5860c 2026-04-02 06:11:24,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:11:24,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb65768c06c2799bf5860c HTTP/1.1" 200 None 2026-04-02 06:11:24,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:11:24,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-02 06:11:24,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:11:24,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb65768c06c2799bf5860c HTTP/1.1" 200 None 2026-04-02 06:11:24,222 - lava_callback - ERROR - Error processing callback for node 69cb65768c06c2799bf5860c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46080 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:12:44,183 - lava_callback - INFO - Received callback for node 69cb65c68c06c2799bf58614 INFO: 172.20.0.1:33974 - "POST /node/69cb65c68c06c2799bf58614 HTTP/1.0" 202 Accepted 2026-04-02 06:12:44,188 - lava_callback - INFO - Starting async job submit for node 69cb65c68c06c2799bf58614 2026-04-02 06:12:44,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:12:44,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb65c68c06c2799bf58614 HTTP/1.1" 200 None 2026-04-02 06:12:44,212 - lava_callback - DEBUG - Processing node 69cb65c68c06c2799bf58614: baseline-arm64-clabbe 2026-04-02 06:12:44,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb65c68c06c2799bf58614 2026-04-02 06:12:44,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:12:44,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb65c68c06c2799bf58614 HTTP/1.1" 200 None 2026-04-02 06:12:44,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:12:44,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 06:12:44,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:12:44,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb65c68c06c2799bf58614 HTTP/1.1" 200 None 2026-04-02 06:12:44,264 - lava_callback - ERROR - Error processing callback for node 69cb65c68c06c2799bf58614: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45582 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:14:04,237 - lava_callback - INFO - Received callback for node 69cb66168c06c2799bf58626 INFO: 172.20.0.1:54768 - "POST /node/69cb66168c06c2799bf58626 HTTP/1.0" 202 Accepted 2026-04-02 06:14:04,243 - lava_callback - INFO - Starting async job submit for node 69cb66168c06c2799bf58626 2026-04-02 06:14:04,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:14:04,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb66168c06c2799bf58626 HTTP/1.1" 200 None 2026-04-02 06:14:04,265 - lava_callback - DEBUG - Processing node 69cb66168c06c2799bf58626: baseline-arm64-clabbe 2026-04-02 06:14:04,265 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb66168c06c2799bf58626 2026-04-02 06:14:04,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:14:04,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb66168c06c2799bf58626 HTTP/1.1" 200 None 2026-04-02 06:14:04,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:14:04,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 06:14:04,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:14:04,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb66168c06c2799bf58626 HTTP/1.1" 200 None 2026-04-02 06:14:04,334 - lava_callback - ERROR - Error processing callback for node 69cb66168c06c2799bf58626: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50006 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:15:04,330 - lava_callback - INFO - Received callback for node 69cb66638c06c2799bf58631 INFO: 172.20.0.1:49454 - "POST /node/69cb66638c06c2799bf58631 HTTP/1.0" 202 Accepted 2026-04-02 06:15:04,335 - lava_callback - INFO - Starting async job submit for node 69cb66638c06c2799bf58631 2026-04-02 06:15:04,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:15:04,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb66638c06c2799bf58631 HTTP/1.1" 200 None 2026-04-02 06:15:04,379 - lava_callback - DEBUG - Processing node 69cb66638c06c2799bf58631: baseline-arm64-clabbe 2026-04-02 06:15:04,380 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb66638c06c2799bf58631 2026-04-02 06:15:04,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:15:04,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb66638c06c2799bf58631 HTTP/1.1" 200 None 2026-04-02 06:15:04,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:15:04,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 06:15:04,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:15:04,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb66638c06c2799bf58631 HTTP/1.1" 200 None 2026-04-02 06:15:04,465 - lava_callback - ERROR - Error processing callback for node 69cb66638c06c2799bf58631: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:16:24,353 - lava_callback - INFO - Received callback for node 69cb66b18c06c2799bf5863f INFO: 172.20.0.1:38734 - "POST /node/69cb66b18c06c2799bf5863f HTTP/1.0" 202 Accepted 2026-04-02 06:16:24,359 - lava_callback - INFO - Starting async job submit for node 69cb66b18c06c2799bf5863f 2026-04-02 06:16:24,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:16:24,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb66b18c06c2799bf5863f HTTP/1.1" 200 None 2026-04-02 06:16:24,393 - lava_callback - DEBUG - Processing node 69cb66b18c06c2799bf5863f: baseline-arm64-clabbe 2026-04-02 06:16:24,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb66b18c06c2799bf5863f 2026-04-02 06:16:24,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:16:24,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb66b18c06c2799bf5863f HTTP/1.1" 200 None 2026-04-02 06:16:24,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:16:24,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 06:16:24,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:16:24,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb66b18c06c2799bf5863f HTTP/1.1" 200 None 2026-04-02 06:16:24,477 - lava_callback - ERROR - Error processing callback for node 69cb66b18c06c2799bf5863f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55586 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:17:44,687 - lava_callback - INFO - Received callback for node 69cb66ff8c06c2799bf58654 INFO: 172.20.0.1:42328 - "POST /node/69cb66ff8c06c2799bf58654 HTTP/1.0" 202 Accepted 2026-04-02 06:17:44,692 - lava_callback - INFO - Starting async job submit for node 69cb66ff8c06c2799bf58654 2026-04-02 06:17:44,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:17:44,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb66ff8c06c2799bf58654 HTTP/1.1" 200 None 2026-04-02 06:17:44,710 - lava_callback - DEBUG - Processing node 69cb66ff8c06c2799bf58654: baseline-arm64-clabbe 2026-04-02 06:17:44,710 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb66ff8c06c2799bf58654 2026-04-02 06:17:44,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:17:44,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb66ff8c06c2799bf58654 HTTP/1.1" 200 None 2026-04-02 06:17:44,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:17:44,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 06:17:44,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:17:44,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb66ff8c06c2799bf58654 HTTP/1.1" 200 None 2026-04-02 06:17:44,809 - lava_callback - ERROR - Error processing callback for node 69cb66ff8c06c2799bf58654: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60586 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:19:04,628 - lava_callback - INFO - Received callback for node 69cb674e8c06c2799bf5865f INFO: 172.20.0.1:35120 - "POST /node/69cb674e8c06c2799bf5865f HTTP/1.0" 202 Accepted 2026-04-02 06:19:04,635 - lava_callback - INFO - Starting async job submit for node 69cb674e8c06c2799bf5865f 2026-04-02 06:19:04,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:19:04,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb674e8c06c2799bf5865f HTTP/1.1" 200 None 2026-04-02 06:19:04,654 - lava_callback - DEBUG - Processing node 69cb674e8c06c2799bf5865f: baseline-arm64-clabbe 2026-04-02 06:19:04,654 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb674e8c06c2799bf5865f 2026-04-02 06:19:04,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:19:04,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb674e8c06c2799bf5865f HTTP/1.1" 200 None 2026-04-02 06:19:04,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:19:04,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-04-02 06:19:04,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:19:04,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb674e8c06c2799bf5865f HTTP/1.1" 200 None 2026-04-02 06:19:04,714 - lava_callback - ERROR - Error processing callback for node 69cb674e8c06c2799bf5865f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49198 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:20:14,433 - lava_callback - INFO - Received callback for node 69cb67988c06c2799bf58670 INFO: 172.20.0.1:37760 - "POST /node/69cb67988c06c2799bf58670 HTTP/1.0" 202 Accepted 2026-04-02 06:20:14,439 - lava_callback - INFO - Starting async job submit for node 69cb67988c06c2799bf58670 2026-04-02 06:20:14,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:20:14,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb67988c06c2799bf58670 HTTP/1.1" 200 None 2026-04-02 06:20:14,455 - lava_callback - DEBUG - Processing node 69cb67988c06c2799bf58670: baseline-arm64-baylibre 2026-04-02 06:20:14,455 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb67988c06c2799bf58670 2026-04-02 06:20:14,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:20:14,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb67988c06c2799bf58670 HTTP/1.1" 200 None 2026-04-02 06:20:14,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:20:14,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 06:20:14,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:20:14,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb67988c06c2799bf58670 HTTP/1.1" 200 None 2026-04-02 06:20:14,516 - lava_callback - ERROR - Error processing callback for node 69cb67988c06c2799bf58670: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50216 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:21:44,615 - lava_callback - INFO - Received callback for node 69cb67f28c06c2799bf5867f INFO: 172.20.0.1:53214 - "POST /node/69cb67f28c06c2799bf5867f HTTP/1.0" 202 Accepted 2026-04-02 06:21:44,621 - lava_callback - INFO - Starting async job submit for node 69cb67f28c06c2799bf5867f 2026-04-02 06:21:44,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:21:44,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb67f28c06c2799bf5867f HTTP/1.1" 200 None 2026-04-02 06:21:44,637 - lava_callback - DEBUG - Processing node 69cb67f28c06c2799bf5867f: baseline-arm64-clabbe 2026-04-02 06:21:44,637 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb67f28c06c2799bf5867f 2026-04-02 06:21:44,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:21:44,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb67f28c06c2799bf5867f HTTP/1.1" 200 None 2026-04-02 06:21:44,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:21:44,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 06:21:44,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:21:44,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb67f28c06c2799bf5867f HTTP/1.1" 200 None 2026-04-02 06:21:44,693 - lava_callback - ERROR - Error processing callback for node 69cb67f28c06c2799bf5867f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55698 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:23:04,677 - lava_callback - INFO - Received callback for node 69cb683f8c06c2799bf58685 INFO: 172.20.0.1:36056 - "POST /node/69cb683f8c06c2799bf58685 HTTP/1.0" 202 Accepted 2026-04-02 06:23:04,683 - lava_callback - INFO - Starting async job submit for node 69cb683f8c06c2799bf58685 2026-04-02 06:23:04,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:23:04,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb683f8c06c2799bf58685 HTTP/1.1" 200 None 2026-04-02 06:23:04,699 - lava_callback - DEBUG - Processing node 69cb683f8c06c2799bf58685: baseline-arm64-clabbe 2026-04-02 06:23:04,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb683f8c06c2799bf58685 2026-04-02 06:23:04,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:23:04,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb683f8c06c2799bf58685 HTTP/1.1" 200 None 2026-04-02 06:23:04,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:23:04,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 06:23:04,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:23:04,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb683f8c06c2799bf58685 HTTP/1.1" 200 None 2026-04-02 06:23:04,760 - lava_callback - ERROR - Error processing callback for node 69cb683f8c06c2799bf58685: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50878 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:24:24,750 - lava_callback - INFO - Received callback for node 69cb688d8c06c2799bf58694 INFO: 172.20.0.1:49318 - "POST /node/69cb688d8c06c2799bf58694 HTTP/1.0" 202 Accepted 2026-04-02 06:24:24,755 - lava_callback - INFO - Starting async job submit for node 69cb688d8c06c2799bf58694 2026-04-02 06:24:24,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:24:24,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb688d8c06c2799bf58694 HTTP/1.1" 200 None 2026-04-02 06:24:24,777 - lava_callback - DEBUG - Processing node 69cb688d8c06c2799bf58694: baseline-arm64-clabbe 2026-04-02 06:24:24,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb688d8c06c2799bf58694 2026-04-02 06:24:24,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:24:24,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb688d8c06c2799bf58694 HTTP/1.1" 200 None 2026-04-02 06:24:24,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:24:24,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 06:24:24,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:24:24,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb688d8c06c2799bf58694 HTTP/1.1" 200 None 2026-04-02 06:24:24,831 - lava_callback - ERROR - Error processing callback for node 69cb688d8c06c2799bf58694: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36784 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:25:44,791 - lava_callback - INFO - Received callback for node 69cb68dc8c06c2799bf586a4 INFO: 172.20.0.1:44130 - "POST /node/69cb68dc8c06c2799bf586a4 HTTP/1.0" 202 Accepted 2026-04-02 06:25:44,796 - lava_callback - INFO - Starting async job submit for node 69cb68dc8c06c2799bf586a4 2026-04-02 06:25:44,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:25:44,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb68dc8c06c2799bf586a4 HTTP/1.1" 200 None 2026-04-02 06:25:44,812 - lava_callback - DEBUG - Processing node 69cb68dc8c06c2799bf586a4: baseline-arm64-clabbe 2026-04-02 06:25:44,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb68dc8c06c2799bf586a4 2026-04-02 06:25:44,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:25:44,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb68dc8c06c2799bf586a4 HTTP/1.1" 200 None 2026-04-02 06:25:44,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:25:44,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 06:25:44,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:25:44,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb68dc8c06c2799bf586a4 HTTP/1.1" 200 None 2026-04-02 06:25:44,867 - lava_callback - ERROR - Error processing callback for node 69cb68dc8c06c2799bf586a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57798 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:27:04,804 - lava_callback - INFO - Received callback for node 69cb692a8c06c2799bf586aa INFO: 172.20.0.1:34548 - "POST /node/69cb692a8c06c2799bf586aa HTTP/1.0" 202 Accepted 2026-04-02 06:27:04,810 - lava_callback - INFO - Starting async job submit for node 69cb692a8c06c2799bf586aa 2026-04-02 06:27:04,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:27:04,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb692a8c06c2799bf586aa HTTP/1.1" 200 None 2026-04-02 06:27:04,826 - lava_callback - DEBUG - Processing node 69cb692a8c06c2799bf586aa: baseline-arm64-clabbe 2026-04-02 06:27:04,826 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb692a8c06c2799bf586aa 2026-04-02 06:27:04,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:27:04,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb692a8c06c2799bf586aa HTTP/1.1" 200 None 2026-04-02 06:27:04,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:27:04,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 06:27:04,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:27:04,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb692a8c06c2799bf586aa HTTP/1.1" 200 None 2026-04-02 06:27:04,906 - lava_callback - ERROR - Error processing callback for node 69cb692a8c06c2799bf586aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60170 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:28:24,850 - lava_callback - INFO - Received callback for node 69cb69788c06c2799bf586bb INFO: 172.20.0.1:50874 - "POST /node/69cb69788c06c2799bf586bb HTTP/1.0" 202 Accepted 2026-04-02 06:28:24,855 - lava_callback - INFO - Starting async job submit for node 69cb69788c06c2799bf586bb 2026-04-02 06:28:24,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:28:24,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb69788c06c2799bf586bb HTTP/1.1" 200 None 2026-04-02 06:28:24,875 - lava_callback - DEBUG - Processing node 69cb69788c06c2799bf586bb: baseline-arm64-clabbe 2026-04-02 06:28:24,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb69788c06c2799bf586bb 2026-04-02 06:28:24,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:28:24,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb69788c06c2799bf586bb HTTP/1.1" 200 None 2026-04-02 06:28:24,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:28:24,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7ff4f123b47fd2376d75b HTTP/1.1" 200 None 2026-04-02 06:28:24,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:28:24,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb69788c06c2799bf586bb HTTP/1.1" 200 None 2026-04-02 06:28:24,936 - lava_callback - ERROR - Error processing callback for node 69cb69788c06c2799bf586bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53654 - "GET /node/69c7337229015d0f9599264c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49204 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:35:55,221 - lava_callback - INFO - Received callback for node 69cb6b3f8c06c2799bf586c3 INFO: 172.20.0.1:46454 - "POST /node/69cb6b3f8c06c2799bf586c3 HTTP/1.0" 202 Accepted 2026-04-02 06:35:55,226 - lava_callback - INFO - Starting async job submit for node 69cb6b3f8c06c2799bf586c3 2026-04-02 06:35:55,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:35:55,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6b3f8c06c2799bf586c3 HTTP/1.1" 200 None 2026-04-02 06:35:55,243 - lava_callback - DEBUG - Processing node 69cb6b3f8c06c2799bf586c3: baseline-arm-baylibre 2026-04-02 06:35:55,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6b3f8c06c2799bf586c3 2026-04-02 06:35:55,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:35:55,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6b3f8c06c2799bf586c3 HTTP/1.1" 200 None 2026-04-02 06:35:55,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:35:55,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:35:55,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:35:55,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6b3f8c06c2799bf586c3 HTTP/1.1" 200 None 2026-04-02 06:35:55,300 - lava_callback - ERROR - Error processing callback for node 69cb6b3f8c06c2799bf586c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40640 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:36:35,238 - lava_callback - INFO - Received callback for node 69cb6b658c06c2799bf586d7 INFO: 172.20.0.1:38954 - "POST /node/69cb6b658c06c2799bf586d7 HTTP/1.0" 202 Accepted 2026-04-02 06:36:35,244 - lava_callback - INFO - Starting async job submit for node 69cb6b658c06c2799bf586d7 2026-04-02 06:36:35,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:36:35,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6b658c06c2799bf586d7 HTTP/1.1" 200 None 2026-04-02 06:36:35,275 - lava_callback - DEBUG - Processing node 69cb6b658c06c2799bf586d7: baseline-arm-baylibre 2026-04-02 06:36:35,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6b658c06c2799bf586d7 2026-04-02 06:36:35,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:36:35,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6b658c06c2799bf586d7 HTTP/1.1" 200 None 2026-04-02 06:36:35,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:36:35,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:36:35,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:36:35,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6b658c06c2799bf586d7 HTTP/1.1" 200 None 2026-04-02 06:36:35,332 - lava_callback - ERROR - Error processing callback for node 69cb6b658c06c2799bf586d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40706 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 06:38:15,366 - lava_callback - INFO - Received callback for node 69cb6bcd8c06c2799bf586f0 INFO: 172.20.0.1:40714 - "POST /node/69cb6bcd8c06c2799bf586f0 HTTP/1.0" 202 Accepted 2026-04-02 06:38:15,375 - lava_callback - INFO - Starting async job submit for node 69cb6bcd8c06c2799bf586f0 2026-04-02 06:38:15,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:38:15,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6bcd8c06c2799bf586f0 HTTP/1.1" 200 None 2026-04-02 06:38:15,405 - lava_callback - DEBUG - Processing node 69cb6bcd8c06c2799bf586f0: baseline-arm-baylibre 2026-04-02 06:38:15,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6bcd8c06c2799bf586f0 2026-04-02 06:38:15,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:38:15,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6bcd8c06c2799bf586f0 HTTP/1.1" 200 None 2026-04-02 06:38:15,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:38:15,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:38:15,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:38:15,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6bcd8c06c2799bf586f0 HTTP/1.1" 200 None 2026-04-02 06:38:15,460 - lava_callback - ERROR - Error processing callback for node 69cb6bcd8c06c2799bf586f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44674 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:38:55,312 - lava_callback - INFO - Received callback for node 69cb6bf18c06c2799bf586fb INFO: 172.20.0.1:36700 - "POST /node/69cb6bf18c06c2799bf586fb HTTP/1.0" 202 Accepted 2026-04-02 06:38:55,318 - lava_callback - INFO - Starting async job submit for node 69cb6bf18c06c2799bf586fb 2026-04-02 06:38:55,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:38:55,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6bf18c06c2799bf586fb HTTP/1.1" 200 None 2026-04-02 06:38:55,334 - lava_callback - DEBUG - Processing node 69cb6bf18c06c2799bf586fb: baseline-arm-baylibre 2026-04-02 06:38:55,334 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6bf18c06c2799bf586fb 2026-04-02 06:38:55,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:38:55,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6bf18c06c2799bf586fb HTTP/1.1" 200 None 2026-04-02 06:38:55,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:38:55,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:38:55,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:38:55,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6bf18c06c2799bf586fb HTTP/1.1" 200 None 2026-04-02 06:38:55,389 - lava_callback - ERROR - Error processing callback for node 69cb6bf18c06c2799bf586fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51322 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:39:35,391 - lava_callback - INFO - Received callback for node 69cb6c178c06c2799bf58704 INFO: 172.20.0.1:49044 - "POST /node/69cb6c178c06c2799bf58704 HTTP/1.0" 202 Accepted 2026-04-02 06:39:35,397 - lava_callback - INFO - Starting async job submit for node 69cb6c178c06c2799bf58704 2026-04-02 06:39:35,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:39:35,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6c178c06c2799bf58704 HTTP/1.1" 200 None 2026-04-02 06:39:35,412 - lava_callback - DEBUG - Processing node 69cb6c178c06c2799bf58704: baseline-arm-baylibre 2026-04-02 06:39:35,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6c178c06c2799bf58704 2026-04-02 06:39:35,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:39:35,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6c178c06c2799bf58704 HTTP/1.1" 200 None 2026-04-02 06:39:35,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:39:35,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:39:35,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:39:35,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6c178c06c2799bf58704 HTTP/1.1" 200 None 2026-04-02 06:39:35,476 - lava_callback - ERROR - Error processing callback for node 69cb6c178c06c2799bf58704: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42916 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 06:40:15,430 - lava_callback - INFO - Received callback for node 69cb6c3b8c06c2799bf58715 INFO: 172.20.0.1:42928 - "POST /node/69cb6c3b8c06c2799bf58715 HTTP/1.0" 202 Accepted 2026-04-02 06:40:15,437 - lava_callback - INFO - Starting async job submit for node 69cb6c3b8c06c2799bf58715 2026-04-02 06:40:15,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:40:15,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6c3b8c06c2799bf58715 HTTP/1.1" 200 None 2026-04-02 06:40:15,452 - lava_callback - DEBUG - Processing node 69cb6c3b8c06c2799bf58715: baseline-arm-baylibre 2026-04-02 06:40:15,453 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6c3b8c06c2799bf58715 2026-04-02 06:40:15,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:40:15,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6c3b8c06c2799bf58715 HTTP/1.1" 200 None 2026-04-02 06:40:15,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:40:15,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:40:15,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:40:15,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6c3b8c06c2799bf58715 HTTP/1.1" 200 None 2026-04-02 06:40:15,509 - lava_callback - ERROR - Error processing callback for node 69cb6c3b8c06c2799bf58715: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52320 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:40:35,416 - lava_callback - INFO - Received callback for node 69cb6c608c06c2799bf58722 INFO: 172.20.0.1:46714 - "POST /node/69cb6c608c06c2799bf58722 HTTP/1.0" 202 Accepted 2026-04-02 06:40:35,421 - lava_callback - INFO - Starting async job submit for node 69cb6c608c06c2799bf58722 2026-04-02 06:40:35,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:40:35,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6c608c06c2799bf58722 HTTP/1.1" 200 None 2026-04-02 06:40:35,438 - lava_callback - DEBUG - Processing node 69cb6c608c06c2799bf58722: baseline-arm-baylibre 2026-04-02 06:40:35,438 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6c608c06c2799bf58722 2026-04-02 06:40:35,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:40:35,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6c608c06c2799bf58722 HTTP/1.1" 200 None 2026-04-02 06:40:35,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:40:35,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:40:35,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:40:35,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6c608c06c2799bf58722 HTTP/1.1" 200 None 2026-04-02 06:40:35,500 - lava_callback - ERROR - Error processing callback for node 69cb6c608c06c2799bf58722: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43690 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 06:41:15,481 - lava_callback - INFO - Received callback for node 69cb6c848c06c2799bf58730 INFO: 172.20.0.1:43698 - "POST /node/69cb6c848c06c2799bf58730 HTTP/1.0" 202 Accepted 2026-04-02 06:41:15,486 - lava_callback - INFO - Starting async job submit for node 69cb6c848c06c2799bf58730 2026-04-02 06:41:15,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:41:15,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6c848c06c2799bf58730 HTTP/1.1" 200 None 2026-04-02 06:41:15,506 - lava_callback - DEBUG - Processing node 69cb6c848c06c2799bf58730: baseline-arm-baylibre 2026-04-02 06:41:15,507 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6c848c06c2799bf58730 2026-04-02 06:41:15,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:41:15,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6c848c06c2799bf58730 HTTP/1.1" 200 None 2026-04-02 06:41:15,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:41:15,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:41:15,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:41:15,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6c848c06c2799bf58730 HTTP/1.1" 200 None 2026-04-02 06:41:15,567 - lava_callback - ERROR - Error processing callback for node 69cb6c848c06c2799bf58730: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:41:55,490 - lava_callback - INFO - Received callback for node 69cb6caa8c06c2799bf5873b INFO: 172.20.0.1:38608 - "POST /node/69cb6caa8c06c2799bf5873b HTTP/1.0" 202 Accepted 2026-04-02 06:41:55,496 - lava_callback - INFO - Starting async job submit for node 69cb6caa8c06c2799bf5873b 2026-04-02 06:41:55,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:41:55,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6caa8c06c2799bf5873b HTTP/1.1" 200 None 2026-04-02 06:41:55,512 - lava_callback - DEBUG - Processing node 69cb6caa8c06c2799bf5873b: baseline-arm-baylibre 2026-04-02 06:41:55,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6caa8c06c2799bf5873b 2026-04-02 06:41:55,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:41:55,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6caa8c06c2799bf5873b HTTP/1.1" 200 None 2026-04-02 06:41:55,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:41:55,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:41:55,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:41:55,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6caa8c06c2799bf5873b HTTP/1.1" 200 None 2026-04-02 06:41:55,563 - lava_callback - ERROR - Error processing callback for node 69cb6caa8c06c2799bf5873b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42016 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:42:45,453 - lava_callback - INFO - Received callback for node 69cb6cd08c06c2799bf58746 INFO: 172.20.0.1:35090 - "POST /node/69cb6cd08c06c2799bf58746 HTTP/1.0" 202 Accepted 2026-04-02 06:42:45,458 - lava_callback - INFO - Starting async job submit for node 69cb6cd08c06c2799bf58746 2026-04-02 06:42:45,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:42:45,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6cd08c06c2799bf58746 HTTP/1.1" 200 None 2026-04-02 06:42:45,475 - lava_callback - DEBUG - Processing node 69cb6cd08c06c2799bf58746: baseline-arm-clabbe 2026-04-02 06:42:45,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6cd08c06c2799bf58746 2026-04-02 06:42:45,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:42:45,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6cd08c06c2799bf58746 HTTP/1.1" 200 None 2026-04-02 06:42:45,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:42:45,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:42:45,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:42:45,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6cd08c06c2799bf58746 HTTP/1.1" 200 None 2026-04-02 06:42:45,532 - lava_callback - ERROR - Error processing callback for node 69cb6cd08c06c2799bf58746: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39828 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 06:43:15,568 - lava_callback - INFO - Received callback for node 69cb6cf38c06c2799bf58752 INFO: 172.20.0.1:39840 - "POST /node/69cb6cf38c06c2799bf58752 HTTP/1.0" 202 Accepted 2026-04-02 06:43:15,573 - lava_callback - INFO - Starting async job submit for node 69cb6cf38c06c2799bf58752 2026-04-02 06:43:15,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:43:15,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6cf38c06c2799bf58752 HTTP/1.1" 200 None 2026-04-02 06:43:15,589 - lava_callback - DEBUG - Processing node 69cb6cf38c06c2799bf58752: baseline-arm-baylibre 2026-04-02 06:43:15,590 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6cf38c06c2799bf58752 2026-04-02 06:43:15,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:43:15,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6cf38c06c2799bf58752 HTTP/1.1" 200 None 2026-04-02 06:43:15,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:43:15,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:43:15,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:43:15,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6cf38c06c2799bf58752 HTTP/1.1" 200 None 2026-04-02 06:43:15,670 - lava_callback - ERROR - Error processing callback for node 69cb6cf38c06c2799bf58752: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34884 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:43:45,496 - lava_callback - INFO - Received callback for node 69cb6d1a8c06c2799bf58764 INFO: 172.20.0.1:44670 - "POST /node/69cb6d1a8c06c2799bf58764 HTTP/1.0" 202 Accepted 2026-04-02 06:43:45,502 - lava_callback - INFO - Starting async job submit for node 69cb6d1a8c06c2799bf58764 2026-04-02 06:43:45,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:43:45,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6d1a8c06c2799bf58764 HTTP/1.1" 200 None 2026-04-02 06:43:45,519 - lava_callback - DEBUG - Processing node 69cb6d1a8c06c2799bf58764: baseline-arm-clabbe 2026-04-02 06:43:45,520 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6d1a8c06c2799bf58764 2026-04-02 06:43:45,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:43:45,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6d1a8c06c2799bf58764 HTTP/1.1" 200 None 2026-04-02 06:43:45,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:43:45,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:43:45,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:43:45,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6d1a8c06c2799bf58764 HTTP/1.1" 200 None 2026-04-02 06:43:45,579 - lava_callback - ERROR - Error processing callback for node 69cb6d1a8c06c2799bf58764: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44848 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 06:44:15,574 - lava_callback - INFO - Received callback for node 69cb6d3d8c06c2799bf58773 INFO: 172.20.0.1:44850 - "POST /node/69cb6d3d8c06c2799bf58773 HTTP/1.0" 202 Accepted 2026-04-02 06:44:15,579 - lava_callback - INFO - Starting async job submit for node 69cb6d3d8c06c2799bf58773 2026-04-02 06:44:15,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:44:15,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6d3d8c06c2799bf58773 HTTP/1.1" 200 None 2026-04-02 06:44:15,595 - lava_callback - DEBUG - Processing node 69cb6d3d8c06c2799bf58773: baseline-arm-baylibre 2026-04-02 06:44:15,595 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6d3d8c06c2799bf58773 2026-04-02 06:44:15,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:44:15,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6d3d8c06c2799bf58773 HTTP/1.1" 200 None 2026-04-02 06:44:15,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:44:15,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:44:15,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:44:15,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6d3d8c06c2799bf58773 HTTP/1.1" 200 None 2026-04-02 06:44:15,664 - lava_callback - ERROR - Error processing callback for node 69cb6d3d8c06c2799bf58773: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42286 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:45:05,630 - lava_callback - INFO - Received callback for node 69cb6d648c06c2799bf58780 INFO: 172.20.0.1:52780 - "POST /node/69cb6d648c06c2799bf58780 HTTP/1.0" 202 Accepted 2026-04-02 06:45:05,636 - lava_callback - INFO - Starting async job submit for node 69cb6d648c06c2799bf58780 2026-04-02 06:45:05,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:45:05,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6d648c06c2799bf58780 HTTP/1.1" 200 None 2026-04-02 06:45:05,658 - lava_callback - DEBUG - Processing node 69cb6d648c06c2799bf58780: baseline-arm-clabbe 2026-04-02 06:45:05,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6d648c06c2799bf58780 2026-04-02 06:45:05,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:45:05,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6d648c06c2799bf58780 HTTP/1.1" 200 None 2026-04-02 06:45:05,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:45:05,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:45:05,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:45:05,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6d648c06c2799bf58780 HTTP/1.1" 200 None 2026-04-02 06:45:05,720 - lava_callback - ERROR - Error processing callback for node 69cb6d648c06c2799bf58780: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38290 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:45:35,727 - lava_callback - INFO - Received callback for node 69cb6d878c06c2799bf5878c INFO: 172.20.0.1:50054 - "POST /node/69cb6d878c06c2799bf5878c HTTP/1.0" 202 Accepted 2026-04-02 06:45:35,733 - lava_callback - INFO - Starting async job submit for node 69cb6d878c06c2799bf5878c 2026-04-02 06:45:35,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:45:35,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6d878c06c2799bf5878c HTTP/1.1" 200 None 2026-04-02 06:45:35,749 - lava_callback - DEBUG - Processing node 69cb6d878c06c2799bf5878c: baseline-arm-baylibre 2026-04-02 06:45:35,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6d878c06c2799bf5878c 2026-04-02 06:45:35,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:45:35,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6d878c06c2799bf5878c HTTP/1.1" 200 None 2026-04-02 06:45:35,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:45:35,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:45:35,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:45:35,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6d878c06c2799bf5878c HTTP/1.1" 200 None 2026-04-02 06:45:35,813 - lava_callback - ERROR - Error processing callback for node 69cb6d878c06c2799bf5878c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50058 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:46:25,610 - lava_callback - INFO - Received callback for node 69cb6dad8c06c2799bf5879f INFO: 172.20.0.1:42156 - "POST /node/69cb6dad8c06c2799bf5879f HTTP/1.0" 202 Accepted 2026-04-02 06:46:25,615 - lava_callback - INFO - Starting async job submit for node 69cb6dad8c06c2799bf5879f 2026-04-02 06:46:25,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:46:25,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6dad8c06c2799bf5879f HTTP/1.1" 200 None 2026-04-02 06:46:25,631 - lava_callback - DEBUG - Processing node 69cb6dad8c06c2799bf5879f: baseline-arm-clabbe 2026-04-02 06:46:25,631 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6dad8c06c2799bf5879f 2026-04-02 06:46:25,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:46:25,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6dad8c06c2799bf5879f HTTP/1.1" 200 None 2026-04-02 06:46:25,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:46:25,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:46:25,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:46:25,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6dad8c06c2799bf5879f HTTP/1.1" 200 None 2026-04-02 06:46:25,698 - lava_callback - ERROR - Error processing callback for node 69cb6dad8c06c2799bf5879f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58900 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:46:55,841 - lava_callback - INFO - Received callback for node 69cb6dd08c06c2799bf587a6 INFO: 172.20.0.1:56530 - "POST /node/69cb6dd08c06c2799bf587a6 HTTP/1.0" 202 Accepted 2026-04-02 06:46:55,847 - lava_callback - INFO - Starting async job submit for node 69cb6dd08c06c2799bf587a6 2026-04-02 06:46:55,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:46:55,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6dd08c06c2799bf587a6 HTTP/1.1" 200 None 2026-04-02 06:46:55,862 - lava_callback - DEBUG - Processing node 69cb6dd08c06c2799bf587a6: baseline-arm-baylibre 2026-04-02 06:46:55,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6dd08c06c2799bf587a6 2026-04-02 06:46:55,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:46:55,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6dd08c06c2799bf587a6 HTTP/1.1" 200 None 2026-04-02 06:46:55,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:46:55,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:46:55,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:46:55,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6dd08c06c2799bf587a6 HTTP/1.1" 200 None 2026-04-02 06:46:55,925 - lava_callback - ERROR - Error processing callback for node 69cb6dd08c06c2799bf587a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46042 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:47:35,805 - lava_callback - INFO - Received callback for node 69cb6df48c06c2799bf587b2 INFO: 172.20.0.1:50884 - "POST /node/69cb6df48c06c2799bf587b2 HTTP/1.0" 202 Accepted 2026-04-02 06:47:35,811 - lava_callback - INFO - Starting async job submit for node 69cb6df48c06c2799bf587b2 2026-04-02 06:47:35,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:47:35,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6df48c06c2799bf587b2 HTTP/1.1" 200 None 2026-04-02 06:47:35,826 - lava_callback - DEBUG - Processing node 69cb6df48c06c2799bf587b2: baseline-arm-baylibre 2026-04-02 06:47:35,827 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6df48c06c2799bf587b2 2026-04-02 06:47:35,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:47:35,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6df48c06c2799bf587b2 HTTP/1.1" 200 None 2026-04-02 06:47:35,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:47:35,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:47:35,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:47:35,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6df48c06c2799bf587b2 HTTP/1.1" 200 None 2026-04-02 06:47:35,881 - lava_callback - ERROR - Error processing callback for node 69cb6df48c06c2799bf587b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34970 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:47:55,820 - lava_callback - INFO - Received callback for node 69cb6e178c06c2799bf587bd INFO: 172.20.0.1:51394 - "POST /node/69cb6e178c06c2799bf587bd HTTP/1.0" 202 Accepted 2026-04-02 06:47:55,825 - lava_callback - INFO - Starting async job submit for node 69cb6e178c06c2799bf587bd 2026-04-02 06:47:55,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:47:56,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6e178c06c2799bf587bd HTTP/1.1" 200 None 2026-04-02 06:47:56,013 - lava_callback - DEBUG - Processing node 69cb6e178c06c2799bf587bd: baseline-arm-baylibre 2026-04-02 06:47:56,014 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6e178c06c2799bf587bd 2026-04-02 06:47:56,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:47:56,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6e178c06c2799bf587bd HTTP/1.1" 200 None 2026-04-02 06:47:56,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:47:56,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:47:56,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:47:56,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6e178c06c2799bf587bd HTTP/1.1" 200 None 2026-04-02 06:47:56,090 - lava_callback - ERROR - Error processing callback for node 69cb6e178c06c2799bf587bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55684 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:48:35,864 - lava_callback - INFO - Received callback for node 69cb6e3d8c06c2799bf587cb INFO: 172.20.0.1:37924 - "POST /node/69cb6e3d8c06c2799bf587cb HTTP/1.0" 202 Accepted 2026-04-02 06:48:35,869 - lava_callback - INFO - Starting async job submit for node 69cb6e3d8c06c2799bf587cb 2026-04-02 06:48:35,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:48:35,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6e3d8c06c2799bf587cb HTTP/1.1" 200 None 2026-04-02 06:48:35,885 - lava_callback - DEBUG - Processing node 69cb6e3d8c06c2799bf587cb: baseline-arm-baylibre 2026-04-02 06:48:35,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6e3d8c06c2799bf587cb 2026-04-02 06:48:35,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:48:35,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6e3d8c06c2799bf587cb HTTP/1.1" 200 None 2026-04-02 06:48:35,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:48:35,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:48:35,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:48:35,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6e3d8c06c2799bf587cb HTTP/1.1" 200 None 2026-04-02 06:48:35,940 - lava_callback - ERROR - Error processing callback for node 69cb6e3d8c06c2799bf587cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49426 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 06:49:15,857 - lava_callback - INFO - Received callback for node 69cb6e5f8c06c2799bf587de INFO: 172.20.0.1:49432 - "POST /node/69cb6e5f8c06c2799bf587de HTTP/1.0" 202 Accepted 2026-04-02 06:49:15,862 - lava_callback - INFO - Starting async job submit for node 69cb6e5f8c06c2799bf587de 2026-04-02 06:49:15,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:49:15,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6e5f8c06c2799bf587de HTTP/1.1" 200 None 2026-04-02 06:49:15,878 - lava_callback - DEBUG - Processing node 69cb6e5f8c06c2799bf587de: baseline-arm-baylibre 2026-04-02 06:49:15,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6e5f8c06c2799bf587de 2026-04-02 06:49:15,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:49:15,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6e5f8c06c2799bf587de HTTP/1.1" 200 None 2026-04-02 06:49:15,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:49:15,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:49:15,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:49:15,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6e5f8c06c2799bf587de HTTP/1.1" 200 None 2026-04-02 06:49:15,944 - lava_callback - ERROR - Error processing callback for node 69cb6e5f8c06c2799bf587de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:49:55,947 - lava_callback - INFO - Received callback for node 69cb6e858c06c2799bf587e7 INFO: 172.20.0.1:43122 - "POST /node/69cb6e858c06c2799bf587e7 HTTP/1.0" 202 Accepted 2026-04-02 06:49:55,953 - lava_callback - INFO - Starting async job submit for node 69cb6e858c06c2799bf587e7 2026-04-02 06:49:55,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:49:55,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6e858c06c2799bf587e7 HTTP/1.1" 200 None 2026-04-02 06:49:55,970 - lava_callback - DEBUG - Processing node 69cb6e858c06c2799bf587e7: baseline-arm-baylibre 2026-04-02 06:49:55,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6e858c06c2799bf587e7 2026-04-02 06:49:55,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:49:55,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6e858c06c2799bf587e7 HTTP/1.1" 200 None 2026-04-02 06:49:55,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:49:55,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:49:56,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:49:56,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6e858c06c2799bf587e7 HTTP/1.1" 200 None 2026-04-02 06:49:56,059 - lava_callback - ERROR - Error processing callback for node 69cb6e858c06c2799bf587e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44008 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:50:35,938 - lava_callback - INFO - Received callback for node 69cb6eaa8c06c2799bf587f5 INFO: 172.20.0.1:59278 - "POST /node/69cb6eaa8c06c2799bf587f5 HTTP/1.0" 202 Accepted 2026-04-02 06:50:35,943 - lava_callback - INFO - Starting async job submit for node 69cb6eaa8c06c2799bf587f5 2026-04-02 06:50:35,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:50:35,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6eaa8c06c2799bf587f5 HTTP/1.1" 200 None 2026-04-02 06:50:35,967 - lava_callback - DEBUG - Processing node 69cb6eaa8c06c2799bf587f5: baseline-arm-baylibre 2026-04-02 06:50:35,968 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6eaa8c06c2799bf587f5 2026-04-02 06:50:35,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:50:35,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6eaa8c06c2799bf587f5 HTTP/1.1" 200 None 2026-04-02 06:50:35,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:50:35,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:50:36,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:50:36,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6eaa8c06c2799bf587f5 HTTP/1.1" 200 None 2026-04-02 06:50:36,027 - lava_callback - ERROR - Error processing callback for node 69cb6eaa8c06c2799bf587f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47138 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-02 06:51:16,159 - lava_callback - INFO - Received callback for node 69cb6ecf8c06c2799bf58802 INFO: 172.20.0.1:47154 - "POST /node/69cb6ecf8c06c2799bf58802 HTTP/1.0" 202 Accepted 2026-04-02 06:51:16,165 - lava_callback - INFO - Starting async job submit for node 69cb6ecf8c06c2799bf58802 2026-04-02 06:51:16,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:51:16,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6ecf8c06c2799bf58802 HTTP/1.1" 200 None 2026-04-02 06:51:16,180 - lava_callback - DEBUG - Processing node 69cb6ecf8c06c2799bf58802: baseline-arm-baylibre 2026-04-02 06:51:16,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6ecf8c06c2799bf58802 2026-04-02 06:51:16,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:51:16,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6ecf8c06c2799bf58802 HTTP/1.1" 200 None 2026-04-02 06:51:16,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:51:16,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:51:16,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:51:16,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6ecf8c06c2799bf58802 HTTP/1.1" 200 None 2026-04-02 06:51:16,231 - lava_callback - ERROR - Error processing callback for node 69cb6ecf8c06c2799bf58802: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49974 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:51:45,911 - lava_callback - INFO - Received callback for node 69cb6ef68c06c2799bf58810 INFO: 172.20.0.1:60510 - "POST /node/69cb6ef68c06c2799bf58810 HTTP/1.0" 202 Accepted 2026-04-02 06:51:45,916 - lava_callback - INFO - Starting async job submit for node 69cb6ef68c06c2799bf58810 2026-04-02 06:51:45,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:51:45,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6ef68c06c2799bf58810 HTTP/1.1" 200 None 2026-04-02 06:51:45,937 - lava_callback - DEBUG - Processing node 69cb6ef68c06c2799bf58810: baseline-arm-clabbe 2026-04-02 06:51:45,937 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6ef68c06c2799bf58810 2026-04-02 06:51:45,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:51:45,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6ef68c06c2799bf58810 HTTP/1.1" 200 None 2026-04-02 06:51:45,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:51:45,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:51:45,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:51:45,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6ef68c06c2799bf58810 HTTP/1.1" 200 None 2026-04-02 06:51:45,994 - lava_callback - ERROR - Error processing callback for node 69cb6ef68c06c2799bf58810: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46552 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:52:25,896 - lava_callback - INFO - Received callback for node 69cb6f1b8c06c2799bf5881a INFO: 172.20.0.1:52730 - "POST /node/69cb6f1b8c06c2799bf5881a HTTP/1.0" 202 Accepted 2026-04-02 06:52:25,902 - lava_callback - INFO - Starting async job submit for node 69cb6f1b8c06c2799bf5881a 2026-04-02 06:52:25,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:52:25,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6f1b8c06c2799bf5881a HTTP/1.1" 200 None 2026-04-02 06:52:25,917 - lava_callback - DEBUG - Processing node 69cb6f1b8c06c2799bf5881a: baseline-arm-clabbe 2026-04-02 06:52:25,918 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6f1b8c06c2799bf5881a 2026-04-02 06:52:25,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:52:25,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6f1b8c06c2799bf5881a HTTP/1.1" 200 None 2026-04-02 06:52:25,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:52:25,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:52:25,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:52:25,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6f1b8c06c2799bf5881a HTTP/1.1" 200 None 2026-04-02 06:52:25,973 - lava_callback - ERROR - Error processing callback for node 69cb6f1b8c06c2799bf5881a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42050 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:53:05,964 - lava_callback - INFO - Received callback for node 69cb6f3f8c06c2799bf58826 INFO: 172.20.0.1:35546 - "POST /node/69cb6f3f8c06c2799bf58826 HTTP/1.0" 202 Accepted 2026-04-02 06:53:05,969 - lava_callback - INFO - Starting async job submit for node 69cb6f3f8c06c2799bf58826 2026-04-02 06:53:05,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:53:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6f3f8c06c2799bf58826 HTTP/1.1" 200 None 2026-04-02 06:53:05,986 - lava_callback - DEBUG - Processing node 69cb6f3f8c06c2799bf58826: baseline-arm-clabbe 2026-04-02 06:53:05,987 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6f3f8c06c2799bf58826 2026-04-02 06:53:05,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:53:06,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6f3f8c06c2799bf58826 HTTP/1.1" 200 None 2026-04-02 06:53:06,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:53:06,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:53:06,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:53:06,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6f3f8c06c2799bf58826 HTTP/1.1" 200 None 2026-04-02 06:53:06,063 - lava_callback - ERROR - Error processing callback for node 69cb6f3f8c06c2799bf58826: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35716 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:53:45,941 - lava_callback - INFO - Received callback for node 69cb6f658c06c2799bf58833 INFO: 172.20.0.1:36044 - "POST /node/69cb6f658c06c2799bf58833 HTTP/1.0" 202 Accepted 2026-04-02 06:53:45,946 - lava_callback - INFO - Starting async job submit for node 69cb6f658c06c2799bf58833 2026-04-02 06:53:45,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:53:45,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6f658c06c2799bf58833 HTTP/1.1" 200 None 2026-04-02 06:53:45,963 - lava_callback - DEBUG - Processing node 69cb6f658c06c2799bf58833: baseline-arm-clabbe 2026-04-02 06:53:45,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6f658c06c2799bf58833 2026-04-02 06:53:45,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:53:45,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6f658c06c2799bf58833 HTTP/1.1" 200 None 2026-04-02 06:53:45,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:53:46,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:53:46,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:53:46,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6f658c06c2799bf58833 HTTP/1.1" 200 None 2026-04-02 06:53:46,064 - lava_callback - ERROR - Error processing callback for node 69cb6f658c06c2799bf58833: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56436 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:54:05,924 - lava_callback - INFO - Received callback for node 69cb6f8b8c06c2799bf58842 INFO: 172.20.0.1:50702 - "POST /node/69cb6f8b8c06c2799bf58842 HTTP/1.0" 202 Accepted 2026-04-02 06:54:05,930 - lava_callback - INFO - Starting async job submit for node 69cb6f8b8c06c2799bf58842 2026-04-02 06:54:05,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:54:05,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6f8b8c06c2799bf58842 HTTP/1.1" 200 None 2026-04-02 06:54:05,947 - lava_callback - DEBUG - Processing node 69cb6f8b8c06c2799bf58842: baseline-arm-clabbe 2026-04-02 06:54:05,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6f8b8c06c2799bf58842 2026-04-02 06:54:05,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:54:05,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6f8b8c06c2799bf58842 HTTP/1.1" 200 None 2026-04-02 06:54:05,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:54:05,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:54:05,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:54:05,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6f8b8c06c2799bf58842 HTTP/1.1" 200 None 2026-04-02 06:54:05,999 - lava_callback - ERROR - Error processing callback for node 69cb6f8b8c06c2799bf58842: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44078 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:54:45,966 - lava_callback - INFO - Received callback for node 69cb6faf8c06c2799bf5884f INFO: 172.20.0.1:44538 - "POST /node/69cb6faf8c06c2799bf5884f HTTP/1.0" 202 Accepted 2026-04-02 06:54:45,971 - lava_callback - INFO - Starting async job submit for node 69cb6faf8c06c2799bf5884f 2026-04-02 06:54:45,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:54:45,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6faf8c06c2799bf5884f HTTP/1.1" 200 None 2026-04-02 06:54:45,987 - lava_callback - DEBUG - Processing node 69cb6faf8c06c2799bf5884f: baseline-arm-clabbe 2026-04-02 06:54:45,987 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6faf8c06c2799bf5884f 2026-04-02 06:54:45,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:54:46,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6faf8c06c2799bf5884f HTTP/1.1" 200 None 2026-04-02 06:54:46,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:54:46,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:54:46,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:54:46,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6faf8c06c2799bf5884f HTTP/1.1" 200 None 2026-04-02 06:54:46,038 - lava_callback - ERROR - Error processing callback for node 69cb6faf8c06c2799bf5884f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43284 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:55:26,067 - lava_callback - INFO - Received callback for node 69cb6fd58c06c2799bf58859 INFO: 172.20.0.1:59656 - "POST /node/69cb6fd58c06c2799bf58859 HTTP/1.0" 202 Accepted 2026-04-02 06:55:26,073 - lava_callback - INFO - Starting async job submit for node 69cb6fd58c06c2799bf58859 2026-04-02 06:55:26,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:55:26,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6fd58c06c2799bf58859 HTTP/1.1" 200 None 2026-04-02 06:55:26,089 - lava_callback - DEBUG - Processing node 69cb6fd58c06c2799bf58859: baseline-arm-clabbe 2026-04-02 06:55:26,089 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6fd58c06c2799bf58859 2026-04-02 06:55:26,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:55:26,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6fd58c06c2799bf58859 HTTP/1.1" 200 None 2026-04-02 06:55:26,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:55:26,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:55:26,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:55:26,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6fd58c06c2799bf58859 HTTP/1.1" 200 None 2026-04-02 06:55:26,155 - lava_callback - ERROR - Error processing callback for node 69cb6fd58c06c2799bf58859: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32778 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:56:06,094 - lava_callback - INFO - Received callback for node 69cb6ffa8c06c2799bf58866 INFO: 172.20.0.1:53322 - "POST /node/69cb6ffa8c06c2799bf58866 HTTP/1.0" 202 Accepted 2026-04-02 06:56:06,099 - lava_callback - INFO - Starting async job submit for node 69cb6ffa8c06c2799bf58866 2026-04-02 06:56:06,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:56:06,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6ffa8c06c2799bf58866 HTTP/1.1" 200 None 2026-04-02 06:56:06,127 - lava_callback - DEBUG - Processing node 69cb6ffa8c06c2799bf58866: baseline-arm-clabbe 2026-04-02 06:56:06,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb6ffa8c06c2799bf58866 2026-04-02 06:56:06,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:56:06,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb6ffa8c06c2799bf58866 HTTP/1.1" 200 None 2026-04-02 06:56:06,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:56:06,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:56:06,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:56:06,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb6ffa8c06c2799bf58866 HTTP/1.1" 200 None 2026-04-02 06:56:06,183 - lava_callback - ERROR - Error processing callback for node 69cb6ffa8c06c2799bf58866: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50390 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:56:46,103 - lava_callback - INFO - Received callback for node 69cb701f8c06c2799bf58872 INFO: 172.20.0.1:40642 - "POST /node/69cb701f8c06c2799bf58872 HTTP/1.0" 202 Accepted 2026-04-02 06:56:46,109 - lava_callback - INFO - Starting async job submit for node 69cb701f8c06c2799bf58872 2026-04-02 06:56:46,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:56:46,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb701f8c06c2799bf58872 HTTP/1.1" 200 None 2026-04-02 06:56:46,125 - lava_callback - DEBUG - Processing node 69cb701f8c06c2799bf58872: baseline-arm-clabbe 2026-04-02 06:56:46,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb701f8c06c2799bf58872 2026-04-02 06:56:46,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:56:46,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb701f8c06c2799bf58872 HTTP/1.1" 200 None 2026-04-02 06:56:46,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:56:46,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:56:46,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:56:46,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb701f8c06c2799bf58872 HTTP/1.1" 200 None 2026-04-02 06:56:46,180 - lava_callback - ERROR - Error processing callback for node 69cb701f8c06c2799bf58872: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44070 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:57:26,068 - lava_callback - INFO - Received callback for node 69cb70458c06c2799bf5889b INFO: 172.20.0.1:60070 - "POST /node/69cb70458c06c2799bf5889b HTTP/1.0" 202 Accepted 2026-04-02 06:57:26,075 - lava_callback - INFO - Starting async job submit for node 69cb70458c06c2799bf5889b 2026-04-02 06:57:26,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:57:26,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb70458c06c2799bf5889b HTTP/1.1" 200 None 2026-04-02 06:57:26,095 - lava_callback - DEBUG - Processing node 69cb70458c06c2799bf5889b: baseline-arm-clabbe 2026-04-02 06:57:26,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb70458c06c2799bf5889b 2026-04-02 06:57:26,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:57:26,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb70458c06c2799bf5889b HTTP/1.1" 200 None 2026-04-02 06:57:26,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:57:26,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:57:26,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:57:26,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb70458c06c2799bf5889b HTTP/1.1" 200 None 2026-04-02 06:57:26,147 - lava_callback - ERROR - Error processing callback for node 69cb70458c06c2799bf5889b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43566 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:58:06,140 - lava_callback - INFO - Received callback for node 69cb706a8c06c2799bf588b1 INFO: 172.20.0.1:38182 - "POST /node/69cb706a8c06c2799bf588b1 HTTP/1.0" 202 Accepted 2026-04-02 06:58:06,145 - lava_callback - INFO - Starting async job submit for node 69cb706a8c06c2799bf588b1 2026-04-02 06:58:06,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:58:06,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb706a8c06c2799bf588b1 HTTP/1.1" 200 None 2026-04-02 06:58:06,161 - lava_callback - DEBUG - Processing node 69cb706a8c06c2799bf588b1: baseline-arm-clabbe 2026-04-02 06:58:06,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb706a8c06c2799bf588b1 2026-04-02 06:58:06,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:58:06,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb706a8c06c2799bf588b1 HTTP/1.1" 200 None 2026-04-02 06:58:06,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:58:06,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:58:06,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:58:06,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb706a8c06c2799bf588b1 HTTP/1.1" 200 None 2026-04-02 06:58:06,212 - lava_callback - ERROR - Error processing callback for node 69cb706a8c06c2799bf588b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42652 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:58:26,149 - lava_callback - INFO - Received callback for node 69cb708f8c06c2799bf588be INFO: 172.20.0.1:60032 - "POST /node/69cb708f8c06c2799bf588be HTTP/1.0" 202 Accepted 2026-04-02 06:58:26,154 - lava_callback - INFO - Starting async job submit for node 69cb708f8c06c2799bf588be 2026-04-02 06:58:26,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:58:26,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb708f8c06c2799bf588be HTTP/1.1" 200 None 2026-04-02 06:58:26,171 - lava_callback - DEBUG - Processing node 69cb708f8c06c2799bf588be: baseline-arm-clabbe 2026-04-02 06:58:26,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb708f8c06c2799bf588be 2026-04-02 06:58:26,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:58:26,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb708f8c06c2799bf588be HTTP/1.1" 200 None 2026-04-02 06:58:26,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:58:26,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:58:26,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:58:26,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb708f8c06c2799bf588be HTTP/1.1" 200 None 2026-04-02 06:58:26,235 - lava_callback - ERROR - Error processing callback for node 69cb708f8c06c2799bf588be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33194 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:59:06,211 - lava_callback - INFO - Received callback for node 69cb70b48c06c2799bf588c7 INFO: 172.20.0.1:50182 - "POST /node/69cb70b48c06c2799bf588c7 HTTP/1.0" 202 Accepted 2026-04-02 06:59:06,217 - lava_callback - INFO - Starting async job submit for node 69cb70b48c06c2799bf588c7 2026-04-02 06:59:06,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:59:06,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb70b48c06c2799bf588c7 HTTP/1.1" 200 None 2026-04-02 06:59:06,232 - lava_callback - DEBUG - Processing node 69cb70b48c06c2799bf588c7: baseline-arm-clabbe 2026-04-02 06:59:06,233 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb70b48c06c2799bf588c7 2026-04-02 06:59:06,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:59:06,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb70b48c06c2799bf588c7 HTTP/1.1" 200 None 2026-04-02 06:59:06,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:59:06,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-02 06:59:06,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:59:06,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb70b48c06c2799bf588c7 HTTP/1.1" 200 None 2026-04-02 06:59:06,294 - lava_callback - ERROR - Error processing callback for node 69cb70b48c06c2799bf588c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49024 - "GET / HTTP/1.0" 200 OK 2026-04-02 06:59:46,164 - lava_callback - INFO - Received callback for node 69cb70d98c06c2799bf588d3 INFO: 172.20.0.1:38614 - "POST /node/69cb70d98c06c2799bf588d3 HTTP/1.0" 202 Accepted 2026-04-02 06:59:46,170 - lava_callback - INFO - Starting async job submit for node 69cb70d98c06c2799bf588d3 2026-04-02 06:59:46,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:59:46,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb70d98c06c2799bf588d3 HTTP/1.1" 200 None 2026-04-02 06:59:46,186 - lava_callback - DEBUG - Processing node 69cb70d98c06c2799bf588d3: baseline-arm-clabbe 2026-04-02 06:59:46,186 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb70d98c06c2799bf588d3 2026-04-02 06:59:46,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:59:46,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb70d98c06c2799bf588d3 HTTP/1.1" 200 None 2026-04-02 06:59:46,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:59:46,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 06:59:46,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 06:59:46,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb70d98c06c2799bf588d3 HTTP/1.1" 200 None 2026-04-02 06:59:46,242 - lava_callback - ERROR - Error processing callback for node 69cb70d98c06c2799bf588d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54544 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:00:26,285 - lava_callback - INFO - Received callback for node 69cb70fe8c06c2799bf588db INFO: 172.20.0.1:54214 - "POST /node/69cb70fe8c06c2799bf588db HTTP/1.0" 202 Accepted 2026-04-02 07:00:26,291 - lava_callback - INFO - Starting async job submit for node 69cb70fe8c06c2799bf588db 2026-04-02 07:00:26,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:00:26,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb70fe8c06c2799bf588db HTTP/1.1" 200 None 2026-04-02 07:00:26,307 - lava_callback - DEBUG - Processing node 69cb70fe8c06c2799bf588db: baseline-arm-clabbe 2026-04-02 07:00:26,307 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb70fe8c06c2799bf588db 2026-04-02 07:00:26,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:00:26,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb70fe8c06c2799bf588db HTTP/1.1" 200 None 2026-04-02 07:00:26,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:00:26,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 07:00:26,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:00:26,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb70fe8c06c2799bf588db HTTP/1.1" 200 None 2026-04-02 07:00:26,364 - lava_callback - ERROR - Error processing callback for node 69cb70fe8c06c2799bf588db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59876 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:01:06,248 - lava_callback - INFO - Received callback for node 69cb71238c06c2799bf588e6 INFO: 172.20.0.1:38440 - "POST /node/69cb71238c06c2799bf588e6 HTTP/1.0" 202 Accepted 2026-04-02 07:01:06,258 - lava_callback - INFO - Starting async job submit for node 69cb71238c06c2799bf588e6 2026-04-02 07:01:06,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:01:06,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb71238c06c2799bf588e6 HTTP/1.1" 200 None 2026-04-02 07:01:06,278 - lava_callback - DEBUG - Processing node 69cb71238c06c2799bf588e6: baseline-arm-clabbe 2026-04-02 07:01:06,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb71238c06c2799bf588e6 2026-04-02 07:01:06,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:01:06,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb71238c06c2799bf588e6 HTTP/1.1" 200 None 2026-04-02 07:01:06,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:01:06,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 07:01:06,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:01:06,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb71238c06c2799bf588e6 HTTP/1.1" 200 None 2026-04-02 07:01:06,350 - lava_callback - ERROR - Error processing callback for node 69cb71238c06c2799bf588e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51682 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:01:46,296 - lava_callback - INFO - Received callback for node 69cb71488c06c2799bf588f0 INFO: 172.20.0.1:60000 - "POST /node/69cb71488c06c2799bf588f0 HTTP/1.0" 202 Accepted 2026-04-02 07:01:46,302 - lava_callback - INFO - Starting async job submit for node 69cb71488c06c2799bf588f0 2026-04-02 07:01:46,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:01:46,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb71488c06c2799bf588f0 HTTP/1.1" 200 None 2026-04-02 07:01:46,318 - lava_callback - DEBUG - Processing node 69cb71488c06c2799bf588f0: baseline-arm-clabbe 2026-04-02 07:01:46,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb71488c06c2799bf588f0 2026-04-02 07:01:46,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:01:46,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb71488c06c2799bf588f0 HTTP/1.1" 200 None 2026-04-02 07:01:46,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:01:46,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 07:01:46,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:01:46,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb71488c06c2799bf588f0 HTTP/1.1" 200 None 2026-04-02 07:01:46,377 - lava_callback - ERROR - Error processing callback for node 69cb71488c06c2799bf588f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52756 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:02:26,384 - lava_callback - INFO - Received callback for node 69cb716d8c06c2799bf588fc INFO: 172.20.0.1:56100 - "POST /node/69cb716d8c06c2799bf588fc HTTP/1.0" 202 Accepted 2026-04-02 07:02:26,390 - lava_callback - INFO - Starting async job submit for node 69cb716d8c06c2799bf588fc 2026-04-02 07:02:26,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:02:26,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb716d8c06c2799bf588fc HTTP/1.1" 200 None 2026-04-02 07:02:26,407 - lava_callback - DEBUG - Processing node 69cb716d8c06c2799bf588fc: baseline-arm-clabbe 2026-04-02 07:02:26,407 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb716d8c06c2799bf588fc 2026-04-02 07:02:26,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:02:26,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb716d8c06c2799bf588fc HTTP/1.1" 200 None 2026-04-02 07:02:26,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:02:26,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 07:02:26,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:02:26,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb716d8c06c2799bf588fc HTTP/1.1" 200 None 2026-04-02 07:02:26,470 - lava_callback - ERROR - Error processing callback for node 69cb716d8c06c2799bf588fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42126 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:02:46,634 - lava_callback - INFO - Received callback for node 69cb71928c06c2799bf58904 INFO: 172.20.0.1:42136 - "POST /node/69cb71928c06c2799bf58904 HTTP/1.0" 202 Accepted 2026-04-02 07:02:46,639 - lava_callback - INFO - Starting async job submit for node 69cb71928c06c2799bf58904 2026-04-02 07:02:46,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:02:46,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb71928c06c2799bf58904 HTTP/1.1" 200 None 2026-04-02 07:02:46,663 - lava_callback - DEBUG - Processing node 69cb71928c06c2799bf58904: baseline-arm-clabbe 2026-04-02 07:02:46,663 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb71928c06c2799bf58904 2026-04-02 07:02:46,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:02:46,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb71928c06c2799bf58904 HTTP/1.1" 200 None 2026-04-02 07:02:46,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:02:46,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 07:02:46,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:02:46,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb71928c06c2799bf58904 HTTP/1.1" 200 None 2026-04-02 07:02:46,721 - lava_callback - ERROR - Error processing callback for node 69cb71928c06c2799bf58904: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34796 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:03:26,417 - lava_callback - INFO - Received callback for node 69cb71b78c06c2799bf5890d INFO: 172.20.0.1:53462 - "POST /node/69cb71b78c06c2799bf5890d HTTP/1.0" 202 Accepted 2026-04-02 07:03:26,422 - lava_callback - INFO - Starting async job submit for node 69cb71b78c06c2799bf5890d 2026-04-02 07:03:26,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:03:26,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb71b78c06c2799bf5890d HTTP/1.1" 200 None 2026-04-02 07:03:26,441 - lava_callback - DEBUG - Processing node 69cb71b78c06c2799bf5890d: baseline-arm-clabbe 2026-04-02 07:03:26,441 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb71b78c06c2799bf5890d 2026-04-02 07:03:26,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:03:26,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb71b78c06c2799bf5890d HTTP/1.1" 200 None 2026-04-02 07:03:26,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:03:26,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 07:03:26,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:03:26,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb71b78c06c2799bf5890d HTTP/1.1" 200 None 2026-04-02 07:03:26,497 - lava_callback - ERROR - Error processing callback for node 69cb71b78c06c2799bf5890d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35622 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:04:06,447 - lava_callback - INFO - Received callback for node 69cb71dc8c06c2799bf58917 INFO: 172.20.0.1:46256 - "POST /node/69cb71dc8c06c2799bf58917 HTTP/1.0" 202 Accepted 2026-04-02 07:04:06,452 - lava_callback - INFO - Starting async job submit for node 69cb71dc8c06c2799bf58917 2026-04-02 07:04:06,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:04:06,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb71dc8c06c2799bf58917 HTTP/1.1" 200 None 2026-04-02 07:04:06,469 - lava_callback - DEBUG - Processing node 69cb71dc8c06c2799bf58917: baseline-arm-clabbe 2026-04-02 07:04:06,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb71dc8c06c2799bf58917 2026-04-02 07:04:06,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:04:06,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb71dc8c06c2799bf58917 HTTP/1.1" 200 None 2026-04-02 07:04:06,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:04:06,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-02 07:04:06,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:46258 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:04:06,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb71dc8c06c2799bf58917 HTTP/1.1" 200 None 2026-04-02 07:04:06,538 - lava_callback - ERROR - Error processing callback for node 69cb71dc8c06c2799bf58917: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54144 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:05:56,768 - lava_callback - INFO - Received callback for node 69cb72448c06c2799bf58935 INFO: 172.20.0.1:54146 - "POST /node/69cb72448c06c2799bf58935 HTTP/1.0" 202 Accepted 2026-04-02 07:05:56,774 - lava_callback - INFO - Starting async job submit for node 69cb72448c06c2799bf58935 2026-04-02 07:05:56,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:05:56,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb72448c06c2799bf58935 HTTP/1.1" 200 None 2026-04-02 07:05:56,790 - lava_callback - DEBUG - Processing node 69cb72448c06c2799bf58935: baseline-arm-baylibre 2026-04-02 07:05:56,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb72448c06c2799bf58935 2026-04-02 07:05:56,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:05:56,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb72448c06c2799bf58935 HTTP/1.1" 200 None 2026-04-02 07:05:56,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:05:56,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:05:56,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:05:56,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb72448c06c2799bf58935 HTTP/1.1" 200 None 2026-04-02 07:05:56,858 - lava_callback - ERROR - Error processing callback for node 69cb72448c06c2799bf58935: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58726 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:06:36,766 - lava_callback - INFO - Received callback for node 69cb726a8c06c2799bf5893f INFO: 172.20.0.1:58730 - "POST /node/69cb726a8c06c2799bf5893f HTTP/1.0" 202 Accepted 2026-04-02 07:06:36,771 - lava_callback - INFO - Starting async job submit for node 69cb726a8c06c2799bf5893f 2026-04-02 07:06:36,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:06:36,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb726a8c06c2799bf5893f HTTP/1.1" 200 None 2026-04-02 07:06:36,799 - lava_callback - DEBUG - Processing node 69cb726a8c06c2799bf5893f: baseline-arm-baylibre 2026-04-02 07:06:36,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb726a8c06c2799bf5893f 2026-04-02 07:06:36,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:06:36,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb726a8c06c2799bf5893f HTTP/1.1" 200 None 2026-04-02 07:06:36,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:06:36,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:06:36,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:06:36,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb726a8c06c2799bf5893f HTTP/1.1" 200 None 2026-04-02 07:06:36,854 - lava_callback - ERROR - Error processing callback for node 69cb726a8c06c2799bf5893f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44854 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:06:56,735 - lava_callback - INFO - Received callback for node 69cb728e8c06c2799bf58949 INFO: 172.20.0.1:44866 - "POST /node/69cb728e8c06c2799bf58949 HTTP/1.0" 202 Accepted 2026-04-02 07:06:56,741 - lava_callback - INFO - Starting async job submit for node 69cb728e8c06c2799bf58949 2026-04-02 07:06:56,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:06:56,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb728e8c06c2799bf58949 HTTP/1.1" 200 None 2026-04-02 07:06:56,760 - lava_callback - DEBUG - Processing node 69cb728e8c06c2799bf58949: baseline-arm-baylibre 2026-04-02 07:06:56,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb728e8c06c2799bf58949 2026-04-02 07:06:56,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:06:56,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb728e8c06c2799bf58949 HTTP/1.1" 200 None 2026-04-02 07:06:56,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:06:56,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:06:56,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:06:56,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb728e8c06c2799bf58949 HTTP/1.1" 200 None 2026-04-02 07:06:56,814 - lava_callback - ERROR - Error processing callback for node 69cb728e8c06c2799bf58949: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58384 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:07:36,810 - lava_callback - INFO - Received callback for node 69cb72b38c06c2799bf58956 INFO: 172.20.0.1:58388 - "POST /node/69cb72b38c06c2799bf58956 HTTP/1.0" 202 Accepted 2026-04-02 07:07:36,815 - lava_callback - INFO - Starting async job submit for node 69cb72b38c06c2799bf58956 2026-04-02 07:07:36,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:07:36,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb72b38c06c2799bf58956 HTTP/1.1" 200 None 2026-04-02 07:07:36,832 - lava_callback - DEBUG - Processing node 69cb72b38c06c2799bf58956: baseline-arm-baylibre 2026-04-02 07:07:36,832 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb72b38c06c2799bf58956 2026-04-02 07:07:36,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:07:36,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb72b38c06c2799bf58956 HTTP/1.1" 200 None 2026-04-02 07:07:36,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:07:36,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:07:36,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:07:36,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb72b38c06c2799bf58956 HTTP/1.1" 200 None 2026-04-02 07:07:36,905 - lava_callback - ERROR - Error processing callback for node 69cb72b38c06c2799bf58956: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50568 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:08:16,836 - lava_callback - INFO - Received callback for node 69cb72d98c06c2799bf58960 INFO: 172.20.0.1:50584 - "POST /node/69cb72d98c06c2799bf58960 HTTP/1.0" 202 Accepted 2026-04-02 07:08:16,842 - lava_callback - INFO - Starting async job submit for node 69cb72d98c06c2799bf58960 2026-04-02 07:08:16,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:08:16,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb72d98c06c2799bf58960 HTTP/1.1" 200 None 2026-04-02 07:08:16,858 - lava_callback - DEBUG - Processing node 69cb72d98c06c2799bf58960: baseline-arm-baylibre 2026-04-02 07:08:16,858 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb72d98c06c2799bf58960 2026-04-02 07:08:16,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:08:16,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb72d98c06c2799bf58960 HTTP/1.1" 200 None 2026-04-02 07:08:16,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:08:16,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:08:16,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:08:16,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb72d98c06c2799bf58960 HTTP/1.1" 200 None 2026-04-02 07:08:16,914 - lava_callback - ERROR - Error processing callback for node 69cb72d98c06c2799bf58960: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53930 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:08:56,935 - lava_callback - INFO - Received callback for node 69cb72fd8c06c2799bf5896b INFO: 172.20.0.1:53938 - "POST /node/69cb72fd8c06c2799bf5896b HTTP/1.0" 202 Accepted 2026-04-02 07:08:56,941 - lava_callback - INFO - Starting async job submit for node 69cb72fd8c06c2799bf5896b 2026-04-02 07:08:56,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:08:56,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb72fd8c06c2799bf5896b HTTP/1.1" 200 None 2026-04-02 07:08:56,962 - lava_callback - DEBUG - Processing node 69cb72fd8c06c2799bf5896b: baseline-arm-baylibre 2026-04-02 07:08:56,962 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb72fd8c06c2799bf5896b 2026-04-02 07:08:56,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:08:56,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb72fd8c06c2799bf5896b HTTP/1.1" 200 None 2026-04-02 07:08:56,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:08:56,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:08:56,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:08:57,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb72fd8c06c2799bf5896b HTTP/1.1" 200 None 2026-04-02 07:08:57,021 - lava_callback - ERROR - Error processing callback for node 69cb72fd8c06c2799bf5896b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45646 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:10:46,681 - lava_callback - INFO - Received callback for node 69cb736b8c06c2799bf5898d INFO: 172.20.0.1:45660 - "POST /node/69cb736b8c06c2799bf5898d HTTP/1.0" 202 Accepted 2026-04-02 07:10:46,687 - lava_callback - INFO - Starting async job submit for node 69cb736b8c06c2799bf5898d 2026-04-02 07:10:46,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:10:46,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb736b8c06c2799bf5898d HTTP/1.1" 200 None 2026-04-02 07:10:46,704 - lava_callback - DEBUG - Processing node 69cb736b8c06c2799bf5898d: baseline-arm-clabbe 2026-04-02 07:10:46,704 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb736b8c06c2799bf5898d 2026-04-02 07:10:46,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:10:46,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb736b8c06c2799bf5898d HTTP/1.1" 200 None 2026-04-02 07:10:46,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:10:46,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:10:46,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:10:46,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb736b8c06c2799bf5898d HTTP/1.1" 200 None 2026-04-02 07:10:46,761 - lava_callback - ERROR - Error processing callback for node 69cb736b8c06c2799bf5898d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47436 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:11:26,732 - lava_callback - INFO - Received callback for node 69cb738f8c06c2799bf58996 INFO: 172.20.0.1:47446 - "POST /node/69cb738f8c06c2799bf58996 HTTP/1.0" 202 Accepted 2026-04-02 07:11:26,737 - lava_callback - INFO - Starting async job submit for node 69cb738f8c06c2799bf58996 2026-04-02 07:11:26,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:11:26,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb738f8c06c2799bf58996 HTTP/1.1" 200 None 2026-04-02 07:11:26,754 - lava_callback - DEBUG - Processing node 69cb738f8c06c2799bf58996: baseline-arm-clabbe 2026-04-02 07:11:26,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb738f8c06c2799bf58996 2026-04-02 07:11:26,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:11:26,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb738f8c06c2799bf58996 HTTP/1.1" 200 None 2026-04-02 07:11:26,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:11:26,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:11:26,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:11:26,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb738f8c06c2799bf58996 HTTP/1.1" 200 None 2026-04-02 07:11:26,803 - lava_callback - ERROR - Error processing callback for node 69cb738f8c06c2799bf58996: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46550 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:12:06,751 - lava_callback - INFO - Received callback for node 69cb73b48c06c2799bf5899f INFO: 172.20.0.1:46556 - "POST /node/69cb73b48c06c2799bf5899f HTTP/1.0" 202 Accepted 2026-04-02 07:12:06,757 - lava_callback - INFO - Starting async job submit for node 69cb73b48c06c2799bf5899f 2026-04-02 07:12:06,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:12:06,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb73b48c06c2799bf5899f HTTP/1.1" 200 None 2026-04-02 07:12:06,772 - lava_callback - DEBUG - Processing node 69cb73b48c06c2799bf5899f: baseline-arm-clabbe 2026-04-02 07:12:06,773 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb73b48c06c2799bf5899f 2026-04-02 07:12:06,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:12:06,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb73b48c06c2799bf5899f HTTP/1.1" 200 None 2026-04-02 07:12:06,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:12:06,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:12:06,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:12:06,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb73b48c06c2799bf5899f HTTP/1.1" 200 None 2026-04-02 07:12:06,832 - lava_callback - ERROR - Error processing callback for node 69cb73b48c06c2799bf5899f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52794 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:12:46,742 - lava_callback - INFO - Received callback for node 69cb73d98c06c2799bf589c8 INFO: 172.20.0.1:52802 - "POST /node/69cb73d98c06c2799bf589c8 HTTP/1.0" 202 Accepted 2026-04-02 07:12:46,748 - lava_callback - INFO - Starting async job submit for node 69cb73d98c06c2799bf589c8 2026-04-02 07:12:46,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:12:46,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb73d98c06c2799bf589c8 HTTP/1.1" 200 None 2026-04-02 07:12:46,763 - lava_callback - DEBUG - Processing node 69cb73d98c06c2799bf589c8: baseline-arm-clabbe 2026-04-02 07:12:46,764 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb73d98c06c2799bf589c8 2026-04-02 07:12:46,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:12:46,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb73d98c06c2799bf589c8 HTTP/1.1" 200 None 2026-04-02 07:12:46,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:12:46,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:12:46,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:12:46,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb73d98c06c2799bf589c8 HTTP/1.1" 200 None 2026-04-02 07:12:46,826 - lava_callback - ERROR - Error processing callback for node 69cb73d98c06c2799bf589c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47414 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:13:06,832 - lava_callback - INFO - Received callback for node 69cb73fe8c06c2799bf589df INFO: 172.20.0.1:47420 - "POST /node/69cb73fe8c06c2799bf589df HTTP/1.0" 202 Accepted 2026-04-02 07:13:06,837 - lava_callback - INFO - Starting async job submit for node 69cb73fe8c06c2799bf589df 2026-04-02 07:13:06,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:13:06,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb73fe8c06c2799bf589df HTTP/1.1" 200 None 2026-04-02 07:13:06,853 - lava_callback - DEBUG - Processing node 69cb73fe8c06c2799bf589df: baseline-arm-clabbe 2026-04-02 07:13:06,853 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb73fe8c06c2799bf589df 2026-04-02 07:13:06,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:13:06,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb73fe8c06c2799bf589df HTTP/1.1" 200 None 2026-04-02 07:13:06,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:13:06,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-02 07:13:06,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:13:06,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb73fe8c06c2799bf589df HTTP/1.1" 200 None 2026-04-02 07:13:06,940 - lava_callback - ERROR - Error processing callback for node 69cb73fe8c06c2799bf589df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37922 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:13:57,195 - lava_callback - INFO - Received callback for node 69cb742a8c06c2799bf589ef INFO: 172.20.0.1:37932 - "POST /node/69cb742a8c06c2799bf589ef HTTP/1.0" 202 Accepted 2026-04-02 07:13:57,200 - lava_callback - INFO - Starting async job submit for node 69cb742a8c06c2799bf589ef 2026-04-02 07:13:57,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:13:57,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb742a8c06c2799bf589ef HTTP/1.1" 200 None 2026-04-02 07:13:57,216 - lava_callback - DEBUG - Processing node 69cb742a8c06c2799bf589ef: baseline-arm64-baylibre 2026-04-02 07:13:57,216 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb742a8c06c2799bf589ef 2026-04-02 07:13:57,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:13:57,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb742a8c06c2799bf589ef HTTP/1.1" 200 None 2026-04-02 07:13:57,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:13:57,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:13:57,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:13:57,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb742a8c06c2799bf589ef HTTP/1.1" 200 None 2026-04-02 07:13:57,267 - lava_callback - ERROR - Error processing callback for node 69cb742a8c06c2799bf589ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43104 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:15:17,293 - lava_callback - INFO - Received callback for node 69cb74788c06c2799bf589ff INFO: 172.20.0.1:43120 - "POST /node/69cb74788c06c2799bf589ff HTTP/1.0" 202 Accepted 2026-04-02 07:15:17,298 - lava_callback - INFO - Starting async job submit for node 69cb74788c06c2799bf589ff 2026-04-02 07:15:17,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:15:17,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb74788c06c2799bf589ff HTTP/1.1" 200 None 2026-04-02 07:15:17,314 - lava_callback - DEBUG - Processing node 69cb74788c06c2799bf589ff: baseline-arm64-baylibre 2026-04-02 07:15:17,314 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb74788c06c2799bf589ff 2026-04-02 07:15:17,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:15:17,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb74788c06c2799bf589ff HTTP/1.1" 200 None 2026-04-02 07:15:17,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:15:17,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:15:17,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:15:17,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb74788c06c2799bf589ff HTTP/1.1" 200 None 2026-04-02 07:15:17,388 - lava_callback - ERROR - Error processing callback for node 69cb74788c06c2799bf589ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38280 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:16:37,358 - lava_callback - INFO - Received callback for node 69cb74c48c06c2799bf58a09 INFO: 172.20.0.1:38296 - "POST /node/69cb74c48c06c2799bf58a09 HTTP/1.0" 202 Accepted 2026-04-02 07:16:37,364 - lava_callback - INFO - Starting async job submit for node 69cb74c48c06c2799bf58a09 2026-04-02 07:16:37,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:16:37,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb74c48c06c2799bf58a09 HTTP/1.1" 200 None 2026-04-02 07:16:37,380 - lava_callback - DEBUG - Processing node 69cb74c48c06c2799bf58a09: baseline-arm64-baylibre 2026-04-02 07:16:37,380 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb74c48c06c2799bf58a09 2026-04-02 07:16:37,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:16:37,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb74c48c06c2799bf58a09 HTTP/1.1" 200 None 2026-04-02 07:16:37,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:16:37,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:16:37,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:16:37,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb74c48c06c2799bf58a09 HTTP/1.1" 200 None 2026-04-02 07:16:37,432 - lava_callback - ERROR - Error processing callback for node 69cb74c48c06c2799bf58a09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52372 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:17:57,428 - lava_callback - INFO - Received callback for node 69cb75128c06c2799bf58a18 INFO: 172.20.0.1:52378 - "POST /node/69cb75128c06c2799bf58a18 HTTP/1.0" 202 Accepted 2026-04-02 07:17:57,434 - lava_callback - INFO - Starting async job submit for node 69cb75128c06c2799bf58a18 2026-04-02 07:17:57,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:17:57,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb75128c06c2799bf58a18 HTTP/1.1" 200 None 2026-04-02 07:17:57,454 - lava_callback - DEBUG - Processing node 69cb75128c06c2799bf58a18: baseline-arm64-baylibre 2026-04-02 07:17:57,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb75128c06c2799bf58a18 2026-04-02 07:17:57,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:17:57,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb75128c06c2799bf58a18 HTTP/1.1" 200 None 2026-04-02 07:17:57,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:17:57,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:17:57,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:17:57,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb75128c06c2799bf58a18 HTTP/1.1" 200 None 2026-04-02 07:17:57,513 - lava_callback - ERROR - Error processing callback for node 69cb75128c06c2799bf58a18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45310 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:18:57,437 - lava_callback - INFO - Received callback for node 69cb755d8c06c2799bf58a24 INFO: 172.20.0.1:45314 - "POST /node/69cb755d8c06c2799bf58a24 HTTP/1.0" 202 Accepted 2026-04-02 07:18:57,442 - lava_callback - INFO - Starting async job submit for node 69cb755d8c06c2799bf58a24 2026-04-02 07:18:57,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:18:57,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb755d8c06c2799bf58a24 HTTP/1.1" 200 None 2026-04-02 07:18:57,458 - lava_callback - DEBUG - Processing node 69cb755d8c06c2799bf58a24: baseline-arm64-baylibre 2026-04-02 07:18:57,458 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb755d8c06c2799bf58a24 2026-04-02 07:18:57,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:18:57,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb755d8c06c2799bf58a24 HTTP/1.1" 200 None 2026-04-02 07:18:57,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:18:57,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:18:57,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:18:57,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb755d8c06c2799bf58a24 HTTP/1.1" 200 None 2026-04-02 07:18:57,513 - lava_callback - ERROR - Error processing callback for node 69cb755d8c06c2799bf58a24: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46812 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:20:17,559 - lava_callback - INFO - Received callback for node 69cb75ad8c06c2799bf58a35 INFO: 172.20.0.1:46822 - "POST /node/69cb75ad8c06c2799bf58a35 HTTP/1.0" 202 Accepted 2026-04-02 07:20:17,564 - lava_callback - INFO - Starting async job submit for node 69cb75ad8c06c2799bf58a35 2026-04-02 07:20:17,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:20:17,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb75ad8c06c2799bf58a35 HTTP/1.1" 200 None 2026-04-02 07:20:17,581 - lava_callback - DEBUG - Processing node 69cb75ad8c06c2799bf58a35: baseline-arm64-baylibre 2026-04-02 07:20:17,581 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb75ad8c06c2799bf58a35 2026-04-02 07:20:17,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:20:17,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb75ad8c06c2799bf58a35 HTTP/1.1" 200 None 2026-04-02 07:20:17,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:20:17,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:20:17,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:20:17,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb75ad8c06c2799bf58a35 HTTP/1.1" 200 None 2026-04-02 07:20:17,637 - lava_callback - ERROR - Error processing callback for node 69cb75ad8c06c2799bf58a35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55816 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:29:18,071 - lava_callback - INFO - Received callback for node 69cb77c48c06c2799bf58a99 INFO: 172.20.0.1:55826 - "POST /node/69cb77c48c06c2799bf58a99 HTTP/1.0" 202 Accepted 2026-04-02 07:29:18,077 - lava_callback - INFO - Starting async job submit for node 69cb77c48c06c2799bf58a99 2026-04-02 07:29:18,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:29:18,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb77c48c06c2799bf58a99 HTTP/1.1" 200 None 2026-04-02 07:29:18,093 - lava_callback - DEBUG - Processing node 69cb77c48c06c2799bf58a99: baseline-arm64-baylibre 2026-04-02 07:29:18,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb77c48c06c2799bf58a99 2026-04-02 07:29:18,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:29:18,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb77c48c06c2799bf58a99 HTTP/1.1" 200 None 2026-04-02 07:29:18,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:29:18,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 07:29:18,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:29:18,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb77c48c06c2799bf58a99 HTTP/1.1" 200 None 2026-04-02 07:29:18,150 - lava_callback - ERROR - Error processing callback for node 69cb77c48c06c2799bf58a99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57760 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:33:18,271 - lava_callback - INFO - Received callback for node 69cb78ac8c06c2799bf58ab9 INFO: 172.20.0.1:57776 - "POST /node/69cb78ac8c06c2799bf58ab9 HTTP/1.0" 202 Accepted 2026-04-02 07:33:18,276 - lava_callback - INFO - Starting async job submit for node 69cb78ac8c06c2799bf58ab9 2026-04-02 07:33:18,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:33:18,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb78ac8c06c2799bf58ab9 HTTP/1.1" 200 None 2026-04-02 07:33:18,292 - lava_callback - DEBUG - Processing node 69cb78ac8c06c2799bf58ab9: baseline-arm64-baylibre 2026-04-02 07:33:18,292 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb78ac8c06c2799bf58ab9 2026-04-02 07:33:18,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:33:18,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb78ac8c06c2799bf58ab9 HTTP/1.1" 200 None 2026-04-02 07:33:18,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:33:18,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 07:33:18,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:33:18,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb78ac8c06c2799bf58ab9 HTTP/1.1" 200 None 2026-04-02 07:33:18,360 - lava_callback - ERROR - Error processing callback for node 69cb78ac8c06c2799bf58ab9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38246 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:34:18,288 - lava_callback - INFO - Received callback for node 69cb78f58c06c2799bf58ac8 INFO: 172.20.0.1:38262 - "POST /node/69cb78f58c06c2799bf58ac8 HTTP/1.0" 202 Accepted 2026-04-02 07:34:18,293 - lava_callback - INFO - Starting async job submit for node 69cb78f58c06c2799bf58ac8 2026-04-02 07:34:18,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:34:18,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb78f58c06c2799bf58ac8 HTTP/1.1" 200 None 2026-04-02 07:34:18,309 - lava_callback - DEBUG - Processing node 69cb78f58c06c2799bf58ac8: baseline-arm64-baylibre 2026-04-02 07:34:18,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb78f58c06c2799bf58ac8 2026-04-02 07:34:18,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:34:18,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb78f58c06c2799bf58ac8 HTTP/1.1" 200 None 2026-04-02 07:34:18,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:34:18,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:34:18,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:34:18,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb78f58c06c2799bf58ac8 HTTP/1.1" 200 None 2026-04-02 07:34:18,365 - lava_callback - ERROR - Error processing callback for node 69cb78f58c06c2799bf58ac8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39778 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:35:47,825 - lava_callback - INFO - Received callback for node 69cb794b8c06c2799bf58ad7 INFO: 172.20.0.1:39786 - "POST /node/69cb794b8c06c2799bf58ad7 HTTP/1.0" 202 Accepted 2026-04-02 07:35:47,830 - lava_callback - INFO - Starting async job submit for node 69cb794b8c06c2799bf58ad7 2026-04-02 07:35:47,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:35:47,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb794b8c06c2799bf58ad7 HTTP/1.1" 200 None 2026-04-02 07:35:47,846 - lava_callback - DEBUG - Processing node 69cb794b8c06c2799bf58ad7: baseline-arm64-clabbe 2026-04-02 07:35:47,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb794b8c06c2799bf58ad7 2026-04-02 07:35:47,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:35:47,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb794b8c06c2799bf58ad7 HTTP/1.1" 200 None 2026-04-02 07:35:47,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:35:47,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 07:35:47,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:35:47,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb794b8c06c2799bf58ad7 HTTP/1.1" 200 None 2026-04-02 07:35:47,898 - lava_callback - ERROR - Error processing callback for node 69cb794b8c06c2799bf58ad7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42520 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:37:07,791 - lava_callback - INFO - Received callback for node 69cb79998c06c2799bf58add INFO: 172.20.0.1:42534 - "POST /node/69cb79998c06c2799bf58add HTTP/1.0" 202 Accepted 2026-04-02 07:37:07,797 - lava_callback - INFO - Starting async job submit for node 69cb79998c06c2799bf58add 2026-04-02 07:37:07,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:37:07,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb79998c06c2799bf58add HTTP/1.1" 200 None 2026-04-02 07:37:07,814 - lava_callback - DEBUG - Processing node 69cb79998c06c2799bf58add: baseline-arm64-clabbe 2026-04-02 07:37:07,814 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb79998c06c2799bf58add 2026-04-02 07:37:07,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:37:07,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb79998c06c2799bf58add HTTP/1.1" 200 None 2026-04-02 07:37:07,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:37:07,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 07:37:07,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:37:07,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb79998c06c2799bf58add HTTP/1.1" 200 None 2026-04-02 07:37:07,871 - lava_callback - ERROR - Error processing callback for node 69cb79998c06c2799bf58add: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38286 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:38:27,893 - lava_callback - INFO - Received callback for node 69cb79e68c06c2799bf58aeb INFO: 172.20.0.1:38302 - "POST /node/69cb79e68c06c2799bf58aeb HTTP/1.0" 202 Accepted 2026-04-02 07:38:27,899 - lava_callback - INFO - Starting async job submit for node 69cb79e68c06c2799bf58aeb 2026-04-02 07:38:27,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:38:27,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb79e68c06c2799bf58aeb HTTP/1.1" 200 None 2026-04-02 07:38:27,914 - lava_callback - DEBUG - Processing node 69cb79e68c06c2799bf58aeb: baseline-arm64-clabbe 2026-04-02 07:38:27,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb79e68c06c2799bf58aeb 2026-04-02 07:38:27,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:38:27,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb79e68c06c2799bf58aeb HTTP/1.1" 200 None 2026-04-02 07:38:27,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:38:27,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 07:38:27,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:38:27,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb79e68c06c2799bf58aeb HTTP/1.1" 200 None 2026-04-02 07:38:27,967 - lava_callback - ERROR - Error processing callback for node 69cb79e68c06c2799bf58aeb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53958 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:39:47,941 - lava_callback - INFO - Received callback for node 69cb7a348c06c2799bf58af2 INFO: 172.20.0.1:53970 - "POST /node/69cb7a348c06c2799bf58af2 HTTP/1.0" 202 Accepted 2026-04-02 07:39:47,946 - lava_callback - INFO - Starting async job submit for node 69cb7a348c06c2799bf58af2 2026-04-02 07:39:47,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:39:47,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7a348c06c2799bf58af2 HTTP/1.1" 200 None 2026-04-02 07:39:47,962 - lava_callback - DEBUG - Processing node 69cb7a348c06c2799bf58af2: baseline-arm64-clabbe 2026-04-02 07:39:47,962 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7a348c06c2799bf58af2 2026-04-02 07:39:47,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:39:47,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7a348c06c2799bf58af2 HTTP/1.1" 200 None 2026-04-02 07:39:47,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:39:47,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 07:39:47,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:39:48,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7a348c06c2799bf58af2 HTTP/1.1" 200 None 2026-04-02 07:39:48,013 - lava_callback - ERROR - Error processing callback for node 69cb7a348c06c2799bf58af2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59528 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:41:07,945 - lava_callback - INFO - Received callback for node 69cb7a828c06c2799bf58afe INFO: 172.20.0.1:59544 - "POST /node/69cb7a828c06c2799bf58afe HTTP/1.0" 202 Accepted 2026-04-02 07:41:07,951 - lava_callback - INFO - Starting async job submit for node 69cb7a828c06c2799bf58afe 2026-04-02 07:41:07,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:41:07,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7a828c06c2799bf58afe HTTP/1.1" 200 None 2026-04-02 07:41:07,967 - lava_callback - DEBUG - Processing node 69cb7a828c06c2799bf58afe: baseline-arm64-clabbe 2026-04-02 07:41:07,967 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7a828c06c2799bf58afe 2026-04-02 07:41:07,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:41:07,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7a828c06c2799bf58afe HTTP/1.1" 200 None 2026-04-02 07:41:07,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:41:07,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 07:41:07,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:41:08,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7a828c06c2799bf58afe HTTP/1.1" 200 None 2026-04-02 07:41:08,032 - lava_callback - ERROR - Error processing callback for node 69cb7a828c06c2799bf58afe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52504 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:42:27,990 - lava_callback - INFO - Received callback for node 69cb7acf8c06c2799bf58b0e INFO: 172.20.0.1:52510 - "POST /node/69cb7acf8c06c2799bf58b0e HTTP/1.0" 202 Accepted 2026-04-02 07:42:27,995 - lava_callback - INFO - Starting async job submit for node 69cb7acf8c06c2799bf58b0e 2026-04-02 07:42:27,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:42:28,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7acf8c06c2799bf58b0e HTTP/1.1" 200 None 2026-04-02 07:42:28,013 - lava_callback - DEBUG - Processing node 69cb7acf8c06c2799bf58b0e: baseline-arm64-clabbe 2026-04-02 07:42:28,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7acf8c06c2799bf58b0e 2026-04-02 07:42:28,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:42:28,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7acf8c06c2799bf58b0e HTTP/1.1" 200 None 2026-04-02 07:42:28,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:42:28,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-02 07:42:28,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:42:28,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7acf8c06c2799bf58b0e HTTP/1.1" 200 None 2026-04-02 07:42:28,067 - lava_callback - ERROR - Error processing callback for node 69cb7acf8c06c2799bf58b0e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48148 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:43:38,903 - lava_callback - INFO - Received callback for node 69cb7b158c06c2799bf58b16 INFO: 172.20.0.1:48158 - "POST /node/69cb7b158c06c2799bf58b16 HTTP/1.0" 202 Accepted 2026-04-02 07:43:38,909 - lava_callback - INFO - Starting async job submit for node 69cb7b158c06c2799bf58b16 2026-04-02 07:43:38,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:43:38,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7b158c06c2799bf58b16 HTTP/1.1" 200 None 2026-04-02 07:43:38,950 - lava_callback - DEBUG - Processing node 69cb7b158c06c2799bf58b16: baseline-arm64-baylibre 2026-04-02 07:43:38,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7b158c06c2799bf58b16 2026-04-02 07:43:38,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:43:38,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7b158c06c2799bf58b16 HTTP/1.1" 200 None 2026-04-02 07:43:38,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:43:38,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:43:38,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:43:39,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7b158c06c2799bf58b16 HTTP/1.1" 200 None 2026-04-02 07:43:39,018 - lava_callback - ERROR - Error processing callback for node 69cb7b158c06c2799bf58b16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35940 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:44:48,223 - lava_callback - INFO - Received callback for node 69cb7b698c06c2799bf58b22 INFO: 172.20.0.1:35952 - "POST /node/69cb7b698c06c2799bf58b22 HTTP/1.0" 202 Accepted 2026-04-02 07:44:48,229 - lava_callback - INFO - Starting async job submit for node 69cb7b698c06c2799bf58b22 2026-04-02 07:44:48,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:44:48,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7b698c06c2799bf58b22 HTTP/1.1" 200 None 2026-04-02 07:44:48,245 - lava_callback - DEBUG - Processing node 69cb7b698c06c2799bf58b22: baseline-arm64-clabbe 2026-04-02 07:44:48,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7b698c06c2799bf58b22 2026-04-02 07:44:48,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:44:48,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7b698c06c2799bf58b22 HTTP/1.1" 200 None 2026-04-02 07:44:48,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:44:48,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:44:48,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:44:48,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7b698c06c2799bf58b22 HTTP/1.1" 200 None 2026-04-02 07:44:48,298 - lava_callback - ERROR - Error processing callback for node 69cb7b698c06c2799bf58b22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37316 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:45:58,988 - lava_callback - INFO - Received callback for node 69cb7bb28c06c2799bf58b2a INFO: 172.20.0.1:37318 - "POST /node/69cb7bb28c06c2799bf58b2a HTTP/1.0" 202 Accepted 2026-04-02 07:45:59,000 - lava_callback - INFO - Starting async job submit for node 69cb7bb28c06c2799bf58b2a 2026-04-02 07:45:59,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:45:59,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7bb28c06c2799bf58b2a HTTP/1.1" 200 None 2026-04-02 07:45:59,048 - lava_callback - DEBUG - Processing node 69cb7bb28c06c2799bf58b2a: baseline-arm64-baylibre 2026-04-02 07:45:59,049 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7bb28c06c2799bf58b2a 2026-04-02 07:45:59,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:45:59,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7bb28c06c2799bf58b2a HTTP/1.1" 200 None 2026-04-02 07:45:59,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:45:59,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:45:59,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:45:59,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7bb28c06c2799bf58b2a HTTP/1.1" 200 None 2026-04-02 07:45:59,119 - lava_callback - ERROR - Error processing callback for node 69cb7bb28c06c2799bf58b2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39364 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:47:28,215 - lava_callback - INFO - Received callback for node 69cb7c068c06c2799bf58b37 INFO: 172.20.0.1:39366 - "POST /node/69cb7c068c06c2799bf58b37 HTTP/1.0" 202 Accepted 2026-04-02 07:47:28,221 - lava_callback - INFO - Starting async job submit for node 69cb7c068c06c2799bf58b37 2026-04-02 07:47:28,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:47:28,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7c068c06c2799bf58b37 HTTP/1.1" 200 None 2026-04-02 07:47:28,236 - lava_callback - DEBUG - Processing node 69cb7c068c06c2799bf58b37: baseline-arm64-clabbe 2026-04-02 07:47:28,237 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7c068c06c2799bf58b37 2026-04-02 07:47:28,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:47:28,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7c068c06c2799bf58b37 HTTP/1.1" 200 None 2026-04-02 07:47:28,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:47:28,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:47:28,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:47:28,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7c068c06c2799bf58b37 HTTP/1.1" 200 None 2026-04-02 07:47:28,292 - lava_callback - ERROR - Error processing callback for node 69cb7c068c06c2799bf58b37: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59126 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:48:48,264 - lava_callback - INFO - Received callback for node 69cb7c548c06c2799bf58b45 INFO: 172.20.0.1:59130 - "POST /node/69cb7c548c06c2799bf58b45 HTTP/1.0" 202 Accepted 2026-04-02 07:48:48,270 - lava_callback - INFO - Starting async job submit for node 69cb7c548c06c2799bf58b45 2026-04-02 07:48:48,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:48:48,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7c548c06c2799bf58b45 HTTP/1.1" 200 None 2026-04-02 07:48:48,294 - lava_callback - DEBUG - Processing node 69cb7c548c06c2799bf58b45: baseline-arm64-clabbe 2026-04-02 07:48:48,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7c548c06c2799bf58b45 2026-04-02 07:48:48,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:48:48,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7c548c06c2799bf58b45 HTTP/1.1" 200 None 2026-04-02 07:48:48,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:48:48,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:48:48,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:48:48,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7c548c06c2799bf58b45 HTTP/1.1" 200 None 2026-04-02 07:48:48,351 - lava_callback - ERROR - Error processing callback for node 69cb7c548c06c2799bf58b45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55762 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:50:08,308 - lava_callback - INFO - Received callback for node 69cb7ca28c06c2799bf58b4f INFO: 172.20.0.1:55778 - "POST /node/69cb7ca28c06c2799bf58b4f HTTP/1.0" 202 Accepted 2026-04-02 07:50:08,314 - lava_callback - INFO - Starting async job submit for node 69cb7ca28c06c2799bf58b4f 2026-04-02 07:50:08,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:50:08,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7ca28c06c2799bf58b4f HTTP/1.1" 200 None 2026-04-02 07:50:08,330 - lava_callback - DEBUG - Processing node 69cb7ca28c06c2799bf58b4f: baseline-arm64-clabbe 2026-04-02 07:50:08,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7ca28c06c2799bf58b4f 2026-04-02 07:50:08,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:50:08,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7ca28c06c2799bf58b4f HTTP/1.1" 200 None 2026-04-02 07:50:08,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:50:08,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:50:08,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:50:08,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7ca28c06c2799bf58b4f HTTP/1.1" 200 None 2026-04-02 07:50:08,388 - lava_callback - ERROR - Error processing callback for node 69cb7ca28c06c2799bf58b4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:51:19,221 - lava_callback - INFO - Received callback for node 69cb7ceb8c06c2799bf58b5e INFO: 172.20.0.1:40536 - "POST /node/69cb7ceb8c06c2799bf58b5e HTTP/1.0" 202 Accepted 2026-04-02 07:51:19,227 - lava_callback - INFO - Starting async job submit for node 69cb7ceb8c06c2799bf58b5e 2026-04-02 07:51:19,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:51:19,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7ceb8c06c2799bf58b5e HTTP/1.1" 200 None 2026-04-02 07:51:19,243 - lava_callback - DEBUG - Processing node 69cb7ceb8c06c2799bf58b5e: baseline-arm64-baylibre 2026-04-02 07:51:19,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7ceb8c06c2799bf58b5e 2026-04-02 07:51:19,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:51:19,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7ceb8c06c2799bf58b5e HTTP/1.1" 200 None 2026-04-02 07:51:19,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:51:19,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:51:19,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:51:19,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7ceb8c06c2799bf58b5e HTTP/1.1" 200 None 2026-04-02 07:51:19,300 - lava_callback - ERROR - Error processing callback for node 69cb7ceb8c06c2799bf58b5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49692 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:52:48,475 - lava_callback - INFO - Received callback for node 69cb7d3f8c06c2799bf58b66 INFO: 172.20.0.1:49708 - "POST /node/69cb7d3f8c06c2799bf58b66 HTTP/1.0" 202 Accepted 2026-04-02 07:52:48,481 - lava_callback - INFO - Starting async job submit for node 69cb7d3f8c06c2799bf58b66 2026-04-02 07:52:48,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:52:48,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7d3f8c06c2799bf58b66 HTTP/1.1" 200 None 2026-04-02 07:52:48,646 - lava_callback - DEBUG - Processing node 69cb7d3f8c06c2799bf58b66: baseline-arm64-clabbe 2026-04-02 07:52:48,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7d3f8c06c2799bf58b66 2026-04-02 07:52:48,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:52:48,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7d3f8c06c2799bf58b66 HTTP/1.1" 200 None 2026-04-02 07:52:48,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:52:48,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:52:48,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:52:48,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7d3f8c06c2799bf58b66 HTTP/1.1" 200 None 2026-04-02 07:52:48,708 - lava_callback - ERROR - Error processing callback for node 69cb7d3f8c06c2799bf58b66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38952 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:54:08,539 - lava_callback - INFO - Received callback for node 69cb7d8c8c06c2799bf58b73 INFO: 172.20.0.1:38962 - "POST /node/69cb7d8c8c06c2799bf58b73 HTTP/1.0" 202 Accepted 2026-04-02 07:54:08,545 - lava_callback - INFO - Starting async job submit for node 69cb7d8c8c06c2799bf58b73 2026-04-02 07:54:08,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:54:08,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7d8c8c06c2799bf58b73 HTTP/1.1" 200 None 2026-04-02 07:54:08,564 - lava_callback - DEBUG - Processing node 69cb7d8c8c06c2799bf58b73: baseline-arm64-clabbe 2026-04-02 07:54:08,564 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7d8c8c06c2799bf58b73 2026-04-02 07:54:08,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:54:08,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7d8c8c06c2799bf58b73 HTTP/1.1" 200 None 2026-04-02 07:54:08,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:54:08,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:54:08,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:54:08,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7d8c8c06c2799bf58b73 HTTP/1.1" 200 None 2026-04-02 07:54:08,622 - lava_callback - ERROR - Error processing callback for node 69cb7d8c8c06c2799bf58b73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53218 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:55:28,532 - lava_callback - INFO - Received callback for node 69cb7dda8c06c2799bf58b81 INFO: 172.20.0.1:53230 - "POST /node/69cb7dda8c06c2799bf58b81 HTTP/1.0" 202 Accepted 2026-04-02 07:55:28,537 - lava_callback - INFO - Starting async job submit for node 69cb7dda8c06c2799bf58b81 2026-04-02 07:55:28,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:55:28,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7dda8c06c2799bf58b81 HTTP/1.1" 200 None 2026-04-02 07:55:28,552 - lava_callback - DEBUG - Processing node 69cb7dda8c06c2799bf58b81: baseline-arm64-clabbe 2026-04-02 07:55:28,552 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7dda8c06c2799bf58b81 2026-04-02 07:55:28,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:55:28,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7dda8c06c2799bf58b81 HTTP/1.1" 200 None 2026-04-02 07:55:28,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:55:28,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:55:28,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:55:28,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7dda8c06c2799bf58b81 HTTP/1.1" 200 None 2026-04-02 07:55:28,615 - lava_callback - ERROR - Error processing callback for node 69cb7dda8c06c2799bf58b81: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58136 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:56:28,639 - lava_callback - INFO - Received callback for node 69cb7e298c06c2799bf58b8c INFO: 172.20.0.1:58150 - "POST /node/69cb7e298c06c2799bf58b8c HTTP/1.0" 202 Accepted 2026-04-02 07:56:28,645 - lava_callback - INFO - Starting async job submit for node 69cb7e298c06c2799bf58b8c 2026-04-02 07:56:28,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:56:28,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7e298c06c2799bf58b8c HTTP/1.1" 200 None 2026-04-02 07:56:28,662 - lava_callback - DEBUG - Processing node 69cb7e298c06c2799bf58b8c: baseline-arm64-clabbe 2026-04-02 07:56:28,662 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7e298c06c2799bf58b8c 2026-04-02 07:56:28,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:56:28,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7e298c06c2799bf58b8c HTTP/1.1" 200 None 2026-04-02 07:56:28,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:56:28,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:56:28,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:56:28,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7e298c06c2799bf58b8c HTTP/1.1" 200 None 2026-04-02 07:56:28,719 - lava_callback - ERROR - Error processing callback for node 69cb7e298c06c2799bf58b8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:57:48,709 - lava_callback - INFO - Received callback for node 69cb7e778c06c2799bf58b9c INFO: 172.20.0.1:54862 - "POST /node/69cb7e778c06c2799bf58b9c HTTP/1.0" 202 Accepted 2026-04-02 07:57:48,715 - lava_callback - INFO - Starting async job submit for node 69cb7e778c06c2799bf58b9c 2026-04-02 07:57:48,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:57:48,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7e778c06c2799bf58b9c HTTP/1.1" 200 None 2026-04-02 07:57:48,731 - lava_callback - DEBUG - Processing node 69cb7e778c06c2799bf58b9c: baseline-arm64-clabbe 2026-04-02 07:57:48,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7e778c06c2799bf58b9c 2026-04-02 07:57:48,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:57:48,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7e778c06c2799bf58b9c HTTP/1.1" 200 None 2026-04-02 07:57:48,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:57:48,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:57:48,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:57:48,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7e778c06c2799bf58b9c HTTP/1.1" 200 None 2026-04-02 07:57:48,794 - lava_callback - ERROR - Error processing callback for node 69cb7e778c06c2799bf58b9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54754 - "GET / HTTP/1.0" 200 OK 2026-04-02 07:59:08,752 - lava_callback - INFO - Received callback for node 69cb7ec78c06c2799bf58ba5 INFO: 172.20.0.1:54760 - "POST /node/69cb7ec78c06c2799bf58ba5 HTTP/1.0" 202 Accepted 2026-04-02 07:59:08,757 - lava_callback - INFO - Starting async job submit for node 69cb7ec78c06c2799bf58ba5 2026-04-02 07:59:08,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:59:08,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7ec78c06c2799bf58ba5 HTTP/1.1" 200 None 2026-04-02 07:59:08,772 - lava_callback - DEBUG - Processing node 69cb7ec78c06c2799bf58ba5: baseline-arm64-clabbe 2026-04-02 07:59:08,773 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7ec78c06c2799bf58ba5 2026-04-02 07:59:08,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:59:08,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7ec78c06c2799bf58ba5 HTTP/1.1" 200 None 2026-04-02 07:59:08,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:59:08,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 07:59:08,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 07:59:08,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7ec78c06c2799bf58ba5 HTTP/1.1" 200 None 2026-04-02 07:59:08,832 - lava_callback - ERROR - Error processing callback for node 69cb7ec78c06c2799bf58ba5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42720 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:00:28,846 - lava_callback - INFO - Received callback for node 69cb7f178c06c2799bf58bb5 INFO: 172.20.0.1:42730 - "POST /node/69cb7f178c06c2799bf58bb5 HTTP/1.0" 202 Accepted 2026-04-02 08:00:28,852 - lava_callback - INFO - Starting async job submit for node 69cb7f178c06c2799bf58bb5 2026-04-02 08:00:28,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:00:28,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7f178c06c2799bf58bb5 HTTP/1.1" 200 None 2026-04-02 08:00:28,870 - lava_callback - DEBUG - Processing node 69cb7f178c06c2799bf58bb5: baseline-arm64-clabbe 2026-04-02 08:00:28,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7f178c06c2799bf58bb5 2026-04-02 08:00:28,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:00:28,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7f178c06c2799bf58bb5 HTTP/1.1" 200 None 2026-04-02 08:00:28,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:00:28,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 08:00:28,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:00:28,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7f178c06c2799bf58bb5 HTTP/1.1" 200 None 2026-04-02 08:00:28,939 - lava_callback - ERROR - Error processing callback for node 69cb7f178c06c2799bf58bb5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44662 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:01:48,859 - lava_callback - INFO - Received callback for node 69cb7f668c06c2799bf58bc2 INFO: 172.20.0.1:44668 - "POST /node/69cb7f668c06c2799bf58bc2 HTTP/1.0" 202 Accepted 2026-04-02 08:01:48,865 - lava_callback - INFO - Starting async job submit for node 69cb7f668c06c2799bf58bc2 2026-04-02 08:01:48,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:01:48,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7f668c06c2799bf58bc2 HTTP/1.1" 200 None 2026-04-02 08:01:48,883 - lava_callback - DEBUG - Processing node 69cb7f668c06c2799bf58bc2: baseline-arm64-clabbe 2026-04-02 08:01:48,884 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7f668c06c2799bf58bc2 2026-04-02 08:01:48,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:01:48,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7f668c06c2799bf58bc2 HTTP/1.1" 200 None 2026-04-02 08:01:48,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:01:48,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 08:01:48,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:01:48,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7f668c06c2799bf58bc2 HTTP/1.1" 200 None 2026-04-02 08:01:48,934 - lava_callback - ERROR - Error processing callback for node 69cb7f668c06c2799bf58bc2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50274 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:03:08,966 - lava_callback - INFO - Received callback for node 69cb7fb68c06c2799bf58bc9 INFO: 172.20.0.1:50286 - "POST /node/69cb7fb68c06c2799bf58bc9 HTTP/1.0" 202 Accepted 2026-04-02 08:03:08,971 - lava_callback - INFO - Starting async job submit for node 69cb7fb68c06c2799bf58bc9 2026-04-02 08:03:08,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:03:08,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7fb68c06c2799bf58bc9 HTTP/1.1" 200 None 2026-04-02 08:03:08,988 - lava_callback - DEBUG - Processing node 69cb7fb68c06c2799bf58bc9: baseline-arm64-clabbe 2026-04-02 08:03:08,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb7fb68c06c2799bf58bc9 2026-04-02 08:03:08,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:03:09,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb7fb68c06c2799bf58bc9 HTTP/1.1" 200 None 2026-04-02 08:03:09,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:03:09,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 08:03:09,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:03:09,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb7fb68c06c2799bf58bc9 HTTP/1.1" 200 None 2026-04-02 08:03:09,053 - lava_callback - ERROR - Error processing callback for node 69cb7fb68c06c2799bf58bc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58280 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:04:28,979 - lava_callback - INFO - Received callback for node 69cb80048c06c2799bf58bd7 INFO: 172.20.0.1:58294 - "POST /node/69cb80048c06c2799bf58bd7 HTTP/1.0" 202 Accepted 2026-04-02 08:04:28,985 - lava_callback - INFO - Starting async job submit for node 69cb80048c06c2799bf58bd7 2026-04-02 08:04:28,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:04:29,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb80048c06c2799bf58bd7 HTTP/1.1" 200 None 2026-04-02 08:04:29,003 - lava_callback - DEBUG - Processing node 69cb80048c06c2799bf58bd7: baseline-arm64-clabbe 2026-04-02 08:04:29,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb80048c06c2799bf58bd7 2026-04-02 08:04:29,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:04:29,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb80048c06c2799bf58bd7 HTTP/1.1" 200 None 2026-04-02 08:04:29,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:04:29,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 08:04:29,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:04:29,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb80048c06c2799bf58bd7 HTTP/1.1" 200 None 2026-04-02 08:04:29,127 - lava_callback - ERROR - Error processing callback for node 69cb80048c06c2799bf58bd7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41554 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:05:49,063 - lava_callback - INFO - Received callback for node 69cb80548c06c2799bf58be2 INFO: 172.20.0.1:41560 - "POST /node/69cb80548c06c2799bf58be2 HTTP/1.0" 202 Accepted 2026-04-02 08:05:49,068 - lava_callback - INFO - Starting async job submit for node 69cb80548c06c2799bf58be2 2026-04-02 08:05:49,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:05:49,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb80548c06c2799bf58be2 HTTP/1.1" 200 None 2026-04-02 08:05:49,084 - lava_callback - DEBUG - Processing node 69cb80548c06c2799bf58be2: baseline-arm64-clabbe 2026-04-02 08:05:49,084 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb80548c06c2799bf58be2 2026-04-02 08:05:49,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:05:49,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb80548c06c2799bf58be2 HTTP/1.1" 200 None 2026-04-02 08:05:49,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:05:49,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 08:05:49,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:05:49,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb80548c06c2799bf58be2 HTTP/1.1" 200 None 2026-04-02 08:05:49,139 - lava_callback - ERROR - Error processing callback for node 69cb80548c06c2799bf58be2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60312 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:07:09,086 - lava_callback - INFO - Received callback for node 69cb80a48c06c2799bf58beb INFO: 172.20.0.1:60314 - "POST /node/69cb80a48c06c2799bf58beb HTTP/1.0" 202 Accepted 2026-04-02 08:07:09,092 - lava_callback - INFO - Starting async job submit for node 69cb80a48c06c2799bf58beb 2026-04-02 08:07:09,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:07:09,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb80a48c06c2799bf58beb HTTP/1.1" 200 None 2026-04-02 08:07:09,109 - lava_callback - DEBUG - Processing node 69cb80a48c06c2799bf58beb: baseline-arm64-clabbe 2026-04-02 08:07:09,109 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb80a48c06c2799bf58beb 2026-04-02 08:07:09,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:07:09,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb80a48c06c2799bf58beb HTTP/1.1" 200 None 2026-04-02 08:07:09,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:07:09,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 08:07:09,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:07:09,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb80a48c06c2799bf58beb HTTP/1.1" 200 None 2026-04-02 08:07:09,160 - lava_callback - ERROR - Error processing callback for node 69cb80a48c06c2799bf58beb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57312 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:08:29,178 - lava_callback - INFO - Received callback for node 69cb80f18c06c2799bf58bf7 INFO: 172.20.0.1:57318 - "POST /node/69cb80f18c06c2799bf58bf7 HTTP/1.0" 202 Accepted 2026-04-02 08:08:29,183 - lava_callback - INFO - Starting async job submit for node 69cb80f18c06c2799bf58bf7 2026-04-02 08:08:29,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:08:29,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb80f18c06c2799bf58bf7 HTTP/1.1" 200 None 2026-04-02 08:08:29,199 - lava_callback - DEBUG - Processing node 69cb80f18c06c2799bf58bf7: baseline-arm64-clabbe 2026-04-02 08:08:29,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb80f18c06c2799bf58bf7 2026-04-02 08:08:29,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:08:29,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb80f18c06c2799bf58bf7 HTTP/1.1" 200 None 2026-04-02 08:08:29,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:08:29,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 08:08:29,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:08:29,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb80f18c06c2799bf58bf7 HTTP/1.1" 200 None 2026-04-02 08:08:29,259 - lava_callback - ERROR - Error processing callback for node 69cb80f18c06c2799bf58bf7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45494 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:09:49,250 - lava_callback - INFO - Received callback for node 69cb813d8c06c2799bf58bfd INFO: 172.20.0.1:45496 - "POST /node/69cb813d8c06c2799bf58bfd HTTP/1.0" 202 Accepted 2026-04-02 08:09:49,255 - lava_callback - INFO - Starting async job submit for node 69cb813d8c06c2799bf58bfd 2026-04-02 08:09:49,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:09:49,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb813d8c06c2799bf58bfd HTTP/1.1" 200 None 2026-04-02 08:09:49,271 - lava_callback - DEBUG - Processing node 69cb813d8c06c2799bf58bfd: baseline-arm64-clabbe 2026-04-02 08:09:49,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb813d8c06c2799bf58bfd 2026-04-02 08:09:49,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:09:49,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb813d8c06c2799bf58bfd HTTP/1.1" 200 None 2026-04-02 08:09:49,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:09:49,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-02 08:09:49,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:09:49,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb813d8c06c2799bf58bfd HTTP/1.1" 200 None 2026-04-02 08:09:49,332 - lava_callback - ERROR - Error processing callback for node 69cb813d8c06c2799bf58bfd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51664 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:11:00,286 - lava_callback - INFO - Received callback for node 69cb81878c06c2799bf58c0b INFO: 172.20.0.1:60752 - "POST /node/69cb81878c06c2799bf58c0b HTTP/1.0" 202 Accepted 2026-04-02 08:11:00,293 - lava_callback - INFO - Starting async job submit for node 69cb81878c06c2799bf58c0b 2026-04-02 08:11:00,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:11:00,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb81878c06c2799bf58c0b HTTP/1.1" 200 None 2026-04-02 08:11:00,309 - lava_callback - DEBUG - Processing node 69cb81878c06c2799bf58c0b: baseline-arm64-baylibre 2026-04-02 08:11:00,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb81878c06c2799bf58c0b 2026-04-02 08:11:00,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:11:00,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb81878c06c2799bf58c0b HTTP/1.1" 200 None 2026-04-02 08:11:00,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:11:00,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-02 08:11:00,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:11:00,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb81878c06c2799bf58c0b HTTP/1.1" 200 None 2026-04-02 08:11:00,383 - lava_callback - ERROR - Error processing callback for node 69cb81878c06c2799bf58c0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:12:29,355 - lava_callback - INFO - Received callback for node 69cb81da8c06c2799bf58c19 INFO: 172.20.0.1:54710 - "POST /node/69cb81da8c06c2799bf58c19 HTTP/1.0" 202 Accepted 2026-04-02 08:12:29,364 - lava_callback - INFO - Starting async job submit for node 69cb81da8c06c2799bf58c19 2026-04-02 08:12:29,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:12:29,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb81da8c06c2799bf58c19 HTTP/1.1" 200 None 2026-04-02 08:12:29,384 - lava_callback - DEBUG - Processing node 69cb81da8c06c2799bf58c19: baseline-arm64-clabbe 2026-04-02 08:12:29,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb81da8c06c2799bf58c19 2026-04-02 08:12:29,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:12:29,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb81da8c06c2799bf58c19 HTTP/1.1" 200 None 2026-04-02 08:12:29,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:12:29,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-02 08:12:29,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:12:29,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb81da8c06c2799bf58c19 HTTP/1.1" 200 None 2026-04-02 08:12:29,441 - lava_callback - ERROR - Error processing callback for node 69cb81da8c06c2799bf58c19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60640 - "GET / HTTP/1.0" 200 OK 2026-04-02 08:13:49,498 - lava_callback - INFO - Received callback for node 69cb822a8c06c2799bf58c1e INFO: 172.20.0.1:60646 - "POST /node/69cb822a8c06c2799bf58c1e HTTP/1.0" 202 Accepted 2026-04-02 08:13:49,507 - lava_callback - INFO - Starting async job submit for node 69cb822a8c06c2799bf58c1e 2026-04-02 08:13:49,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:13:49,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb822a8c06c2799bf58c1e HTTP/1.1" 200 None 2026-04-02 08:13:49,533 - lava_callback - DEBUG - Processing node 69cb822a8c06c2799bf58c1e: baseline-arm64-clabbe 2026-04-02 08:13:49,533 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cb822a8c06c2799bf58c1e 2026-04-02 08:13:49,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:13:49,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb822a8c06c2799bf58c1e HTTP/1.1" 200 None 2026-04-02 08:13:49,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:13:49,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-02 08:13:49,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-02 08:13:49,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cb822a8c06c2799bf58c1e HTTP/1.1" 200 None 2026-04-02 08:13:49,626 - lava_callback - ERROR - Error processing callback for node 69cb822a8c06c2799bf58c1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]