INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:52474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57502 - "GET /node/699448525ae459b9df3dad23%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:47284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42322 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:39:43,781 - lava_callback - INFO - Received callback for node 69ac54c346f7ae8ff33651d9 2026-03-09 16:39:43,787 - lava_callback - INFO - Starting async job submit for node 69ac54c346f7ae8ff33651d9 INFO: 172.21.0.1:36950 - "POST /node/69ac54c346f7ae8ff33651d9 HTTP/1.0" 202 Accepted 2026-03-09 16:39:43,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:39:43,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac54c346f7ae8ff33651d9 HTTP/1.1" 200 None 2026-03-09 16:39:43,810 - lava_callback - DEBUG - Processing node 69ac54c346f7ae8ff33651d9: baseline-arm-baylibre 2026-03-09 16:39:43,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac54c346f7ae8ff33651d9 2026-03-09 16:39:43,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:39:43,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac54c346f7ae8ff33651d9 HTTP/1.1" 200 None 2026-03-09 16:39:43,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:39:43,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a4fe31b554803095c08 HTTP/1.1" 200 None 2026-03-09 16:39:43,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:39:43,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac54c346f7ae8ff33651d9 HTTP/1.1" 200 None 2026-03-09 16:39:43,864 - lava_callback - ERROR - Error processing callback for node 69ac54c346f7ae8ff33651d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42092 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:40:03,292 - lava_callback - INFO - Received callback for node 69aef766ce46d8ec186abbfa INFO: 172.21.0.1:52294 - "POST /node/69aef766ce46d8ec186abbfa HTTP/1.0" 202 Accepted 2026-03-09 16:40:03,299 - lava_callback - INFO - Starting async job submit for node 69aef766ce46d8ec186abbfa 2026-03-09 16:40:03,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:03,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef766ce46d8ec186abbfa HTTP/1.1" 200 None 2026-03-09 16:40:03,353 - lava_callback - DEBUG - Processing node 69aef766ce46d8ec186abbfa: baseline-arm 2026-03-09 16:40:03,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:40:03,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:40:03,700 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aef766ce46d8ec186abbfa/log.txt.gz 2026-03-09 16:40:03,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:40:04,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:40:04,017 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef766ce46d8ec186abbfa/lava-logs.yaml 2026-03-09 16:40:04,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:40:04,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:40:04,200 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef766ce46d8ec186abbfa/lava_callback.json.gz 2026-03-09 16:40:04,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:04,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef766ce46d8ec186abbfa HTTP/1.1" 200 None 2026-03-09 16:40:04,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:04,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef42ace46d8ec186ab814 HTTP/1.1" 200 None 2026-03-09 16:40:04,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:04,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef766ce46d8ec186abbfa HTTP/1.1" 200 None 2026-03-09 16:40:04,292 - lava_callback - INFO - Completed processing callback for node 69aef766ce46d8ec186abbfa INFO: 172.21.0.1:52306 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:40:13,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:13,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42054 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:40:23,752 - lava_callback - INFO - Received callback for node 69ac54e446f7ae8ff33651e7 INFO: 172.21.0.1:33144 - "POST /node/69ac54e446f7ae8ff33651e7 HTTP/1.0" 202 Accepted 2026-03-09 16:40:23,757 - lava_callback - INFO - Starting async job submit for node 69ac54e446f7ae8ff33651e7 2026-03-09 16:40:23,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:23,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac54e446f7ae8ff33651e7 HTTP/1.1" 200 None 2026-03-09 16:40:23,774 - lava_callback - DEBUG - Processing node 69ac54e446f7ae8ff33651e7: baseline-arm-baylibre 2026-03-09 16:40:23,774 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac54e446f7ae8ff33651e7 2026-03-09 16:40:23,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:23,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac54e446f7ae8ff33651e7 HTTP/1.1" 200 None 2026-03-09 16:40:23,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:23,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a4fe31b554803095c08 HTTP/1.1" 200 None 2026-03-09 16:40:23,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:23,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac54e446f7ae8ff33651e7 HTTP/1.1" 200 None 2026-03-09 16:40:23,845 - lava_callback - ERROR - Error processing callback for node 69ac54e446f7ae8ff33651e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54248 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:40:43,777 - lava_callback - INFO - Received callback for node 69ac550846f7ae8ff33651f6 INFO: 172.21.0.1:41056 - "POST /node/69ac550846f7ae8ff33651f6 HTTP/1.0" 202 Accepted 2026-03-09 16:40:43,783 - lava_callback - INFO - Starting async job submit for node 69ac550846f7ae8ff33651f6 2026-03-09 16:40:43,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:43,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac550846f7ae8ff33651f6 HTTP/1.1" 200 None 2026-03-09 16:40:43,799 - lava_callback - DEBUG - Processing node 69ac550846f7ae8ff33651f6: baseline-arm-baylibre 2026-03-09 16:40:43,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac550846f7ae8ff33651f6 2026-03-09 16:40:43,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:43,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac550846f7ae8ff33651f6 HTTP/1.1" 200 None 2026-03-09 16:40:43,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:43,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a4fe31b554803095c08 HTTP/1.1" 200 None 2026-03-09 16:40:43,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:43,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac550846f7ae8ff33651f6 HTTP/1.1" 200 None 2026-03-09 16:40:43,863 - lava_callback - ERROR - Error processing callback for node 69ac550846f7ae8ff33651f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:41:23,876 - lava_callback - INFO - Received callback for node 69ac552c46f7ae8ff3365211 INFO: 172.21.0.1:54994 - "POST /node/69ac552c46f7ae8ff3365211 HTTP/1.0" 202 Accepted 2026-03-09 16:41:23,881 - lava_callback - INFO - Starting async job submit for node 69ac552c46f7ae8ff3365211 2026-03-09 16:41:23,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:41:23,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac552c46f7ae8ff3365211 HTTP/1.1" 200 None 2026-03-09 16:41:23,915 - lava_callback - DEBUG - Processing node 69ac552c46f7ae8ff3365211: baseline-arm-baylibre 2026-03-09 16:41:23,916 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac552c46f7ae8ff3365211 2026-03-09 16:41:23,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:41:23,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac552c46f7ae8ff3365211 HTTP/1.1" 200 None 2026-03-09 16:41:23,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:41:23,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:41:23,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:41:23,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac552c46f7ae8ff3365211 HTTP/1.1" 200 None 2026-03-09 16:41:23,990 - lava_callback - ERROR - Error processing callback for node 69ac552c46f7ae8ff3365211: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56146 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:42:03,826 - lava_callback - INFO - Received callback for node 69ac555146f7ae8ff3365230 INFO: 172.21.0.1:39730 - "POST /node/69ac555146f7ae8ff3365230 HTTP/1.0" 202 Accepted 2026-03-09 16:42:03,835 - lava_callback - INFO - Starting async job submit for node 69ac555146f7ae8ff3365230 2026-03-09 16:42:03,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:03,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac555146f7ae8ff3365230 HTTP/1.1" 200 None 2026-03-09 16:42:03,867 - lava_callback - DEBUG - Processing node 69ac555146f7ae8ff3365230: baseline-arm-baylibre 2026-03-09 16:42:03,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac555146f7ae8ff3365230 2026-03-09 16:42:03,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:03,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac555146f7ae8ff3365230 HTTP/1.1" 200 None 2026-03-09 16:42:03,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:03,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:42:03,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:03,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac555146f7ae8ff3365230 HTTP/1.1" 200 None 2026-03-09 16:42:03,986 - lava_callback - ERROR - Error processing callback for node 69ac555146f7ae8ff3365230: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60968 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:42:44,171 - lava_callback - INFO - Received callback for node 69ac557746f7ae8ff3365244 INFO: 172.21.0.1:43144 - "POST /node/69ac557746f7ae8ff3365244 HTTP/1.0" 202 Accepted 2026-03-09 16:42:44,176 - lava_callback - INFO - Starting async job submit for node 69ac557746f7ae8ff3365244 2026-03-09 16:42:44,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:44,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac557746f7ae8ff3365244 HTTP/1.1" 200 None 2026-03-09 16:42:44,192 - lava_callback - DEBUG - Processing node 69ac557746f7ae8ff3365244: baseline-arm-baylibre 2026-03-09 16:42:44,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac557746f7ae8ff3365244 2026-03-09 16:42:44,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:44,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac557746f7ae8ff3365244 HTTP/1.1" 200 None 2026-03-09 16:42:44,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:44,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:42:44,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:44,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac557746f7ae8ff3365244 HTTP/1.1" 200 None 2026-03-09 16:42:44,245 - lava_callback - ERROR - Error processing callback for node 69ac557746f7ae8ff3365244: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45982 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:43:03,918 - lava_callback - INFO - Received callback for node 69ac558a46f7ae8ff3365285 INFO: 172.21.0.1:41616 - "POST /node/69ac558a46f7ae8ff3365285 HTTP/1.0" 202 Accepted 2026-03-09 16:43:03,924 - lava_callback - INFO - Starting async job submit for node 69ac558a46f7ae8ff3365285 2026-03-09 16:43:03,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:03,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558a46f7ae8ff3365285 HTTP/1.1" 200 None 2026-03-09 16:43:03,941 - lava_callback - DEBUG - Processing node 69ac558a46f7ae8ff3365285: baseline-arm-baylibre 2026-03-09 16:43:03,941 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558a46f7ae8ff3365285 2026-03-09 16:43:03,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:03,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558a46f7ae8ff3365285 HTTP/1.1" 200 None 2026-03-09 16:43:03,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:03,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:03,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:03,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558a46f7ae8ff3365285 HTTP/1.1" 200 None 2026-03-09 16:43:03,999 - lava_callback - ERROR - Error processing callback for node 69ac558a46f7ae8ff3365285: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:04,576 - lava_callback - INFO - Received callback for node 69ac558d46f7ae8ff3365294 INFO: 172.21.0.1:41628 - "POST /node/69ac558d46f7ae8ff3365294 HTTP/1.0" 202 Accepted 2026-03-09 16:43:04,582 - lava_callback - INFO - Starting async job submit for node 69ac558d46f7ae8ff3365294 2026-03-09 16:43:04,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:04,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558d46f7ae8ff3365294 HTTP/1.1" 200 None 2026-03-09 16:43:04,598 - lava_callback - DEBUG - Processing node 69ac558d46f7ae8ff3365294: baseline-arm-baylibre 2026-03-09 16:43:04,598 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558d46f7ae8ff3365294 2026-03-09 16:43:04,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:04,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558d46f7ae8ff3365294 HTTP/1.1" 200 None 2026-03-09 16:43:04,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:04,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:04,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:04,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558d46f7ae8ff3365294 HTTP/1.1" 200 None 2026-03-09 16:43:04,651 - lava_callback - ERROR - Error processing callback for node 69ac558d46f7ae8ff3365294: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:05,261 - lava_callback - INFO - Received callback for node 69ac558746f7ae8ff336526a INFO: 172.21.0.1:41632 - "POST /node/69ac558746f7ae8ff336526a HTTP/1.0" 202 Accepted 2026-03-09 16:43:05,269 - lava_callback - INFO - Starting async job submit for node 69ac558746f7ae8ff336526a 2026-03-09 16:43:05,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558746f7ae8ff336526a HTTP/1.1" 200 None 2026-03-09 16:43:05,290 - lava_callback - DEBUG - Processing node 69ac558746f7ae8ff336526a: baseline-arm-baylibre 2026-03-09 16:43:05,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558746f7ae8ff336526a 2026-03-09 16:43:05,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558746f7ae8ff336526a HTTP/1.1" 200 None 2026-03-09 16:43:05,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:05,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558746f7ae8ff336526a HTTP/1.1" 200 None 2026-03-09 16:43:05,346 - lava_callback - ERROR - Error processing callback for node 69ac558746f7ae8ff336526a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:05,925 - lava_callback - INFO - Received callback for node 69ac558846f7ae8ff336526d INFO: 172.21.0.1:41646 - "POST /node/69ac558846f7ae8ff336526d HTTP/1.0" 202 Accepted 2026-03-09 16:43:05,931 - lava_callback - INFO - Starting async job submit for node 69ac558846f7ae8ff336526d 2026-03-09 16:43:05,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558846f7ae8ff336526d HTTP/1.1" 200 None 2026-03-09 16:43:05,956 - lava_callback - DEBUG - Processing node 69ac558846f7ae8ff336526d: baseline-arm-baylibre 2026-03-09 16:43:05,956 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558846f7ae8ff336526d 2026-03-09 16:43:05,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558846f7ae8ff336526d HTTP/1.1" 200 None 2026-03-09 16:43:05,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:05,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:06,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558846f7ae8ff336526d HTTP/1.1" 200 None 2026-03-09 16:43:06,014 - lava_callback - ERROR - Error processing callback for node 69ac558846f7ae8ff336526d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41662 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:43:06,612 - lava_callback - INFO - Received callback for node 69ac558b46f7ae8ff3365288 INFO: 172.21.0.1:41678 - "POST /node/69ac558b46f7ae8ff3365288 HTTP/1.0" 202 Accepted 2026-03-09 16:43:06,617 - lava_callback - INFO - Starting async job submit for node 69ac558b46f7ae8ff3365288 2026-03-09 16:43:06,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:06,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558b46f7ae8ff3365288 HTTP/1.1" 200 None 2026-03-09 16:43:06,635 - lava_callback - DEBUG - Processing node 69ac558b46f7ae8ff3365288: baseline-arm-baylibre 2026-03-09 16:43:06,635 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558b46f7ae8ff3365288 2026-03-09 16:43:06,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:06,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558b46f7ae8ff3365288 HTTP/1.1" 200 None 2026-03-09 16:43:06,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:06,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:06,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:06,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558b46f7ae8ff3365288 HTTP/1.1" 200 None 2026-03-09 16:43:06,688 - lava_callback - ERROR - Error processing callback for node 69ac558b46f7ae8ff3365288: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:07,296 - lava_callback - INFO - Received callback for node 69ac558f46f7ae8ff3365298 INFO: 172.21.0.1:41694 - "POST /node/69ac558f46f7ae8ff3365298 HTTP/1.0" 202 Accepted 2026-03-09 16:43:07,301 - lava_callback - INFO - Starting async job submit for node 69ac558f46f7ae8ff3365298 2026-03-09 16:43:07,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:07,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558f46f7ae8ff3365298 HTTP/1.1" 200 None 2026-03-09 16:43:07,325 - lava_callback - DEBUG - Processing node 69ac558f46f7ae8ff3365298: baseline-arm-baylibre 2026-03-09 16:43:07,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558f46f7ae8ff3365298 2026-03-09 16:43:07,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:07,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558f46f7ae8ff3365298 HTTP/1.1" 200 None 2026-03-09 16:43:07,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:07,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:07,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:07,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558f46f7ae8ff3365298 HTTP/1.1" 200 None 2026-03-09 16:43:07,381 - lava_callback - ERROR - Error processing callback for node 69ac558f46f7ae8ff3365298: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40362 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:43:17,380 - lava_callback - INFO - Received callback for node 69ac559b46f7ae8ff33652bf INFO: 172.21.0.1:40368 - "POST /node/69ac559b46f7ae8ff33652bf HTTP/1.0" 202 Accepted 2026-03-09 16:43:17,385 - lava_callback - INFO - Starting async job submit for node 69ac559b46f7ae8ff33652bf 2026-03-09 16:43:17,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:17,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559b46f7ae8ff33652bf HTTP/1.1" 200 None 2026-03-09 16:43:17,401 - lava_callback - DEBUG - Processing node 69ac559b46f7ae8ff33652bf: baseline-arm-clabbe 2026-03-09 16:43:17,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac559b46f7ae8ff33652bf 2026-03-09 16:43:17,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:17,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559b46f7ae8ff33652bf HTTP/1.1" 200 None 2026-03-09 16:43:17,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:17,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:17,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:17,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac559b46f7ae8ff33652bf HTTP/1.1" 200 None 2026-03-09 16:43:17,461 - lava_callback - ERROR - Error processing callback for node 69ac559b46f7ae8ff33652bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:18,047 - lava_callback - INFO - Received callback for node 69ac559c46f7ae8ff33652c4 INFO: 172.21.0.1:40382 - "POST /node/69ac559c46f7ae8ff33652c4 HTTP/1.0" 202 Accepted 2026-03-09 16:43:18,052 - lava_callback - INFO - Starting async job submit for node 69ac559c46f7ae8ff33652c4 2026-03-09 16:43:18,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559c46f7ae8ff33652c4 HTTP/1.1" 200 None 2026-03-09 16:43:18,073 - lava_callback - DEBUG - Processing node 69ac559c46f7ae8ff33652c4: baseline-arm-clabbe 2026-03-09 16:43:18,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac559c46f7ae8ff33652c4 2026-03-09 16:43:18,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559c46f7ae8ff33652c4 HTTP/1.1" 200 None 2026-03-09 16:43:18,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:18,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac559c46f7ae8ff33652c4 HTTP/1.1" 200 None 2026-03-09 16:43:18,133 - lava_callback - ERROR - Error processing callback for node 69ac559c46f7ae8ff33652c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:18,707 - lava_callback - INFO - Received callback for node 69ac55a046f7ae8ff33652d1 INFO: 172.21.0.1:40384 - "POST /node/69ac55a046f7ae8ff33652d1 HTTP/1.0" 202 Accepted 2026-03-09 16:43:18,712 - lava_callback - INFO - Starting async job submit for node 69ac55a046f7ae8ff33652d1 2026-03-09 16:43:18,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55a046f7ae8ff33652d1 HTTP/1.1" 200 None 2026-03-09 16:43:18,734 - lava_callback - DEBUG - Processing node 69ac55a046f7ae8ff33652d1: baseline-arm-clabbe 2026-03-09 16:43:18,734 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac55a046f7ae8ff33652d1 2026-03-09 16:43:18,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55a046f7ae8ff33652d1 HTTP/1.1" 200 None 2026-03-09 16:43:18,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:18,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac55a046f7ae8ff33652d1 HTTP/1.1" 200 None 2026-03-09 16:43:18,809 - lava_callback - ERROR - Error processing callback for node 69ac55a046f7ae8ff33652d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:23,891 - lava_callback - INFO - Received callback for node 69ac559b46f7ae8ff33652c1 INFO: 172.21.0.1:35426 - "POST /node/69ac559b46f7ae8ff33652c1 HTTP/1.0" 202 Accepted 2026-03-09 16:43:23,897 - lava_callback - INFO - Starting async job submit for node 69ac559b46f7ae8ff33652c1 2026-03-09 16:43:23,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:23,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559b46f7ae8ff33652c1 HTTP/1.1" 200 None 2026-03-09 16:43:23,913 - lava_callback - DEBUG - Processing node 69ac559b46f7ae8ff33652c1: baseline-arm-baylibre 2026-03-09 16:43:23,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac559b46f7ae8ff33652c1 2026-03-09 16:43:23,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:23,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559b46f7ae8ff33652c1 HTTP/1.1" 200 None 2026-03-09 16:43:23,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:23,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:43:23,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:23,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac559b46f7ae8ff33652c1 HTTP/1.1" 200 None 2026-03-09 16:43:23,965 - lava_callback - ERROR - Error processing callback for node 69ac559b46f7ae8ff33652c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43238 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:43:57,367 - lava_callback - INFO - Received callback for node 69ac55c146f7ae8ff336532f INFO: 172.21.0.1:43252 - "POST /node/69ac55c146f7ae8ff336532f HTTP/1.0" 202 Accepted 2026-03-09 16:43:57,373 - lava_callback - INFO - Starting async job submit for node 69ac55c146f7ae8ff336532f 2026-03-09 16:43:57,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:57,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55c146f7ae8ff336532f HTTP/1.1" 200 None 2026-03-09 16:43:57,389 - lava_callback - DEBUG - Processing node 69ac55c146f7ae8ff336532f: baseline-arm-clabbe 2026-03-09 16:43:57,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac55c146f7ae8ff336532f 2026-03-09 16:43:57,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:57,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55c146f7ae8ff336532f HTTP/1.1" 200 None 2026-03-09 16:43:57,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:57,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:43:57,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:57,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac55c146f7ae8ff336532f HTTP/1.1" 200 None 2026-03-09 16:43:57,458 - lava_callback - ERROR - Error processing callback for node 69ac55c146f7ae8ff336532f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54246 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:44:37,412 - lava_callback - INFO - Received callback for node 69ac55e946f7ae8ff3365397 INFO: 172.21.0.1:54250 - "POST /node/69ac55e946f7ae8ff3365397 HTTP/1.0" 202 Accepted 2026-03-09 16:44:37,417 - lava_callback - INFO - Starting async job submit for node 69ac55e946f7ae8ff3365397 2026-03-09 16:44:37,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:44:37,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55e946f7ae8ff3365397 HTTP/1.1" 200 None 2026-03-09 16:44:37,434 - lava_callback - DEBUG - Processing node 69ac55e946f7ae8ff3365397: baseline-arm-clabbe 2026-03-09 16:44:37,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac55e946f7ae8ff3365397 2026-03-09 16:44:37,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:44:37,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55e946f7ae8ff3365397 HTTP/1.1" 200 None 2026-03-09 16:44:37,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:44:37,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:44:37,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:44:37,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac55e946f7ae8ff3365397 HTTP/1.1" 200 None 2026-03-09 16:44:37,503 - lava_callback - ERROR - Error processing callback for node 69ac55e946f7ae8ff3365397: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57904 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:45:17,372 - lava_callback - INFO - Received callback for node 69ac561046f7ae8ff33653d9 INFO: 172.21.0.1:57908 - "POST /node/69ac561046f7ae8ff33653d9 HTTP/1.0" 202 Accepted 2026-03-09 16:45:17,377 - lava_callback - INFO - Starting async job submit for node 69ac561046f7ae8ff33653d9 2026-03-09 16:45:17,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:17,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac561046f7ae8ff33653d9 HTTP/1.1" 200 None 2026-03-09 16:45:17,393 - lava_callback - DEBUG - Processing node 69ac561046f7ae8ff33653d9: baseline-arm-clabbe 2026-03-09 16:45:17,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac561046f7ae8ff33653d9 2026-03-09 16:45:17,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:17,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac561046f7ae8ff33653d9 HTTP/1.1" 200 None 2026-03-09 16:45:17,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:17,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:45:17,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:17,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac561046f7ae8ff33653d9 HTTP/1.1" 200 None 2026-03-09 16:45:17,444 - lava_callback - ERROR - Error processing callback for node 69ac561046f7ae8ff33653d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52878 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:45:44,179 - lava_callback - INFO - Received callback for node 69ac563446f7ae8ff33653fa INFO: 172.21.0.1:38932 - "POST /node/69ac563446f7ae8ff33653fa HTTP/1.0" 202 Accepted 2026-03-09 16:45:44,185 - lava_callback - INFO - Starting async job submit for node 69ac563446f7ae8ff33653fa 2026-03-09 16:45:44,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:44,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac563446f7ae8ff33653fa HTTP/1.1" 200 None 2026-03-09 16:45:44,201 - lava_callback - DEBUG - Processing node 69ac563446f7ae8ff33653fa: baseline-arm-baylibre 2026-03-09 16:45:44,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac563446f7ae8ff33653fa 2026-03-09 16:45:44,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:44,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac563446f7ae8ff33653fa HTTP/1.1" 200 None 2026-03-09 16:45:44,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:44,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a4fe31b554803095c08 HTTP/1.1" 200 None 2026-03-09 16:45:44,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:44,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac563446f7ae8ff33653fa HTTP/1.1" 200 None 2026-03-09 16:45:44,274 - lava_callback - ERROR - Error processing callback for node 69ac563446f7ae8ff33653fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48726 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:49:16,778 - lava_callback - INFO - Received callback for node 69aef9fdce46d8ec186abe8e INFO: 172.21.0.1:48736 - "POST /node/69aef9fdce46d8ec186abe8e HTTP/1.0" 202 Accepted 2026-03-09 16:49:16,784 - lava_callback - INFO - Starting async job submit for node 69aef9fdce46d8ec186abe8e 2026-03-09 16:49:16,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:16,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fdce46d8ec186abe8e HTTP/1.1" 200 None 2026-03-09 16:49:16,813 - lava_callback - DEBUG - Processing node 69aef9fdce46d8ec186abe8e: baseline-arm 2026-03-09 16:49:16,818 - lava_callback - WARNING - Failed to upload log 2026-03-09 16:49:16,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:16,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:16,986 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fdce46d8ec186abe8e/lava-logs.yaml 2026-03-09 16:49:16,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:17,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:17,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fdce46d8ec186abe8e/lava_callback.json.gz 2026-03-09 16:49:17,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:17,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fdce46d8ec186abe8e HTTP/1.1" 200 None 2026-03-09 16:49:17,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:17,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:49:17,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:17,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef9fdce46d8ec186abe8e HTTP/1.1" 200 None 2026-03-09 16:49:17,229 - lava_callback - INFO - Completed processing callback for node 69aef9fdce46d8ec186abe8e INFO: 172.21.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46698 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:49:43,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:43,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60146 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:49:51,816 - lava_callback - INFO - Received callback for node 69aefa00ce46d8ec186abec0 INFO: 172.21.0.1:41862 - "POST /node/69aefa00ce46d8ec186abec0 HTTP/1.0" 202 Accepted 2026-03-09 16:49:51,865 - lava_callback - INFO - Starting async job submit for node 69aefa00ce46d8ec186abec0 2026-03-09 16:49:51,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:51,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa00ce46d8ec186abec0 HTTP/1.1" 200 None 2026-03-09 16:49:51,898 - lava_callback - DEBUG - Processing node 69aefa00ce46d8ec186abec0: baseline-arm 2026-03-09 16:49:52,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:52,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:52,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aefa00ce46d8ec186abec0/log.txt.gz 2026-03-09 16:49:52,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:52,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:52,398 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aefa00ce46d8ec186abec0/lava-logs.yaml 2026-03-09 16:49:52,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:52,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:52,545 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aefa00ce46d8ec186abec0/lava_callback.json.gz 2026-03-09 16:49:52,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:52,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa00ce46d8ec186abec0 HTTP/1.1" 200 None 2026-03-09 16:49:52,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:52,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:49:52,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:52,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa00ce46d8ec186abec0 HTTP/1.1" 200 None 2026-03-09 16:49:52,638 - lava_callback - INFO - Completed processing callback for node 69aefa00ce46d8ec186abec0 2026-03-09 16:49:53,241 - lava_callback - INFO - Received callback for node 69aef9fdce46d8ec186abe91 INFO: 172.21.0.1:41866 - "POST /node/69aef9fdce46d8ec186abe91 HTTP/1.0" 202 Accepted 2026-03-09 16:49:53,290 - lava_callback - INFO - Starting async job submit for node 69aef9fdce46d8ec186abe91 2026-03-09 16:49:53,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:53,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fdce46d8ec186abe91 HTTP/1.1" 200 None 2026-03-09 16:49:53,323 - lava_callback - DEBUG - Processing node 69aef9fdce46d8ec186abe91: baseline-arm 2026-03-09 16:49:53,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:53,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:53,589 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fdce46d8ec186abe91/log.txt.gz 2026-03-09 16:49:53,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:53,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:53,833 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fdce46d8ec186abe91/lava-logs.yaml 2026-03-09 16:49:53,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:54,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:54,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fdce46d8ec186abe91/lava_callback.json.gz 2026-03-09 16:49:54,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:54,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fdce46d8ec186abe91 HTTP/1.1" 200 None 2026-03-09 16:49:54,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:54,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:49:54,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:54,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef9fdce46d8ec186abe91 HTTP/1.1" 200 None 2026-03-09 16:49:54,147 - lava_callback - INFO - Completed processing callback for node 69aef9fdce46d8ec186abe91 INFO: 172.21.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44748 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:50:13,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:13,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51754 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:50:26,987 - lava_callback - INFO - Received callback for node 69aef9ffce46d8ec186abebd INFO: 172.21.0.1:51770 - "POST /node/69aef9ffce46d8ec186abebd HTTP/1.0" 202 Accepted 2026-03-09 16:50:26,993 - lava_callback - INFO - Starting async job submit for node 69aef9ffce46d8ec186abebd 2026-03-09 16:50:27,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:27,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9ffce46d8ec186abebd HTTP/1.1" 200 None 2026-03-09 16:50:27,033 - lava_callback - DEBUG - Processing node 69aef9ffce46d8ec186abebd: baseline-arm 2026-03-09 16:50:27,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:27,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:27,414 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9ffce46d8ec186abebd/log.txt.gz 2026-03-09 16:50:27,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:27,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:27,765 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9ffce46d8ec186abebd/lava-logs.yaml 2026-03-09 16:50:27,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:27,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:27,918 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9ffce46d8ec186abebd/lava_callback.json.gz 2026-03-09 16:50:27,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:27,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9ffce46d8ec186abebd HTTP/1.1" 200 None 2026-03-09 16:50:27,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:27,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:27,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:28,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef9ffce46d8ec186abebd HTTP/1.1" 200 None 2026-03-09 16:50:28,006 - lava_callback - INFO - Completed processing callback for node 69aef9ffce46d8ec186abebd INFO: 172.21.0.1:52604 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:50:40,789 - lava_callback - INFO - Received callback for node 69aef9fcce46d8ec186abe8b INFO: 172.21.0.1:44378 - "POST /node/69aef9fcce46d8ec186abe8b HTTP/1.0" 202 Accepted 2026-03-09 16:50:40,795 - lava_callback - INFO - Starting async job submit for node 69aef9fcce46d8ec186abe8b 2026-03-09 16:50:40,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:40,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fcce46d8ec186abe8b HTTP/1.1" 200 None 2026-03-09 16:50:40,831 - lava_callback - DEBUG - Processing node 69aef9fcce46d8ec186abe8b: baseline-arm 2026-03-09 16:50:41,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:41,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:41,160 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fcce46d8ec186abe8b/log.txt.gz 2026-03-09 16:50:41,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:41,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:41,455 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fcce46d8ec186abe8b/lava-logs.yaml 2026-03-09 16:50:41,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:41,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:41,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fcce46d8ec186abe8b/lava_callback.json.gz 2026-03-09 16:50:41,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:41,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fcce46d8ec186abe8b HTTP/1.1" 200 None 2026-03-09 16:50:41,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:41,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:41,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:41,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef9fcce46d8ec186abe8b HTTP/1.1" 200 None 2026-03-09 16:50:41,693 - lava_callback - INFO - Completed processing callback for node 69aef9fcce46d8ec186abe8b 2026-03-09 16:50:41,811 - lava_callback - INFO - Received callback for node 69aefa0dce46d8ec186abedf INFO: 172.21.0.1:44388 - "POST /node/69aefa0dce46d8ec186abedf HTTP/1.0" 202 Accepted 2026-03-09 16:50:41,818 - lava_callback - INFO - Starting async job submit for node 69aefa0dce46d8ec186abedf 2026-03-09 16:50:41,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:41,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa0dce46d8ec186abedf HTTP/1.1" 200 None 2026-03-09 16:50:41,867 - lava_callback - DEBUG - Processing node 69aefa0dce46d8ec186abedf: baseline-arm-broonie 2026-03-09 16:50:42,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:42,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:42,247 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0dce46d8ec186abedf/log.txt.gz 2026-03-09 16:50:42,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:42,396 - lava_callback - INFO - Received callback for node 69aefa1dce46d8ec186abf53 2026-03-09 16:50:42,402 - lava_callback - INFO - Starting async job submit for node 69aefa1dce46d8ec186abf53 INFO: 172.21.0.1:44394 - "POST /node/69aefa1dce46d8ec186abf53 HTTP/1.0" 202 Accepted 2026-03-09 16:50:42,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:42,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1dce46d8ec186abf53 HTTP/1.1" 200 None 2026-03-09 16:50:42,454 - lava_callback - DEBUG - Processing node 69aefa1dce46d8ec186abf53: baseline-arm-pengutronix 2026-03-09 16:50:42,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:42,600 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0dce46d8ec186abedf/lava-logs.yaml 2026-03-09 16:50:42,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:42,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:42,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:42,723 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1dce46d8ec186abf53/log.txt.gz 2026-03-09 16:50:42,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:42,828 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0dce46d8ec186abedf/lava_callback.json.gz 2026-03-09 16:50:42,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:42,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:42,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa0dce46d8ec186abedf HTTP/1.1" 200 None 2026-03-09 16:50:42,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:42,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:42,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:42,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa0dce46d8ec186abedf HTTP/1.1" 200 None 2026-03-09 16:50:42,921 - lava_callback - INFO - Completed processing callback for node 69aefa0dce46d8ec186abedf 2026-03-09 16:50:43,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:43,059 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1dce46d8ec186abf53/lava-logs.yaml 2026-03-09 16:50:43,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:43,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:43,215 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1dce46d8ec186abf53/lava_callback.json.gz 2026-03-09 16:50:43,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:43,234 - lava_callback - INFO - Received callback for node 69aefa0ece46d8ec186abee5 INFO: 172.21.0.1:44396 - "POST /node/69aefa0ece46d8ec186abee5 HTTP/1.0" 202 Accepted 2026-03-09 16:50:43,240 - lava_callback - INFO - Starting async job submit for node 69aefa0ece46d8ec186abee5 2026-03-09 16:50:43,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:43,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa0ece46d8ec186abee5 HTTP/1.1" 200 None 2026-03-09 16:50:43,282 - lava_callback - DEBUG - Processing node 69aefa0ece46d8ec186abee5: baseline-arm-broonie 2026-03-09 16:50:43,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1dce46d8ec186abf53 HTTP/1.1" 200 None 2026-03-09 16:50:43,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:43,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:43,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:43,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa1dce46d8ec186abf53 HTTP/1.1" 200 None 2026-03-09 16:50:43,554 - lava_callback - INFO - Completed processing callback for node 69aefa1dce46d8ec186abf53 2026-03-09 16:50:43,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:43,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:43,802 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0ece46d8ec186abee5/log.txt.gz 2026-03-09 16:50:43,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:43,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:43,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-09 16:50:44,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:44,115 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0ece46d8ec186abee5/lava-logs.yaml 2026-03-09 16:50:44,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:44,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:44,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0ece46d8ec186abee5/lava_callback.json.gz 2026-03-09 16:50:44,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:44,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa0ece46d8ec186abee5 HTTP/1.1" 200 None 2026-03-09 16:50:44,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:44,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:44,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:44,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa0ece46d8ec186abee5 HTTP/1.1" 200 None 2026-03-09 16:50:44,350 - lava_callback - INFO - Completed processing callback for node 69aefa0ece46d8ec186abee5 INFO: 172.21.0.1:44398 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:50:48,532 - lava_callback - INFO - Received callback for node 69aec8aefeb96b28aa34b74d INFO: 172.21.0.1:44406 - "POST /node/69aec8aefeb96b28aa34b74d HTTP/1.0" 202 Accepted 2026-03-09 16:50:48,538 - lava_callback - INFO - Starting async job submit for node 69aec8aefeb96b28aa34b74d 2026-03-09 16:50:48,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:48,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec8aefeb96b28aa34b74d HTTP/1.1" 200 None 2026-03-09 16:50:48,588 - lava_callback - DEBUG - Processing node 69aec8aefeb96b28aa34b74d: baseline-arm64-broonie 2026-03-09 16:50:48,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:49,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:49,141 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aec8aefeb96b28aa34b74d/log.txt.gz 2026-03-09 16:50:49,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:49,568 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:49,570 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aec8aefeb96b28aa34b74d/lava-logs.yaml 2026-03-09 16:50:49,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:49,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:49,734 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aec8aefeb96b28aa34b74d/lava_callback.json.gz DEBUG: Setup failed for 69aec8aefeb96b28aa34b74d. Transitting job node result: incomplete -> fail 2026-03-09 16:50:49,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:49,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec8aefeb96b28aa34b74d HTTP/1.1" 200 None 2026-03-09 16:50:49,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:49,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-09 16:50:49,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:49,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aec8aefeb96b28aa34b74d HTTP/1.1" 200 None 2026-03-09 16:50:49,934 - lava_callback - INFO - Completed processing callback for node 69aec8aefeb96b28aa34b74d 2026-03-09 16:50:52,093 - lava_callback - INFO - Received callback for node 69aefa21ce46d8ec186abf5c INFO: 172.21.0.1:49112 - "POST /node/69aefa21ce46d8ec186abf5c HTTP/1.0" 202 Accepted 2026-03-09 16:50:52,141 - lava_callback - INFO - Starting async job submit for node 69aefa21ce46d8ec186abf5c 2026-03-09 16:50:52,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:52,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa21ce46d8ec186abf5c HTTP/1.1" 200 None 2026-03-09 16:50:52,176 - lava_callback - DEBUG - Processing node 69aefa21ce46d8ec186abf5c: baseline-arm-pengutronix 2026-03-09 16:50:52,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:52,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:52,397 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa21ce46d8ec186abf5c/log.txt.gz 2026-03-09 16:50:52,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:52,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:52,657 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa21ce46d8ec186abf5c/lava-logs.yaml 2026-03-09 16:50:52,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:52,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:52,781 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa21ce46d8ec186abf5c/lava_callback.json.gz 2026-03-09 16:50:52,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:52,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa21ce46d8ec186abf5c HTTP/1.1" 200 None 2026-03-09 16:50:52,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:52,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:52,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:52,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa21ce46d8ec186abf5c HTTP/1.1" 200 None 2026-03-09 16:50:52,866 - lava_callback - INFO - Completed processing callback for node 69aefa21ce46d8ec186abf5c INFO: 172.21.0.1:49126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36884 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:51:08,280 - lava_callback - INFO - Received callback for node 69aefa1ace46d8ec186abf42 INFO: 172.21.0.1:36896 - "POST /node/69aefa1ace46d8ec186abf42 HTTP/1.0" 202 Accepted 2026-03-09 16:51:08,286 - lava_callback - INFO - Starting async job submit for node 69aefa1ace46d8ec186abf42 2026-03-09 16:51:08,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:08,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1ace46d8ec186abf42 HTTP/1.1" 200 None 2026-03-09 16:51:08,322 - lava_callback - DEBUG - Processing node 69aefa1ace46d8ec186abf42: baseline-arm-kontron 2026-03-09 16:51:08,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:08,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:08,632 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69aefa1ace46d8ec186abf42/log.txt.gz 2026-03-09 16:51:08,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:08,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:08,900 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69aefa1ace46d8ec186abf42/lava-logs.yaml 2026-03-09 16:51:08,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:09,030 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:09,032 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69aefa1ace46d8ec186abf42/lava_callback.json.gz 2026-03-09 16:51:09,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:09,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1ace46d8ec186abf42 HTTP/1.1" 200 None 2026-03-09 16:51:09,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:09,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:51:09,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:09,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa1ace46d8ec186abf42 HTTP/1.1" 200 None 2026-03-09 16:51:09,123 - lava_callback - INFO - Completed processing callback for node 69aefa1ace46d8ec186abf42 2026-03-09 16:51:09,152 - lava_callback - INFO - Received callback for node 69aefa12ce46d8ec186abef7 INFO: 172.21.0.1:36910 - "POST /node/69aefa12ce46d8ec186abef7 HTTP/1.0" 202 Accepted 2026-03-09 16:51:09,162 - lava_callback - INFO - Starting async job submit for node 69aefa12ce46d8ec186abef7 2026-03-09 16:51:09,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:09,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa12ce46d8ec186abef7 HTTP/1.1" 200 None 2026-03-09 16:51:09,231 - lava_callback - DEBUG - Processing node 69aefa12ce46d8ec186abef7: baseline-arm-clabbe 2026-03-09 16:51:09,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:09,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:09,646 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa12ce46d8ec186abef7/log.txt.gz 2026-03-09 16:51:09,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:09,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:09,970 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa12ce46d8ec186abef7/lava-logs.yaml 2026-03-09 16:51:09,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:10,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:10,179 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa12ce46d8ec186abef7/lava_callback.json.gz 2026-03-09 16:51:10,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:10,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa12ce46d8ec186abef7 HTTP/1.1" 200 None 2026-03-09 16:51:10,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:10,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:51:10,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:10,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa12ce46d8ec186abef7 HTTP/1.1" 200 None 2026-03-09 16:51:10,257 - lava_callback - INFO - Completed processing callback for node 69aefa12ce46d8ec186abef7 2026-03-09 16:51:13,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:13,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:49012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37966 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:51:41,441 - lava_callback - INFO - Received callback for node 69aefa1ece46d8ec186abf56 INFO: 172.21.0.1:49440 - "POST /node/69aefa1ece46d8ec186abf56 HTTP/1.0" 202 Accepted 2026-03-09 16:51:41,447 - lava_callback - INFO - Starting async job submit for node 69aefa1ece46d8ec186abf56 2026-03-09 16:51:41,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:41,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1ece46d8ec186abf56 HTTP/1.1" 200 None 2026-03-09 16:51:41,484 - lava_callback - DEBUG - Processing node 69aefa1ece46d8ec186abf56: baseline-arm-pengutronix 2026-03-09 16:51:41,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:41,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:41,761 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1ece46d8ec186abf56/log.txt.gz 2026-03-09 16:51:41,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:42,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:42,042 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1ece46d8ec186abf56/lava-logs.yaml 2026-03-09 16:51:42,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:42,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:42,214 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1ece46d8ec186abf56/lava_callback.json.gz 2026-03-09 16:51:42,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:42,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1ece46d8ec186abf56 HTTP/1.1" 200 None 2026-03-09 16:51:42,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:42,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:51:42,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:42,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa1ece46d8ec186abf56 HTTP/1.1" 200 None 2026-03-09 16:51:42,395 - lava_callback - INFO - Completed processing callback for node 69aefa1ece46d8ec186abf56 2026-03-09 16:51:43,830 - lava_callback - INFO - Received callback for node 69aefa36ce46d8ec186abfa1 INFO: 172.21.0.1:49448 - "POST /node/69aefa36ce46d8ec186abfa1 HTTP/1.0" 202 Accepted 2026-03-09 16:51:43,836 - lava_callback - INFO - Starting async job submit for node 69aefa36ce46d8ec186abfa1 2026-03-09 16:51:43,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:43,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa36ce46d8ec186abfa1 HTTP/1.1" 200 None 2026-03-09 16:51:43,888 - lava_callback - DEBUG - Processing node 69aefa36ce46d8ec186abfa1: ltp-dio 2026-03-09 16:51:43,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:44,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 16:51:44,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:44,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:44,398 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69aefa36ce46d8ec186abfa1/log.txt.gz 2026-03-09 16:51:44,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:44,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:44,844 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69aefa36ce46d8ec186abfa1/lava-logs.yaml 2026-03-09 16:51:44,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:45,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:45,041 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69aefa36ce46d8ec186abfa1/lava_callback.json.gz 2026-03-09 16:51:45,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:45,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa36ce46d8ec186abfa1 HTTP/1.1" 200 None 2026-03-09 16:51:45,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:45,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:51:45,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:45,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa36ce46d8ec186abfa1 HTTP/1.1" 200 None 2026-03-09 16:51:45,139 - lava_callback - INFO - Completed processing callback for node 69aefa36ce46d8ec186abfa1 INFO: 172.21.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50370 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:52:08,414 - lava_callback - INFO - Received callback for node 69aefa10ce46d8ec186abee8 INFO: 172.21.0.1:50372 - "POST /node/69aefa10ce46d8ec186abee8 HTTP/1.0" 202 Accepted 2026-03-09 16:52:08,420 - lava_callback - INFO - Starting async job submit for node 69aefa10ce46d8ec186abee8 2026-03-09 16:52:08,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:08,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa10ce46d8ec186abee8 HTTP/1.1" 200 None 2026-03-09 16:52:08,458 - lava_callback - DEBUG - Processing node 69aefa10ce46d8ec186abee8: baseline-arm-broonie 2026-03-09 16:52:08,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:52:08,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:08,876 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa10ce46d8ec186abee8/log.txt.gz 2026-03-09 16:52:09,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:52:09,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:09,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa10ce46d8ec186abee8/lava-logs.yaml 2026-03-09 16:52:09,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:52:09,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:09,589 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa10ce46d8ec186abee8/lava_callback.json.gz 2026-03-09 16:52:09,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:09,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa10ce46d8ec186abee8 HTTP/1.1" 200 None 2026-03-09 16:52:09,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:09,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:52:09,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:09,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa10ce46d8ec186abee8 HTTP/1.1" 200 None 2026-03-09 16:52:09,803 - lava_callback - INFO - Completed processing callback for node 69aefa10ce46d8ec186abee8 2026-03-09 16:52:14,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:14,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:52:56,234 - lava_callback - INFO - Received callback for node 69aefa13ce46d8ec186abefb INFO: 172.21.0.1:39042 - "POST /node/69aefa13ce46d8ec186abefb HTTP/1.0" 202 Accepted 2026-03-09 16:52:56,240 - lava_callback - INFO - Starting async job submit for node 69aefa13ce46d8ec186abefb 2026-03-09 16:52:56,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:56,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa13ce46d8ec186abefb HTTP/1.1" 200 None 2026-03-09 16:52:56,277 - lava_callback - DEBUG - Processing node 69aefa13ce46d8ec186abefb: baseline-arm-clabbe 2026-03-09 16:52:56,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:39048 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:52:56,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:56,623 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa13ce46d8ec186abefb/log.txt.gz 2026-03-09 16:52:56,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:52:56,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:56,925 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa13ce46d8ec186abefb/lava-logs.yaml 2026-03-09 16:52:56,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:52:57,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:57,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa13ce46d8ec186abefb/lava_callback.json.gz 2026-03-09 16:52:57,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:57,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa13ce46d8ec186abefb HTTP/1.1" 200 None 2026-03-09 16:52:57,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:57,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:52:57,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:57,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa13ce46d8ec186abefb HTTP/1.1" 200 None 2026-03-09 16:52:57,145 - lava_callback - INFO - Completed processing callback for node 69aefa13ce46d8ec186abefb INFO: 172.21.0.1:55306 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:53:14,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:53:14,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40948 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:54:35,354 - lava_callback - INFO - Received callback for node 69aefa17ce46d8ec186abf07 INFO: 172.21.0.1:40110 - "POST /node/69aefa17ce46d8ec186abf07 HTTP/1.0" 202 Accepted 2026-03-09 16:54:35,367 - lava_callback - INFO - Starting async job submit for node 69aefa17ce46d8ec186abf07 2026-03-09 16:54:35,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:35,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa17ce46d8ec186abf07 HTTP/1.1" 200 None 2026-03-09 16:54:35,422 - lava_callback - DEBUG - Processing node 69aefa17ce46d8ec186abf07: baseline-arm-clabbe 2026-03-09 16:54:35,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:54:36,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:54:36,010 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa17ce46d8ec186abf07/log.txt.gz 2026-03-09 16:54:36,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40112 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:54:36,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:54:36,682 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa17ce46d8ec186abf07/lava-logs.yaml 2026-03-09 16:54:36,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:54:36,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:54:36,961 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa17ce46d8ec186abf07/lava_callback.json.gz 2026-03-09 16:54:36,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:36,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa17ce46d8ec186abf07 HTTP/1.1" 200 None 2026-03-09 16:54:36,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:37,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:54:37,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:37,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa17ce46d8ec186abf07 HTTP/1.1" 200 None 2026-03-09 16:54:37,046 - lava_callback - INFO - Completed processing callback for node 69aefa17ce46d8ec186abf07 2026-03-09 16:54:44,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:44,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32858 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:54:54,937 - lava_callback - INFO - Received callback for node 69aefb48ce46d8ec186ac13a INFO: 172.21.0.1:39816 - "POST /node/69aefb48ce46d8ec186ac13a HTTP/1.0" 202 Accepted 2026-03-09 16:54:54,947 - lava_callback - INFO - Starting async job submit for node 69aefb48ce46d8ec186ac13a 2026-03-09 16:54:54,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:54,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefb48ce46d8ec186ac13a HTTP/1.1" 200 None 2026-03-09 16:54:54,978 - lava_callback - DEBUG - Processing node 69aefb48ce46d8ec186ac13a: baseline-arm 2026-03-09 16:54:54,988 - lava_callback - WARNING - Failed to upload log 2026-03-09 16:54:54,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:54:55,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:54:55,140 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aefb48ce46d8ec186ac13a/lava-logs.yaml 2026-03-09 16:54:55,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:54:55,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:54:55,287 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aefb48ce46d8ec186ac13a/lava_callback.json.gz 2026-03-09 16:54:55,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:55,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefb48ce46d8ec186ac13a HTTP/1.1" 200 None 2026-03-09 16:54:55,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:55,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:54:55,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:55,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefb48ce46d8ec186ac13a HTTP/1.1" 200 None 2026-03-09 16:54:55,381 - lava_callback - INFO - Completed processing callback for node 69aefb48ce46d8ec186ac13a INFO: 172.21.0.1:39820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32956 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:55:14,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:14,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39028 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:55:28,993 - lava_callback - INFO - Received callback for node 69aefa3bce46d8ec186abfbb INFO: 172.21.0.1:39030 - "POST /node/69aefa3bce46d8ec186abfbb HTTP/1.0" 202 Accepted 2026-03-09 16:55:29,000 - lava_callback - INFO - Starting async job submit for node 69aefa3bce46d8ec186abfbb 2026-03-09 16:55:29,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:29,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa3bce46d8ec186abfbb HTTP/1.1" 200 None 2026-03-09 16:55:29,039 - lava_callback - DEBUG - Processing node 69aefa3bce46d8ec186abfbb: ltp-syscalls 2026-03-09 16:55:29,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:55:29,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:55:29,967 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69aefa3bce46d8ec186abfbb/log.txt.gz 2026-03-09 16:55:30,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:55:30,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:55:30,662 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69aefa3bce46d8ec186abfbb/lava-logs.yaml 2026-03-09 16:55:30,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:55:30,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:55:30,874 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69aefa3bce46d8ec186abfbb/lava_callback.json.gz 2026-03-09 16:55:30,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:30,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa3bce46d8ec186abfbb HTTP/1.1" 200 None 2026-03-09 16:55:30,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:30,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:55:30,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:30,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa3bce46d8ec186abfbb HTTP/1.1" 200 None 2026-03-09 16:55:30,982 - lava_callback - INFO - Completed processing callback for node 69aefa3bce46d8ec186abfbb INFO: 172.21.0.1:52856 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:55:44,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:44,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34778 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:56:04,778 - lava_callback - INFO - Received callback for node 69ac589146f7ae8ff3365632 INFO: 172.21.0.1:44624 - "POST /node/69ac589146f7ae8ff3365632 HTTP/1.0" 202 Accepted 2026-03-09 16:56:04,783 - lava_callback - INFO - Starting async job submit for node 69ac589146f7ae8ff3365632 2026-03-09 16:56:04,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:04,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589146f7ae8ff3365632 HTTP/1.1" 200 None 2026-03-09 16:56:04,799 - lava_callback - DEBUG - Processing node 69ac589146f7ae8ff3365632: baseline-arm64-baylibre 2026-03-09 16:56:04,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589146f7ae8ff3365632 2026-03-09 16:56:04,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:04,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589146f7ae8ff3365632 HTTP/1.1" 200 None 2026-03-09 16:56:04,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:04,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:04,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:04,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589146f7ae8ff3365632 HTTP/1.1" 200 None 2026-03-09 16:56:04,857 - lava_callback - ERROR - Error processing callback for node 69ac589146f7ae8ff3365632: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:05,444 - lava_callback - INFO - Received callback for node 69ac589246f7ae8ff3365635 INFO: 172.21.0.1:44634 - "POST /node/69ac589246f7ae8ff3365635 HTTP/1.0" 202 Accepted 2026-03-09 16:56:05,450 - lava_callback - INFO - Starting async job submit for node 69ac589246f7ae8ff3365635 2026-03-09 16:56:05,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:05,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589246f7ae8ff3365635 HTTP/1.1" 200 None 2026-03-09 16:56:05,466 - lava_callback - DEBUG - Processing node 69ac589246f7ae8ff3365635: baseline-arm64-baylibre 2026-03-09 16:56:05,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589246f7ae8ff3365635 2026-03-09 16:56:05,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:05,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589246f7ae8ff3365635 HTTP/1.1" 200 None 2026-03-09 16:56:05,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:05,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:05,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:05,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589246f7ae8ff3365635 HTTP/1.1" 200 None 2026-03-09 16:56:05,519 - lava_callback - ERROR - Error processing callback for node 69ac589246f7ae8ff3365635: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:06,113 - lava_callback - INFO - Received callback for node 69ac589446f7ae8ff3365645 INFO: 172.21.0.1:44640 - "POST /node/69ac589446f7ae8ff3365645 HTTP/1.0" 202 Accepted 2026-03-09 16:56:06,119 - lava_callback - INFO - Starting async job submit for node 69ac589446f7ae8ff3365645 2026-03-09 16:56:06,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589446f7ae8ff3365645 HTTP/1.1" 200 None 2026-03-09 16:56:06,141 - lava_callback - DEBUG - Processing node 69ac589446f7ae8ff3365645: baseline-arm64-baylibre 2026-03-09 16:56:06,141 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589446f7ae8ff3365645 2026-03-09 16:56:06,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589446f7ae8ff3365645 HTTP/1.1" 200 None 2026-03-09 16:56:06,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:06,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589446f7ae8ff3365645 HTTP/1.1" 200 None 2026-03-09 16:56:06,195 - lava_callback - ERROR - Error processing callback for node 69ac589446f7ae8ff3365645: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44652 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:56:06,787 - lava_callback - INFO - Received callback for node 69ac589646f7ae8ff3365668 INFO: 172.21.0.1:44666 - "POST /node/69ac589646f7ae8ff3365668 HTTP/1.0" 202 Accepted 2026-03-09 16:56:06,794 - lava_callback - INFO - Starting async job submit for node 69ac589646f7ae8ff3365668 2026-03-09 16:56:06,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589646f7ae8ff3365668 HTTP/1.1" 200 None 2026-03-09 16:56:06,824 - lava_callback - DEBUG - Processing node 69ac589646f7ae8ff3365668: baseline-arm64-baylibre 2026-03-09 16:56:06,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589646f7ae8ff3365668 2026-03-09 16:56:06,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589646f7ae8ff3365668 HTTP/1.1" 200 None 2026-03-09 16:56:06,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:06,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589646f7ae8ff3365668 HTTP/1.1" 200 None 2026-03-09 16:56:06,885 - lava_callback - ERROR - Error processing callback for node 69ac589646f7ae8ff3365668: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:07,433 - lava_callback - INFO - Received callback for node 69ac589846f7ae8ff336566d INFO: 172.21.0.1:44672 - "POST /node/69ac589846f7ae8ff336566d HTTP/1.0" 202 Accepted 2026-03-09 16:56:07,442 - lava_callback - INFO - Starting async job submit for node 69ac589846f7ae8ff336566d 2026-03-09 16:56:07,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:07,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589846f7ae8ff336566d HTTP/1.1" 200 None 2026-03-09 16:56:07,472 - lava_callback - DEBUG - Processing node 69ac589846f7ae8ff336566d: baseline-arm64-baylibre 2026-03-09 16:56:07,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589846f7ae8ff336566d 2026-03-09 16:56:07,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:07,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589846f7ae8ff336566d HTTP/1.1" 200 None 2026-03-09 16:56:07,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:07,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:07,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:07,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589846f7ae8ff336566d HTTP/1.1" 200 None 2026-03-09 16:56:07,547 - lava_callback - ERROR - Error processing callback for node 69ac589846f7ae8ff336566d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:08,099 - lava_callback - INFO - Received callback for node 69ac589946f7ae8ff3365674 INFO: 172.21.0.1:44682 - "POST /node/69ac589946f7ae8ff3365674 HTTP/1.0" 202 Accepted 2026-03-09 16:56:08,108 - lava_callback - INFO - Starting async job submit for node 69ac589946f7ae8ff3365674 2026-03-09 16:56:08,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:08,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589946f7ae8ff3365674 HTTP/1.1" 200 None 2026-03-09 16:56:08,143 - lava_callback - DEBUG - Processing node 69ac589946f7ae8ff3365674: baseline-arm64-baylibre 2026-03-09 16:56:08,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589946f7ae8ff3365674 2026-03-09 16:56:08,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:08,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589946f7ae8ff3365674 HTTP/1.1" 200 None 2026-03-09 16:56:08,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:08,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:08,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:08,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589946f7ae8ff3365674 HTTP/1.1" 200 None 2026-03-09 16:56:08,219 - lava_callback - ERROR - Error processing callback for node 69ac589946f7ae8ff3365674: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57778 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:56:17,836 - lava_callback - INFO - Received callback for node 69ac58ab46f7ae8ff33656ab INFO: 172.21.0.1:57780 - "POST /node/69ac58ab46f7ae8ff33656ab HTTP/1.0" 202 Accepted 2026-03-09 16:56:17,845 - lava_callback - INFO - Starting async job submit for node 69ac58ab46f7ae8ff33656ab 2026-03-09 16:56:17,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:17,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac58ab46f7ae8ff33656ab HTTP/1.1" 200 None 2026-03-09 16:56:17,873 - lava_callback - DEBUG - Processing node 69ac58ab46f7ae8ff33656ab: baseline-arm64-clabbe 2026-03-09 16:56:17,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac58ab46f7ae8ff33656ab 2026-03-09 16:56:17,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:17,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac58ab46f7ae8ff33656ab HTTP/1.1" 200 None 2026-03-09 16:56:17,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:17,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:17,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:17,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac58ab46f7ae8ff33656ab HTTP/1.1" 200 None 2026-03-09 16:56:17,976 - lava_callback - ERROR - Error processing callback for node 69ac58ab46f7ae8ff33656ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:18,524 - lava_callback - INFO - Received callback for node 69aefa18ce46d8ec186abf0a INFO: 172.21.0.1:57796 - "POST /node/69aefa18ce46d8ec186abf0a HTTP/1.0" 202 Accepted 2026-03-09 16:56:18,533 - lava_callback - INFO - Starting async job submit for node 69aefa18ce46d8ec186abf0a 2026-03-09 16:56:18,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:18,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa18ce46d8ec186abf0a HTTP/1.1" 200 None 2026-03-09 16:56:18,593 - lava_callback - DEBUG - Processing node 69aefa18ce46d8ec186abf0a: baseline-arm-clabbe 2026-03-09 16:56:18,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:56:19,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:56:19,079 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa18ce46d8ec186abf0a/log.txt.gz 2026-03-09 16:56:19,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:56:19,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:56:19,719 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa18ce46d8ec186abf0a/lava-logs.yaml 2026-03-09 16:56:19,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:56:19,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:56:19,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa18ce46d8ec186abf0a/lava_callback.json.gz 2026-03-09 16:56:19,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:19,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa18ce46d8ec186abf0a HTTP/1.1" 200 None 2026-03-09 16:56:19,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:19,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:56:19,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:19,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa18ce46d8ec186abf0a HTTP/1.1" 200 None 2026-03-09 16:56:19,959 - lava_callback - INFO - Completed processing callback for node 69aefa18ce46d8ec186abf0a INFO: 172.21.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58860 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:56:37,868 - lava_callback - INFO - Received callback for node 69ac58b246f7ae8ff33656c2 INFO: 172.21.0.1:58862 - "POST /node/69ac58b246f7ae8ff33656c2 HTTP/1.0" 202 Accepted 2026-03-09 16:56:37,874 - lava_callback - INFO - Starting async job submit for node 69ac58b246f7ae8ff33656c2 2026-03-09 16:56:37,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:37,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac58b246f7ae8ff33656c2 HTTP/1.1" 200 None 2026-03-09 16:56:37,898 - lava_callback - DEBUG - Processing node 69ac58b246f7ae8ff33656c2: baseline-arm64-clabbe 2026-03-09 16:56:37,898 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac58b246f7ae8ff33656c2 2026-03-09 16:56:37,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:37,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac58b246f7ae8ff33656c2 HTTP/1.1" 200 None 2026-03-09 16:56:37,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:37,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:37,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:37,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac58b246f7ae8ff33656c2 HTTP/1.1" 200 None 2026-03-09 16:56:37,962 - lava_callback - ERROR - Error processing callback for node 69ac58b246f7ae8ff33656c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:44,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:44,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36670 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:57:22,579 - lava_callback - INFO - Received callback for node 69aef767ce46d8ec186abbfd INFO: 172.21.0.1:37972 - "POST /node/69aef767ce46d8ec186abbfd HTTP/1.0" 202 Accepted 2026-03-09 16:57:22,586 - lava_callback - INFO - Starting async job submit for node 69aef767ce46d8ec186abbfd 2026-03-09 16:57:22,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:57:22,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef767ce46d8ec186abbfd HTTP/1.1" 200 None 2026-03-09 16:57:22,660 - lava_callback - DEBUG - Processing node 69aef767ce46d8ec186abbfd: ltp-smoke 2026-03-09 16:57:23,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:57:23,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:57:23,329 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69aef767ce46d8ec186abbfd/log.txt.gz 2026-03-09 16:57:23,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:57:23,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:57:23,855 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69aef767ce46d8ec186abbfd/lava-logs.yaml 2026-03-09 16:57:23,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:57:24,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:57:24,033 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69aef767ce46d8ec186abbfd/lava_callback.json.gz 2026-03-09 16:57:24,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:57:24,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef767ce46d8ec186abbfd HTTP/1.1" 200 None 2026-03-09 16:57:24,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:57:24,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef42ace46d8ec186ab814 HTTP/1.1" 200 None 2026-03-09 16:57:24,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:57:24,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef767ce46d8ec186abbfd HTTP/1.1" 200 None 2026-03-09 16:57:24,340 - lava_callback - INFO - Completed processing callback for node 69aef767ce46d8ec186abbfd INFO: 172.21.0.1:37980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40106 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:57:44,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:57:44,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52708 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:00:38,146 - lava_callback - INFO - Received callback for node 69aefc65ce46d8ec186ac289 INFO: 172.21.0.1:52724 - "POST /node/69aefc65ce46d8ec186ac289 HTTP/1.0" 202 Accepted 2026-03-09 17:00:38,156 - lava_callback - INFO - Starting async job submit for node 69aefc65ce46d8ec186ac289 2026-03-09 17:00:38,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:00:38,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc65ce46d8ec186ac289 HTTP/1.1" 200 None 2026-03-09 17:00:38,197 - lava_callback - DEBUG - Processing node 69aefc65ce46d8ec186ac289: baseline-arm-pengutronix 2026-03-09 17:00:38,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:00:38,562 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:00:38,564 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefc65ce46d8ec186ac289/log.txt.gz 2026-03-09 17:00:38,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:00:38,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:00:38,874 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefc65ce46d8ec186ac289/lava-logs.yaml 2026-03-09 17:00:38,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:00:39,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:00:39,041 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefc65ce46d8ec186ac289/lava_callback.json.gz 2026-03-09 17:00:39,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:00:39,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc65ce46d8ec186ac289 HTTP/1.1" 200 None 2026-03-09 17:00:39,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:00:39,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:00:39,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:00:39,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefc65ce46d8ec186ac289 HTTP/1.1" 200 None 2026-03-09 17:00:39,145 - lava_callback - INFO - Completed processing callback for node 69aefc65ce46d8ec186ac289 2026-03-09 17:00:44,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:00:44,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50918 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:01:28,723 - lava_callback - INFO - Received callback for node 69aefc89ce46d8ec186ac2a5 INFO: 172.21.0.1:50932 - "POST /node/69aefc89ce46d8ec186ac2a5 HTTP/1.0" 202 Accepted 2026-03-09 17:01:28,729 - lava_callback - INFO - Starting async job submit for node 69aefc89ce46d8ec186ac2a5 2026-03-09 17:01:28,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:28,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc89ce46d8ec186ac2a5 HTTP/1.1" 200 None 2026-03-09 17:01:28,767 - lava_callback - DEBUG - Processing node 69aefc89ce46d8ec186ac2a5: baseline-arm-broonie 2026-03-09 17:01:28,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:01:29,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:01:29,120 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefc89ce46d8ec186ac2a5/log.txt.gz 2026-03-09 17:01:29,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:01:29,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:01:29,450 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefc89ce46d8ec186ac2a5/lava-logs.yaml 2026-03-09 17:01:29,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:01:29,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:01:29,599 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefc89ce46d8ec186ac2a5/lava_callback.json.gz 2026-03-09 17:01:29,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:29,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc89ce46d8ec186ac2a5 HTTP/1.1" 200 None 2026-03-09 17:01:29,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:29,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:01:29,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:29,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefc89ce46d8ec186ac2a5 HTTP/1.1" 200 None 2026-03-09 17:01:29,688 - lava_callback - INFO - Completed processing callback for node 69aefc89ce46d8ec186ac2a5 INFO: 172.21.0.1:44718 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:01:44,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:44,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58528 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:01:57,531 - lava_callback - INFO - Received callback for node 69aefcf8ce46d8ec186ac33c INFO: 172.21.0.1:58534 - "POST /node/69aefcf8ce46d8ec186ac33c HTTP/1.0" 202 Accepted 2026-03-09 17:01:57,537 - lava_callback - INFO - Starting async job submit for node 69aefcf8ce46d8ec186ac33c 2026-03-09 17:01:57,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:57,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefcf8ce46d8ec186ac33c HTTP/1.1" 200 None 2026-03-09 17:01:57,555 - lava_callback - DEBUG - Processing node 69aefcf8ce46d8ec186ac33c: baseline-arm 2026-03-09 17:01:57,560 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:01:57,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:01:57,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:01:57,702 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aefcf8ce46d8ec186ac33c/lava-logs.yaml 2026-03-09 17:01:57,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:01:57,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:01:57,859 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aefcf8ce46d8ec186ac33c/lava_callback.json.gz 2026-03-09 17:01:57,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:57,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefcf8ce46d8ec186ac33c HTTP/1.1" 200 None 2026-03-09 17:01:57,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:57,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:01:57,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:57,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefcf8ce46d8ec186ac33c HTTP/1.1" 200 None 2026-03-09 17:01:57,924 - lava_callback - INFO - Completed processing callback for node 69aefcf8ce46d8ec186ac33c INFO: 172.21.0.1:56352 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:02:14,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:02:14,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42530 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:05:25,243 - lava_callback - INFO - Received callback for node 69ac5ac946f7ae8ff336590e INFO: 172.21.0.1:56880 - "POST /node/69ac5ac946f7ae8ff336590e HTTP/1.0" 202 Accepted 2026-03-09 17:05:25,252 - lava_callback - INFO - Starting async job submit for node 69ac5ac946f7ae8ff336590e 2026-03-09 17:05:25,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:05:25,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5ac946f7ae8ff336590e HTTP/1.1" 200 None 2026-03-09 17:05:25,268 - lava_callback - DEBUG - Processing node 69ac5ac946f7ae8ff336590e: baseline-arm64-baylibre 2026-03-09 17:05:25,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5ac946f7ae8ff336590e 2026-03-09 17:05:25,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:05:25,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5ac946f7ae8ff336590e HTTP/1.1" 200 None 2026-03-09 17:05:25,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:05:25,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a64e31b554803095c1a HTTP/1.1" 200 None 2026-03-09 17:05:25,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:05:25,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5ac946f7ae8ff336590e HTTP/1.1" 200 None 2026-03-09 17:05:25,365 - lava_callback - ERROR - Error processing callback for node 69ac5ac946f7ae8ff336590e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:06:06,930 - lava_callback - INFO - Received callback for node 69aef9fece46d8ec186abe94 INFO: 172.21.0.1:44030 - "POST /node/69aef9fece46d8ec186abe94 HTTP/1.0" 202 Accepted 2026-03-09 17:06:06,937 - lava_callback - INFO - Starting async job submit for node 69aef9fece46d8ec186abe94 2026-03-09 17:06:06,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:06:07,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fece46d8ec186abe94 HTTP/1.1" 200 None 2026-03-09 17:06:07,011 - lava_callback - DEBUG - Processing node 69aef9fece46d8ec186abe94: baseline-arm 2026-03-09 17:06:07,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:06:07,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:06:07,967 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fece46d8ec186abe94/log.txt.gz 2026-03-09 17:06:08,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:06:08,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:06:08,696 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fece46d8ec186abe94/lava-logs.yaml 2026-03-09 17:06:08,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:06:08,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:06:08,923 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fece46d8ec186abe94/lava_callback.json.gz DEBUG: Setup failed for 69aef9fece46d8ec186abe94. Transitting job node result: incomplete -> fail 2026-03-09 17:06:09,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:06:09,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fece46d8ec186abe94 HTTP/1.1" 200 None 2026-03-09 17:06:09,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:06:09,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:06:09,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:06:09,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef9fece46d8ec186abe94 HTTP/1.1" 200 None 2026-03-09 17:06:09,283 - lava_callback - INFO - Completed processing callback for node 69aef9fece46d8ec186abe94 2026-03-09 17:06:14,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:06:14,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40398 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:07:45,324 - lava_callback - INFO - Received callback for node 69ac5b5c46f7ae8ff3365944 INFO: 172.21.0.1:50242 - "POST /node/69ac5b5c46f7ae8ff3365944 HTTP/1.0" 202 Accepted 2026-03-09 17:07:45,335 - lava_callback - INFO - Starting async job submit for node 69ac5b5c46f7ae8ff3365944 2026-03-09 17:07:45,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:07:45,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5b5c46f7ae8ff3365944 HTTP/1.1" 200 None 2026-03-09 17:07:45,368 - lava_callback - DEBUG - Processing node 69ac5b5c46f7ae8ff3365944: baseline-arm64-baylibre 2026-03-09 17:07:45,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5b5c46f7ae8ff3365944 2026-03-09 17:07:45,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:07:45,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5b5c46f7ae8ff3365944 HTTP/1.1" 200 None 2026-03-09 17:07:45,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:07:45,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a64e31b554803095c1a HTTP/1.1" 200 None 2026-03-09 17:07:45,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:07:45,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5b5c46f7ae8ff3365944 HTTP/1.1" 200 None 2026-03-09 17:07:45,596 - lava_callback - ERROR - Error processing callback for node 69ac5b5c46f7ae8ff3365944: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60556 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:12:05,595 - lava_callback - INFO - Received callback for node 69ac5c5e46f7ae8ff33659a2 INFO: 172.21.0.1:60896 - "POST /node/69ac5c5e46f7ae8ff33659a2 HTTP/1.0" 202 Accepted 2026-03-09 17:12:05,601 - lava_callback - INFO - Starting async job submit for node 69ac5c5e46f7ae8ff33659a2 2026-03-09 17:12:05,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:12:05,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5c5e46f7ae8ff33659a2 HTTP/1.1" 200 None 2026-03-09 17:12:05,644 - lava_callback - DEBUG - Processing node 69ac5c5e46f7ae8ff33659a2: baseline-arm64-baylibre 2026-03-09 17:12:05,644 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5c5e46f7ae8ff33659a2 2026-03-09 17:12:05,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:12:05,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5c5e46f7ae8ff33659a2 HTTP/1.1" 200 None 2026-03-09 17:12:05,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:12:05,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 17:12:05,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:12:05,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5c5e46f7ae8ff33659a2 HTTP/1.1" 200 None 2026-03-09 17:12:05,719 - lava_callback - ERROR - Error processing callback for node 69ac5c5e46f7ae8ff33659a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35200 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:13:25,652 - lava_callback - INFO - Received callback for node 69ac5caa46f7ae8ff33659ae INFO: 172.21.0.1:46330 - "POST /node/69ac5caa46f7ae8ff33659ae HTTP/1.0" 202 Accepted 2026-03-09 17:13:25,658 - lava_callback - INFO - Starting async job submit for node 69ac5caa46f7ae8ff33659ae 2026-03-09 17:13:25,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:13:25,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5caa46f7ae8ff33659ae HTTP/1.1" 200 None 2026-03-09 17:13:25,676 - lava_callback - DEBUG - Processing node 69ac5caa46f7ae8ff33659ae: baseline-arm64-baylibre 2026-03-09 17:13:25,676 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5caa46f7ae8ff33659ae 2026-03-09 17:13:25,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:13:25,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5caa46f7ae8ff33659ae HTTP/1.1" 200 None 2026-03-09 17:13:25,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:13:25,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 17:13:25,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:13:25,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5caa46f7ae8ff33659ae HTTP/1.1" 200 None 2026-03-09 17:13:25,732 - lava_callback - ERROR - Error processing callback for node 69ac5caa46f7ae8ff33659ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54232 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:11,717 - lava_callback - INFO - Received callback for node 69aeffb8ce46d8ec186ac440 INFO: 172.21.0.1:41512 - "POST /node/69aeffb8ce46d8ec186ac440 HTTP/1.0" 202 Accepted 2026-03-09 17:14:11,723 - lava_callback - INFO - Starting async job submit for node 69aeffb8ce46d8ec186ac440 2026-03-09 17:14:11,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:11,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb8ce46d8ec186ac440 HTTP/1.1" 200 None 2026-03-09 17:14:11,758 - lava_callback - DEBUG - Processing node 69aeffb8ce46d8ec186ac440: baseline-x86 2026-03-09 17:14:11,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:12,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:12,099 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac440/log.txt.gz 2026-03-09 17:14:12,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:12,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:12,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac440/lava-logs.yaml 2026-03-09 17:14:12,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:12,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:12,590 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac440/lava_callback.json.gz 2026-03-09 17:14:12,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:12,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb8ce46d8ec186ac440 HTTP/1.1" 200 None 2026-03-09 17:14:12,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:12,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef449ce46d8ec186ab83b HTTP/1.1" 200 None 2026-03-09 17:14:12,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:12,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aeffb8ce46d8ec186ac440 HTTP/1.1" 200 None 2026-03-09 17:14:12,672 - lava_callback - INFO - Completed processing callback for node 69aeffb8ce46d8ec186ac440 2026-03-09 17:14:14,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:14,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41522 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:14:16,090 - lava_callback - INFO - Received callback for node 69aeffb6ce46d8ec186ac435 INFO: 172.21.0.1:41528 - "POST /node/69aeffb6ce46d8ec186ac435 HTTP/1.0" 202 Accepted 2026-03-09 17:14:16,096 - lava_callback - INFO - Starting async job submit for node 69aeffb6ce46d8ec186ac435 2026-03-09 17:14:16,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:16,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb6ce46d8ec186ac435 HTTP/1.1" 200 None 2026-03-09 17:14:16,129 - lava_callback - DEBUG - Processing node 69aeffb6ce46d8ec186ac435: baseline-x86 2026-03-09 17:14:16,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:16,454 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:16,456 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb6ce46d8ec186ac435/log.txt.gz INFO: 172.21.0.1:41544 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:16,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:16,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:16,751 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb6ce46d8ec186ac435/lava-logs.yaml 2026-03-09 17:14:16,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:16,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:16,913 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb6ce46d8ec186ac435/lava_callback.json.gz 2026-03-09 17:14:16,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:16,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb6ce46d8ec186ac435 HTTP/1.1" 200 None 2026-03-09 17:14:16,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:16,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef449ce46d8ec186ab83b HTTP/1.1" 200 None 2026-03-09 17:14:16,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:17,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aeffb6ce46d8ec186ac435 HTTP/1.1" 200 None 2026-03-09 17:14:17,004 - lava_callback - INFO - Completed processing callback for node 69aeffb6ce46d8ec186ac435 INFO: 172.21.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33004 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:40,263 - lava_callback - INFO - Received callback for node 69aeffb7ce46d8ec186ac438 INFO: 172.21.0.1:36044 - "POST /node/69aeffb7ce46d8ec186ac438 HTTP/1.0" 202 Accepted 2026-03-09 17:14:40,269 - lava_callback - INFO - Starting async job submit for node 69aeffb7ce46d8ec186ac438 2026-03-09 17:14:40,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:40,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb7ce46d8ec186ac438 HTTP/1.1" 200 None 2026-03-09 17:14:40,303 - lava_callback - DEBUG - Processing node 69aeffb7ce46d8ec186ac438: baseline-x86 2026-03-09 17:14:40,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:40,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:40,725 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb7ce46d8ec186ac438/log.txt.gz 2026-03-09 17:14:40,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:41,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:41,145 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb7ce46d8ec186ac438/lava-logs.yaml 2026-03-09 17:14:41,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:41,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:41,297 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb7ce46d8ec186ac438/lava_callback.json.gz 2026-03-09 17:14:41,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:41,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb7ce46d8ec186ac438 HTTP/1.1" 200 None 2026-03-09 17:14:41,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:41,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef449ce46d8ec186ab83b HTTP/1.1" 200 None 2026-03-09 17:14:41,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:41,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aeffb7ce46d8ec186ac438 HTTP/1.1" 200 None 2026-03-09 17:14:41,381 - lava_callback - INFO - Completed processing callback for node 69aeffb7ce46d8ec186ac438 2026-03-09 17:14:44,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:44,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 17:14:45,708 - lava_callback - INFO - Received callback for node 69ac5cf446f7ae8ff33659cc INFO: 172.21.0.1:36048 - "POST /node/69ac5cf446f7ae8ff33659cc HTTP/1.0" 202 Accepted 2026-03-09 17:14:45,713 - lava_callback - INFO - Starting async job submit for node 69ac5cf446f7ae8ff33659cc 2026-03-09 17:14:45,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:45,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5cf446f7ae8ff33659cc HTTP/1.1" 200 None 2026-03-09 17:14:45,731 - lava_callback - DEBUG - Processing node 69ac5cf446f7ae8ff33659cc: baseline-arm64-baylibre 2026-03-09 17:14:45,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5cf446f7ae8ff33659cc 2026-03-09 17:14:45,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:45,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5cf446f7ae8ff33659cc HTTP/1.1" 200 None 2026-03-09 17:14:45,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:45,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 17:14:45,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:45,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5cf446f7ae8ff33659cc HTTP/1.1" 200 None 2026-03-09 17:14:45,804 - lava_callback - ERROR - Error processing callback for node 69ac5cf446f7ae8ff33659cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36058 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:56,030 - lava_callback - INFO - Received callback for node 69aeffb8ce46d8ec186ac43b INFO: 172.21.0.1:52418 - "POST /node/69aeffb8ce46d8ec186ac43b HTTP/1.0" 202 Accepted 2026-03-09 17:14:56,036 - lava_callback - INFO - Starting async job submit for node 69aeffb8ce46d8ec186ac43b 2026-03-09 17:14:56,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:56,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb8ce46d8ec186ac43b HTTP/1.1" 200 None 2026-03-09 17:14:56,070 - lava_callback - DEBUG - Processing node 69aeffb8ce46d8ec186ac43b: baseline-x86 INFO: 172.21.0.1:52428 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:56,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:56,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:56,468 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac43b/log.txt.gz INFO: 172.21.0.1:52434 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:56,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:56,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:56,798 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac43b/lava-logs.yaml 2026-03-09 17:14:56,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:56,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:56,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac43b/lava_callback.json.gz 2026-03-09 17:14:56,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:56,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb8ce46d8ec186ac43b HTTP/1.1" 200 None 2026-03-09 17:14:56,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:57,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef449ce46d8ec186ab83b HTTP/1.1" 200 None 2026-03-09 17:14:57,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:57,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aeffb8ce46d8ec186ac43b HTTP/1.1" 200 None 2026-03-09 17:14:57,037 - lava_callback - INFO - Completed processing callback for node 69aeffb8ce46d8ec186ac43b INFO: 172.21.0.1:41534 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:15:14,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:15:14,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52370 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:15:45,732 - lava_callback - INFO - Received callback for node 69ac5d3f46f7ae8ff33659e3 INFO: 172.21.0.1:51174 - "POST /node/69ac5d3f46f7ae8ff33659e3 HTTP/1.0" 202 Accepted 2026-03-09 17:15:45,737 - lava_callback - INFO - Starting async job submit for node 69ac5d3f46f7ae8ff33659e3 2026-03-09 17:15:45,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:15:45,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5d3f46f7ae8ff33659e3 HTTP/1.1" 200 None 2026-03-09 17:15:45,755 - lava_callback - DEBUG - Processing node 69ac5d3f46f7ae8ff33659e3: baseline-arm64-baylibre 2026-03-09 17:15:45,755 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5d3f46f7ae8ff33659e3 2026-03-09 17:15:45,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:15:45,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5d3f46f7ae8ff33659e3 HTTP/1.1" 200 None 2026-03-09 17:15:45,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:15:45,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 17:15:45,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:15:45,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5d3f46f7ae8ff33659e3 HTTP/1.1" 200 None 2026-03-09 17:15:45,813 - lava_callback - ERROR - Error processing callback for node 69ac5d3f46f7ae8ff33659e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59694 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:17:18,500 - lava_callback - INFO - Received callback for node 69af008fce46d8ec186ac515 INFO: 172.21.0.1:59704 - "POST /node/69af008fce46d8ec186ac515 HTTP/1.0" 202 Accepted 2026-03-09 17:17:18,509 - lava_callback - INFO - Starting async job submit for node 69af008fce46d8ec186ac515 2026-03-09 17:17:18,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af008fce46d8ec186ac515 HTTP/1.1" 200 None 2026-03-09 17:17:18,528 - lava_callback - DEBUG - Processing node 69af008fce46d8ec186ac515: baseline-arm 2026-03-09 17:17:18,533 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:17:18,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:17:18,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:17:18,713 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af008fce46d8ec186ac515/lava-logs.yaml 2026-03-09 17:17:18,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:17:18,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:17:18,849 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af008fce46d8ec186ac515/lava_callback.json.gz 2026-03-09 17:17:18,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,859 - lava_callback - INFO - Received callback for node 69ac5d8e46f7ae8ff33659f9 INFO: 172.21.0.1:59716 - "POST /node/69ac5d8e46f7ae8ff33659f9 HTTP/1.0" 202 Accepted 2026-03-09 17:17:18,867 - lava_callback - INFO - Starting async job submit for node 69ac5d8e46f7ae8ff33659f9 2026-03-09 17:17:18,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af008fce46d8ec186ac515 HTTP/1.1" 200 None 2026-03-09 17:17:18,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5d8e46f7ae8ff33659f9 HTTP/1.1" 200 None 2026-03-09 17:17:18,884 - lava_callback - DEBUG - Processing node 69ac5d8e46f7ae8ff33659f9: baseline-arm64-clabbe 2026-03-09 17:17:18,884 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5d8e46f7ae8ff33659f9 2026-03-09 17:17:18,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:17:18,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5d8e46f7ae8ff33659f9 HTTP/1.1" 200 None 2026-03-09 17:17:18,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af008fce46d8ec186ac515 HTTP/1.1" 200 None 2026-03-09 17:17:18,930 - lava_callback - INFO - Completed processing callback for node 69af008fce46d8ec186ac515 2026-03-09 17:17:18,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 17:17:18,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:19,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5d8e46f7ae8ff33659f9 HTTP/1.1" 200 None 2026-03-09 17:17:19,009 - lava_callback - ERROR - Error processing callback for node 69ac5d8e46f7ae8ff33659f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37910 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:17:44,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:44,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44672 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:21:32,721 - lava_callback - INFO - Received callback for node 69af018fce46d8ec186ac613 INFO: 172.21.0.1:38332 - "POST /node/69af018fce46d8ec186ac613 HTTP/1.0" 202 Accepted 2026-03-09 17:21:32,727 - lava_callback - INFO - Starting async job submit for node 69af018fce46d8ec186ac613 2026-03-09 17:21:32,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:32,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018fce46d8ec186ac613 HTTP/1.1" 200 None 2026-03-09 17:21:32,747 - lava_callback - DEBUG - Processing node 69af018fce46d8ec186ac613: baseline-arm64 2026-03-09 17:21:32,751 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:21:32,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:32,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:32,921 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018fce46d8ec186ac613/lava-logs.yaml 2026-03-09 17:21:32,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:33,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:33,072 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018fce46d8ec186ac613/lava_callback.json.gz 2026-03-09 17:21:33,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:33,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018fce46d8ec186ac613 HTTP/1.1" 200 None 2026-03-09 17:21:33,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:33,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:21:33,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:33,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af018fce46d8ec186ac613 HTTP/1.1" 200 None 2026-03-09 17:21:33,160 - lava_callback - INFO - Completed processing callback for node 69af018fce46d8ec186ac613 2026-03-09 17:21:34,890 - lava_callback - INFO - Received callback for node 69af018fce46d8ec186ac610 INFO: 172.21.0.1:38338 - "POST /node/69af018fce46d8ec186ac610 HTTP/1.0" 202 Accepted 2026-03-09 17:21:34,897 - lava_callback - INFO - Starting async job submit for node 69af018fce46d8ec186ac610 2026-03-09 17:21:34,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:34,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018fce46d8ec186ac610 HTTP/1.1" 200 None 2026-03-09 17:21:34,919 - lava_callback - DEBUG - Processing node 69af018fce46d8ec186ac610: baseline-arm64 2026-03-09 17:21:34,923 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:21:34,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:35,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:35,049 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018fce46d8ec186ac610/lava-logs.yaml 2026-03-09 17:21:35,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:35,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:35,175 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018fce46d8ec186ac610/lava_callback.json.gz 2026-03-09 17:21:35,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:35,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018fce46d8ec186ac610 HTTP/1.1" 200 None 2026-03-09 17:21:35,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:35,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:21:35,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:35,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af018fce46d8ec186ac610 HTTP/1.1" 200 None 2026-03-09 17:21:35,245 - lava_callback - INFO - Completed processing callback for node 69af018fce46d8ec186ac610 INFO: 172.21.0.1:38354 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:21:44,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:44,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:21:57,949 - lava_callback - INFO - Received callback for node 69af019bce46d8ec186ac631 INFO: 172.21.0.1:45814 - "POST /node/69af019bce46d8ec186ac631 HTTP/1.0" 202 Accepted 2026-03-09 17:21:57,959 - lava_callback - INFO - Starting async job submit for node 69af019bce46d8ec186ac631 2026-03-09 17:21:57,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:57,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019bce46d8ec186ac631 HTTP/1.1" 200 None 2026-03-09 17:21:57,989 - lava_callback - DEBUG - Processing node 69af019bce46d8ec186ac631: baseline-arm64 2026-03-09 17:21:57,998 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:21:58,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:58,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:58,227 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019bce46d8ec186ac631/lava-logs.yaml 2026-03-09 17:21:58,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:58,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:58,395 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019bce46d8ec186ac631/lava_callback.json.gz 2026-03-09 17:21:58,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:58,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019bce46d8ec186ac631 HTTP/1.1" 200 None 2026-03-09 17:21:58,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:58,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:21:58,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:58,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af019bce46d8ec186ac631 HTTP/1.1" 200 None 2026-03-09 17:21:58,480 - lava_callback - INFO - Completed processing callback for node 69af019bce46d8ec186ac631 2026-03-09 17:22:01,147 - lava_callback - INFO - Received callback for node 69af018ece46d8ec186ac60d INFO: 172.21.0.1:51838 - "POST /node/69af018ece46d8ec186ac60d HTTP/1.0" 202 Accepted 2026-03-09 17:22:01,158 - lava_callback - INFO - Starting async job submit for node 69af018ece46d8ec186ac60d 2026-03-09 17:22:01,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:01,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018ece46d8ec186ac60d HTTP/1.1" 200 None 2026-03-09 17:22:01,223 - lava_callback - DEBUG - Processing node 69af018ece46d8ec186ac60d: baseline-arm64 2026-03-09 17:22:01,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:01,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:01,606 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018ece46d8ec186ac60d/log.txt.gz 2026-03-09 17:22:01,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:01,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:01,882 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018ece46d8ec186ac60d/lava-logs.yaml 2026-03-09 17:22:01,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:02,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:02,059 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018ece46d8ec186ac60d/lava_callback.json.gz 2026-03-09 17:22:02,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:02,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018ece46d8ec186ac60d HTTP/1.1" 200 None 2026-03-09 17:22:02,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:02,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:02,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:02,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af018ece46d8ec186ac60d HTTP/1.1" 200 None 2026-03-09 17:22:02,143 - lava_callback - INFO - Completed processing callback for node 69af018ece46d8ec186ac60d INFO: 172.21.0.1:51850 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:07,192 - lava_callback - INFO - Received callback for node 69af0193ce46d8ec186ac61c INFO: 172.21.0.1:51854 - "POST /node/69af0193ce46d8ec186ac61c HTTP/1.0" 202 Accepted 2026-03-09 17:22:07,199 - lava_callback - INFO - Starting async job submit for node 69af0193ce46d8ec186ac61c 2026-03-09 17:22:07,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:07,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0193ce46d8ec186ac61c HTTP/1.1" 200 None 2026-03-09 17:22:07,234 - lava_callback - DEBUG - Processing node 69af0193ce46d8ec186ac61c: baseline-arm64 2026-03-09 17:22:07,295 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:07,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:07,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:07,462 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0193ce46d8ec186ac61c/lava-logs.yaml 2026-03-09 17:22:07,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:07,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:07,583 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0193ce46d8ec186ac61c/lava_callback.json.gz 2026-03-09 17:22:07,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:07,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0193ce46d8ec186ac61c HTTP/1.1" 200 None 2026-03-09 17:22:07,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:07,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:07,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:07,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0193ce46d8ec186ac61c HTTP/1.1" 200 None 2026-03-09 17:22:07,789 - lava_callback - INFO - Completed processing callback for node 69af0193ce46d8ec186ac61c 2026-03-09 17:22:14,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:14,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 17:22:14,829 - lava_callback - INFO - Received callback for node 69af01aace46d8ec186ac673 INFO: 172.21.0.1:42660 - "POST /node/69af01aace46d8ec186ac673 HTTP/1.0" 202 Accepted 2026-03-09 17:22:14,835 - lava_callback - INFO - Starting async job submit for node 69af01aace46d8ec186ac673 2026-03-09 17:22:14,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:14,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01aace46d8ec186ac673 HTTP/1.1" 200 None 2026-03-09 17:22:14,853 - lava_callback - DEBUG - Processing node 69af01aace46d8ec186ac673: baseline-nfs-arm64 2026-03-09 17:22:14,855 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:14,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:15,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:15,039 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01aace46d8ec186ac673/lava-logs.yaml 2026-03-09 17:22:15,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:42672 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:22:15,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:15,177 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01aace46d8ec186ac673/lava_callback.json.gz 2026-03-09 17:22:15,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:15,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01aace46d8ec186ac673 HTTP/1.1" 200 None 2026-03-09 17:22:15,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:15,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:15,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:15,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01aace46d8ec186ac673 HTTP/1.1" 200 None 2026-03-09 17:22:15,245 - lava_callback - INFO - Completed processing callback for node 69af01aace46d8ec186ac673 2026-03-09 17:22:16,128 - lava_callback - INFO - Received callback for node 69af01a1ce46d8ec186ac65c INFO: 172.21.0.1:42676 - "POST /node/69af01a1ce46d8ec186ac65c HTTP/1.0" 202 Accepted 2026-03-09 17:22:16,135 - lava_callback - INFO - Starting async job submit for node 69af01a1ce46d8ec186ac65c 2026-03-09 17:22:16,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:16,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a1ce46d8ec186ac65c HTTP/1.1" 200 None 2026-03-09 17:22:16,168 - lava_callback - DEBUG - Processing node 69af01a1ce46d8ec186ac65c: baseline-arm64 2026-03-09 17:22:16,219 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:16,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:16,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:16,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af01a1ce46d8ec186ac65c/lava-logs.yaml 2026-03-09 17:22:16,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:42682 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:16,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:16,539 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af01a1ce46d8ec186ac65c/lava_callback.json.gz 2026-03-09 17:22:16,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:16,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a1ce46d8ec186ac65c HTTP/1.1" 200 None 2026-03-09 17:22:16,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:16,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:16,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:16,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a1ce46d8ec186ac65c HTTP/1.1" 200 None 2026-03-09 17:22:16,710 - lava_callback - INFO - Completed processing callback for node 69af01a1ce46d8ec186ac65c 2026-03-09 17:22:18,263 - lava_callback - INFO - Received callback for node 69af019ace46d8ec186ac62e INFO: 172.21.0.1:42688 - "POST /node/69af019ace46d8ec186ac62e HTTP/1.0" 202 Accepted 2026-03-09 17:22:18,270 - lava_callback - INFO - Starting async job submit for node 69af019ace46d8ec186ac62e 2026-03-09 17:22:18,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:18,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019ace46d8ec186ac62e HTTP/1.1" 200 None 2026-03-09 17:22:18,303 - lava_callback - DEBUG - Processing node 69af019ace46d8ec186ac62e: baseline-arm64 2026-03-09 17:22:18,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:18,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:18,617 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ace46d8ec186ac62e/log.txt.gz 2026-03-09 17:22:18,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:18,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:18,926 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ace46d8ec186ac62e/lava-logs.yaml 2026-03-09 17:22:18,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:19,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:19,140 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ace46d8ec186ac62e/lava_callback.json.gz 2026-03-09 17:22:19,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:19,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019ace46d8ec186ac62e HTTP/1.1" 200 None 2026-03-09 17:22:19,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:19,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:19,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:19,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af019ace46d8ec186ac62e HTTP/1.1" 200 None 2026-03-09 17:22:19,226 - lava_callback - INFO - Completed processing callback for node 69af019ace46d8ec186ac62e 2026-03-09 17:22:20,446 - lava_callback - INFO - Received callback for node 69af0191ce46d8ec186ac619 INFO: 172.21.0.1:42562 - "POST /node/69af0191ce46d8ec186ac619 HTTP/1.0" 202 Accepted 2026-03-09 17:22:20,456 - lava_callback - INFO - Starting async job submit for node 69af0191ce46d8ec186ac619 2026-03-09 17:22:20,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:20,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0191ce46d8ec186ac619 HTTP/1.1" 200 None 2026-03-09 17:22:20,492 - lava_callback - DEBUG - Processing node 69af0191ce46d8ec186ac619: baseline-arm64 2026-03-09 17:22:20,553 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:20,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:20,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:20,726 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0191ce46d8ec186ac619/lava-logs.yaml 2026-03-09 17:22:20,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:20,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:20,882 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0191ce46d8ec186ac619/lava_callback.json.gz 2026-03-09 17:22:20,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:20,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0191ce46d8ec186ac619 HTTP/1.1" 200 None 2026-03-09 17:22:20,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:20,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:20,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:20,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0191ce46d8ec186ac619 HTTP/1.1" 200 None 2026-03-09 17:22:21,023 - lava_callback - INFO - Completed processing callback for node 69af0191ce46d8ec186ac619 INFO: 172.21.0.1:42570 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:32,996 - lava_callback - INFO - Received callback for node 69af01abce46d8ec186ac676 INFO: 172.21.0.1:45484 - "POST /node/69af01abce46d8ec186ac676 HTTP/1.0" 202 Accepted 2026-03-09 17:22:33,002 - lava_callback - INFO - Starting async job submit for node 69af01abce46d8ec186ac676 2026-03-09 17:22:33,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:33,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01abce46d8ec186ac676 HTTP/1.1" 200 None 2026-03-09 17:22:33,029 - lava_callback - DEBUG - Processing node 69af01abce46d8ec186ac676: baseline-nfs-arm64 2026-03-09 17:22:33,035 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:33,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:33,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:33,158 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01abce46d8ec186ac676/lava-logs.yaml 2026-03-09 17:22:33,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:33,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:33,314 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01abce46d8ec186ac676/lava_callback.json.gz 2026-03-09 17:22:33,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:33,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01abce46d8ec186ac676 HTTP/1.1" 200 None 2026-03-09 17:22:33,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:33,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:33,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:33,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01abce46d8ec186ac676 HTTP/1.1" 200 None 2026-03-09 17:22:33,438 - lava_callback - INFO - Completed processing callback for node 69af01abce46d8ec186ac676 INFO: 172.21.0.1:45498 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:44,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:44,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36350 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:50,741 - lava_callback - INFO - Received callback for node 69af01dace46d8ec186ac715 INFO: 172.21.0.1:34958 - "POST /node/69af01dace46d8ec186ac715 HTTP/1.0" 202 Accepted 2026-03-09 17:22:50,747 - lava_callback - INFO - Starting async job submit for node 69af01dace46d8ec186ac715 2026-03-09 17:22:50,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:50,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01dace46d8ec186ac715 HTTP/1.1" 200 None 2026-03-09 17:22:50,794 - lava_callback - DEBUG - Processing node 69af01dace46d8ec186ac715: baseline-arm64-foundriesio 2026-03-09 17:22:50,846 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:50,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:51,056 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:51,058 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af01dace46d8ec186ac715/lava-logs.yaml 2026-03-09 17:22:51,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:51,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:51,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af01dace46d8ec186ac715/lava_callback.json.gz 2026-03-09 17:22:51,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:51,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01dace46d8ec186ac715 HTTP/1.1" 200 None 2026-03-09 17:22:51,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:51,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:51,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:51,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01dace46d8ec186ac715 HTTP/1.1" 200 None 2026-03-09 17:22:51,330 - lava_callback - INFO - Completed processing callback for node 69af01dace46d8ec186ac715 INFO: 172.21.0.1:34968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34984 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:57,169 - lava_callback - INFO - Received callback for node 69af01d1ce46d8ec186ac6fd INFO: 172.21.0.1:34988 - "POST /node/69af01d1ce46d8ec186ac6fd HTTP/1.0" 202 Accepted 2026-03-09 17:22:57,174 - lava_callback - INFO - Starting async job submit for node 69af01d1ce46d8ec186ac6fd 2026-03-09 17:22:57,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:57,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d1ce46d8ec186ac6fd HTTP/1.1" 200 None 2026-03-09 17:22:57,192 - lava_callback - DEBUG - Processing node 69af01d1ce46d8ec186ac6fd: baseline-arm64-clabbe 2026-03-09 17:22:57,197 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:57,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:57,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:57,327 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af01d1ce46d8ec186ac6fd/lava-logs.yaml 2026-03-09 17:22:57,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:57,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:57,600 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af01d1ce46d8ec186ac6fd/lava_callback.json.gz 2026-03-09 17:22:57,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:57,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d1ce46d8ec186ac6fd HTTP/1.1" 200 None 2026-03-09 17:22:57,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:57,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:57,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:57,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01d1ce46d8ec186ac6fd HTTP/1.1" 200 None 2026-03-09 17:22:57,689 - lava_callback - INFO - Completed processing callback for node 69af01d1ce46d8ec186ac6fd 2026-03-09 17:23:03,103 - lava_callback - INFO - Received callback for node 69af01ccce46d8ec186ac6ec INFO: 172.21.0.1:51706 - "POST /node/69af01ccce46d8ec186ac6ec HTTP/1.0" 202 Accepted 2026-03-09 17:23:03,111 - lava_callback - INFO - Starting async job submit for node 69af01ccce46d8ec186ac6ec 2026-03-09 17:23:03,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:03,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01ccce46d8ec186ac6ec HTTP/1.1" 200 None 2026-03-09 17:23:03,137 - lava_callback - DEBUG - Processing node 69af01ccce46d8ec186ac6ec: baseline-arm64-broonie 2026-03-09 17:23:03,146 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:23:03,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:03,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:03,286 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01ccce46d8ec186ac6ec/lava-logs.yaml 2026-03-09 17:23:03,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:03,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:03,696 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01ccce46d8ec186ac6ec/lava_callback.json.gz 2026-03-09 17:23:03,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:03,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01ccce46d8ec186ac6ec HTTP/1.1" 200 None 2026-03-09 17:23:03,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:03,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:03,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:03,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01ccce46d8ec186ac6ec HTTP/1.1" 200 None 2026-03-09 17:23:03,818 - lava_callback - INFO - Completed processing callback for node 69af01ccce46d8ec186ac6ec INFO: 172.21.0.1:51712 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:23:13,895 - lava_callback - INFO - Received callback for node 69af0198ce46d8ec186ac628 INFO: 172.21.0.1:49500 - "POST /node/69af0198ce46d8ec186ac628 HTTP/1.0" 202 Accepted 2026-03-09 17:23:13,902 - lava_callback - INFO - Starting async job submit for node 69af0198ce46d8ec186ac628 2026-03-09 17:23:13,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:13,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0198ce46d8ec186ac628 HTTP/1.1" 200 None 2026-03-09 17:23:13,981 - lava_callback - DEBUG - Processing node 69af0198ce46d8ec186ac628: baseline-arm64 2026-03-09 17:23:14,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:14,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:14,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 17:23:14,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:14,938 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0198ce46d8ec186ac628/log.txt.gz INFO: 172.21.0.1:49502 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:23:15,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:15,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:15,710 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0198ce46d8ec186ac628/lava-logs.yaml 2026-03-09 17:23:15,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:15,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:15,839 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0198ce46d8ec186ac628/lava_callback.json.gz DEBUG: Setup failed for 69af0198ce46d8ec186ac628. Transitting job node result: incomplete -> fail 2026-03-09 17:23:15,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:15,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0198ce46d8ec186ac628 HTTP/1.1" 200 None 2026-03-09 17:23:15,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:15,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:15,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:16,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0198ce46d8ec186ac628 HTTP/1.1" 200 None 2026-03-09 17:23:16,179 - lava_callback - INFO - Completed processing callback for node 69af0198ce46d8ec186ac628 INFO: 172.21.0.1:49508 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:23:20,993 - lava_callback - INFO - Received callback for node 69af0196ce46d8ec186ac625 INFO: 172.21.0.1:34424 - "POST /node/69af0196ce46d8ec186ac625 HTTP/1.0" 202 Accepted 2026-03-09 17:23:21,001 - lava_callback - INFO - Starting async job submit for node 69af0196ce46d8ec186ac625 2026-03-09 17:23:21,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:21,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0196ce46d8ec186ac625 HTTP/1.1" 200 None 2026-03-09 17:23:21,051 - lava_callback - DEBUG - Processing node 69af0196ce46d8ec186ac625: baseline-arm64 2026-03-09 17:23:22,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:22,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:22,411 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0196ce46d8ec186ac625/log.txt.gz 2026-03-09 17:23:23,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:23,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:23,388 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0196ce46d8ec186ac625/lava-logs.yaml 2026-03-09 17:23:23,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:23,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:23,799 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0196ce46d8ec186ac625/lava_callback.json.gz DEBUG: Setup failed for 69af0196ce46d8ec186ac625. Transitting job node result: incomplete -> fail 2026-03-09 17:23:23,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:23,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0196ce46d8ec186ac625 HTTP/1.1" 200 None 2026-03-09 17:23:23,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:23,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:23,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:23,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0196ce46d8ec186ac625 HTTP/1.1" 200 None 2026-03-09 17:23:24,034 - lava_callback - INFO - Completed processing callback for node 69af0196ce46d8ec186ac625 INFO: 172.21.0.1:34438 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:23:27,005 - lava_callback - INFO - Received callback for node 69af01a0ce46d8ec186ac659 INFO: 172.21.0.1:34446 - "POST /node/69af01a0ce46d8ec186ac659 HTTP/1.0" 202 Accepted 2026-03-09 17:23:27,016 - lava_callback - INFO - Starting async job submit for node 69af01a0ce46d8ec186ac659 2026-03-09 17:23:27,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:27,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a0ce46d8ec186ac659 HTTP/1.1" 200 None 2026-03-09 17:23:27,097 - lava_callback - DEBUG - Processing node 69af01a0ce46d8ec186ac659: baseline-arm64 2026-03-09 17:23:27,206 - lava_callback - INFO - Received callback for node 69af019ece46d8ec186ac639 INFO: 172.21.0.1:34454 - "POST /node/69af019ece46d8ec186ac639 HTTP/1.0" 202 Accepted 2026-03-09 17:23:27,228 - lava_callback - INFO - Starting async job submit for node 69af019ece46d8ec186ac639 2026-03-09 17:23:27,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:27,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019ece46d8ec186ac639 HTTP/1.1" 200 None 2026-03-09 17:23:27,429 - lava_callback - DEBUG - Processing node 69af019ece46d8ec186ac639: baseline-arm64 2026-03-09 17:23:28,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:28,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:28,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:28,218 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af01a0ce46d8ec186ac659/log.txt.gz 2026-03-09 17:23:28,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:28,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:28,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ece46d8ec186ac639/log.txt.gz 2026-03-09 17:23:28,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:28,952 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af01a0ce46d8ec186ac659/lava-logs.yaml 2026-03-09 17:23:28,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:28,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:29,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:29,480 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af01a0ce46d8ec186ac659/lava_callback.json.gz 2026-03-09 17:23:29,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:29,592 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ece46d8ec186ac639/lava-logs.yaml DEBUG: Setup failed for 69af01a0ce46d8ec186ac659. Transitting job node result: incomplete -> fail 2026-03-09 17:23:29,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:29,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:29,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a0ce46d8ec186ac659 HTTP/1.1" 200 None 2026-03-09 17:23:29,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:29,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:29,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:29,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a0ce46d8ec186ac659 HTTP/1.1" 200 None 2026-03-09 17:23:29,824 - lava_callback - INFO - Completed processing callback for node 69af01a0ce46d8ec186ac659 2026-03-09 17:23:30,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:30,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ece46d8ec186ac639/lava_callback.json.gz DEBUG: Setup failed for 69af019ece46d8ec186ac639. Transitting job node result: incomplete -> fail 2026-03-09 17:23:30,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:30,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019ece46d8ec186ac639 HTTP/1.1" 200 None 2026-03-09 17:23:30,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:30,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:30,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:30,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af019ece46d8ec186ac639 HTTP/1.1" 200 None 2026-03-09 17:23:30,604 - lava_callback - INFO - Completed processing callback for node 69af019ece46d8ec186ac639 INFO: 172.21.0.1:52748 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:23:36,738 - lava_callback - INFO - Received callback for node 69af01d5ce46d8ec186ac706 INFO: 172.21.0.1:52762 - "POST /node/69af01d5ce46d8ec186ac706 HTTP/1.0" 202 Accepted 2026-03-09 17:23:36,747 - lava_callback - INFO - Starting async job submit for node 69af01d5ce46d8ec186ac706 2026-03-09 17:23:36,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:36,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d5ce46d8ec186ac706 HTTP/1.1" 200 None 2026-03-09 17:23:36,806 - lava_callback - DEBUG - Processing node 69af01d5ce46d8ec186ac706: baseline-arm64-pengutronix 2026-03-09 17:23:36,872 - lava_callback - INFO - Received callback for node 69af0199ce46d8ec186ac62b INFO: 172.21.0.1:52768 - "POST /node/69af0199ce46d8ec186ac62b HTTP/1.0" 202 Accepted 2026-03-09 17:23:36,899 - lava_callback - INFO - Starting async job submit for node 69af0199ce46d8ec186ac62b 2026-03-09 17:23:37,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:37,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0199ce46d8ec186ac62b HTTP/1.1" 200 None 2026-03-09 17:23:37,089 - lava_callback - DEBUG - Processing node 69af0199ce46d8ec186ac62b: baseline-arm64 2026-03-09 17:23:37,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:37,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:37,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d5ce46d8ec186ac706/log.txt.gz 2026-03-09 17:23:38,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:38,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:38,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:38,728 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0199ce46d8ec186ac62b/log.txt.gz 2026-03-09 17:23:38,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:38,833 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d5ce46d8ec186ac706/lava-logs.yaml 2026-03-09 17:23:38,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:39,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:39,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:39,131 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d5ce46d8ec186ac706/lava_callback.json.gz 2026-03-09 17:23:39,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:39,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d5ce46d8ec186ac706 HTTP/1.1" 200 None 2026-03-09 17:23:39,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:39,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:39,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:39,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01d5ce46d8ec186ac706 HTTP/1.1" 200 None 2026-03-09 17:23:39,265 - lava_callback - INFO - Completed processing callback for node 69af01d5ce46d8ec186ac706 2026-03-09 17:23:39,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:39,742 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0199ce46d8ec186ac62b/lava-logs.yaml 2026-03-09 17:23:39,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:39,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:39,974 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0199ce46d8ec186ac62b/lava_callback.json.gz DEBUG: Setup failed for 69af0199ce46d8ec186ac62b. Transitting job node result: incomplete -> fail 2026-03-09 17:23:40,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:40,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0199ce46d8ec186ac62b HTTP/1.1" 200 None 2026-03-09 17:23:40,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:40,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:40,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:40,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0199ce46d8ec186ac62b HTTP/1.1" 200 None 2026-03-09 17:23:40,287 - lava_callback - INFO - Completed processing callback for node 69af0199ce46d8ec186ac62b 2026-03-09 17:23:42,241 - lava_callback - INFO - Received callback for node 69af01a8ce46d8ec186ac66b INFO: 172.21.0.1:47594 - "POST /node/69af01a8ce46d8ec186ac66b HTTP/1.0" 202 Accepted 2026-03-09 17:23:42,248 - lava_callback - INFO - Starting async job submit for node 69af01a8ce46d8ec186ac66b 2026-03-09 17:23:42,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:42,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a8ce46d8ec186ac66b HTTP/1.1" 200 None 2026-03-09 17:23:42,325 - lava_callback - DEBUG - Processing node 69af01a8ce46d8ec186ac66b: baseline-nfs-arm64 2026-03-09 17:23:42,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:43,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:43,167 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a8ce46d8ec186ac66b/log.txt.gz 2026-03-09 17:23:43,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:43,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:43,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a8ce46d8ec186ac66b/lava-logs.yaml 2026-03-09 17:23:43,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:44,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:44,218 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a8ce46d8ec186ac66b/lava_callback.json.gz DEBUG: Setup failed for 69af01a8ce46d8ec186ac66b. Transitting job node result: incomplete -> fail 2026-03-09 17:23:44,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:44,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a8ce46d8ec186ac66b HTTP/1.1" 200 None 2026-03-09 17:23:44,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:44,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:44,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:44,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a8ce46d8ec186ac66b HTTP/1.1" 200 None 2026-03-09 17:23:44,429 - lava_callback - INFO - Completed processing callback for node 69af01a8ce46d8ec186ac66b 2026-03-09 17:23:44,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:44,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:47600 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:23:48,418 - lava_callback - INFO - Received callback for node 69af01d6ce46d8ec186ac709 INFO: 172.21.0.1:47612 - "POST /node/69af01d6ce46d8ec186ac709 HTTP/1.0" 202 Accepted 2026-03-09 17:23:48,427 - lava_callback - INFO - Starting async job submit for node 69af01d6ce46d8ec186ac709 2026-03-09 17:23:48,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:48,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d6ce46d8ec186ac709 HTTP/1.1" 200 None 2026-03-09 17:23:48,502 - lava_callback - DEBUG - Processing node 69af01d6ce46d8ec186ac709: baseline-arm64-pengutronix 2026-03-09 17:23:48,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:48,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:48,989 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d6ce46d8ec186ac709/log.txt.gz 2026-03-09 17:23:49,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:49,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:49,576 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d6ce46d8ec186ac709/lava-logs.yaml 2026-03-09 17:23:49,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:49,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:49,755 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d6ce46d8ec186ac709/lava_callback.json.gz 2026-03-09 17:23:49,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:49,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d6ce46d8ec186ac709 HTTP/1.1" 200 None 2026-03-09 17:23:49,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:49,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:49,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:49,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01d6ce46d8ec186ac709 HTTP/1.1" 200 None 2026-03-09 17:23:49,849 - lava_callback - INFO - Completed processing callback for node 69af01d6ce46d8ec186ac709 INFO: 172.21.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48906 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:24:01,573 - lava_callback - INFO - Received callback for node 69af01d9ce46d8ec186ac710 INFO: 172.21.0.1:41442 - "POST /node/69af01d9ce46d8ec186ac710 HTTP/1.0" 202 Accepted 2026-03-09 17:24:01,582 - lava_callback - INFO - Starting async job submit for node 69af01d9ce46d8ec186ac710 2026-03-09 17:24:01,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:01,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d9ce46d8ec186ac710 HTTP/1.1" 200 None 2026-03-09 17:24:01,683 - lava_callback - DEBUG - Processing node 69af01d9ce46d8ec186ac710: baseline-arm64-foundriesio 2026-03-09 17:24:02,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:02,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:02,201 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af01d9ce46d8ec186ac710/log.txt.gz 2026-03-09 17:24:02,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:02,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:02,733 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af01d9ce46d8ec186ac710/lava-logs.yaml 2026-03-09 17:24:02,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:02,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:02,919 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af01d9ce46d8ec186ac710/lava_callback.json.gz DEBUG: Setup failed for 69af01d9ce46d8ec186ac710. Transitting job node result: incomplete -> fail 2026-03-09 17:24:02,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:03,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d9ce46d8ec186ac710 HTTP/1.1" 200 None 2026-03-09 17:24:03,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:03,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:03,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:03,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01d9ce46d8ec186ac710 HTTP/1.1" 200 None 2026-03-09 17:24:03,217 - lava_callback - INFO - Completed processing callback for node 69af01d9ce46d8ec186ac710 2026-03-09 17:24:06,366 - lava_callback - INFO - Received callback for node 69af01aece46d8ec186ac67e INFO: 172.21.0.1:41456 - "POST /node/69af01aece46d8ec186ac67e HTTP/1.0" 202 Accepted 2026-03-09 17:24:06,376 - lava_callback - INFO - Starting async job submit for node 69af01aece46d8ec186ac67e 2026-03-09 17:24:06,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:06,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01aece46d8ec186ac67e HTTP/1.1" 200 None 2026-03-09 17:24:06,469 - lava_callback - DEBUG - Processing node 69af01aece46d8ec186ac67e: baseline-nfs-arm64 INFO: 172.21.0.1:41462 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:24:07,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:07,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:07,227 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01aece46d8ec186ac67e/log.txt.gz 2026-03-09 17:24:07,436 - lava_callback - INFO - Received callback for node 69af01a7ce46d8ec186ac668 INFO: 172.21.0.1:41466 - "POST /node/69af01a7ce46d8ec186ac668 HTTP/1.0" 202 Accepted 2026-03-09 17:24:07,469 - lava_callback - INFO - Starting async job submit for node 69af01a7ce46d8ec186ac668 2026-03-09 17:24:07,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:07,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:07,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a7ce46d8ec186ac668 HTTP/1.1" 200 None 2026-03-09 17:24:07,599 - lava_callback - DEBUG - Processing node 69af01a7ce46d8ec186ac668: baseline-nfs-arm64 2026-03-09 17:24:08,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:08,052 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01aece46d8ec186ac67e/lava-logs.yaml 2026-03-09 17:24:08,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:08,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:08,741 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01aece46d8ec186ac67e/lava_callback.json.gz DEBUG: Setup failed for 69af01aece46d8ec186ac67e. Transitting job node result: incomplete -> fail 2026-03-09 17:24:08,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:08,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:08,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01aece46d8ec186ac67e HTTP/1.1" 200 None 2026-03-09 17:24:08,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:09,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:09,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:09,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01aece46d8ec186ac67e HTTP/1.1" 200 None 2026-03-09 17:24:09,208 - lava_callback - INFO - Completed processing callback for node 69af01aece46d8ec186ac67e 2026-03-09 17:24:09,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:09,498 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a7ce46d8ec186ac668/log.txt.gz 2026-03-09 17:24:10,094 - lava_callback - INFO - Received callback for node 69af01d8ce46d8ec186ac70d INFO: 172.21.0.1:59216 - "POST /node/69af01d8ce46d8ec186ac70d HTTP/1.0" 202 Accepted 2026-03-09 17:24:10,117 - lava_callback - INFO - Starting async job submit for node 69af01d8ce46d8ec186ac70d 2026-03-09 17:24:10,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:10,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:10,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d8ce46d8ec186ac70d HTTP/1.1" 200 None 2026-03-09 17:24:10,255 - lava_callback - DEBUG - Processing node 69af01d8ce46d8ec186ac70d: baseline-arm64-pengutronix 2026-03-09 17:24:10,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:10,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:10,891 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d8ce46d8ec186ac70d/log.txt.gz 2026-03-09 17:24:10,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:10,919 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a7ce46d8ec186ac668/lava-logs.yaml 2026-03-09 17:24:11,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:11,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:11,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:11,237 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a7ce46d8ec186ac668/lava_callback.json.gz DEBUG: Setup failed for 69af01a7ce46d8ec186ac668. Transitting job node result: incomplete -> fail 2026-03-09 17:24:11,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:11,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a7ce46d8ec186ac668 HTTP/1.1" 200 None 2026-03-09 17:24:11,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:11,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:11,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:11,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a7ce46d8ec186ac668 HTTP/1.1" 200 None 2026-03-09 17:24:11,562 - lava_callback - INFO - Completed processing callback for node 69af01a7ce46d8ec186ac668 2026-03-09 17:24:11,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:11,772 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d8ce46d8ec186ac70d/lava-logs.yaml 2026-03-09 17:24:11,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:11,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:11,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d8ce46d8ec186ac70d/lava_callback.json.gz 2026-03-09 17:24:11,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:11,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d8ce46d8ec186ac70d HTTP/1.1" 200 None 2026-03-09 17:24:11,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:11,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:11,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:12,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01d8ce46d8ec186ac70d HTTP/1.1" 200 None 2026-03-09 17:24:12,028 - lava_callback - INFO - Completed processing callback for node 69af01d8ce46d8ec186ac70d 2026-03-09 17:24:14,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:14,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:59218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59220 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:24:21,322 - lava_callback - INFO - Received callback for node 69af01a9ce46d8ec186ac670 INFO: 172.21.0.1:34734 - "POST /node/69af01a9ce46d8ec186ac670 HTTP/1.0" 202 Accepted 2026-03-09 17:24:21,329 - lava_callback - INFO - Starting async job submit for node 69af01a9ce46d8ec186ac670 2026-03-09 17:24:21,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:21,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a9ce46d8ec186ac670 HTTP/1.1" 200 None 2026-03-09 17:24:21,394 - lava_callback - DEBUG - Processing node 69af01a9ce46d8ec186ac670: baseline-nfs-arm64 2026-03-09 17:24:22,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:22,454 - lava_callback - INFO - Received callback for node 69af01b1ce46d8ec186ac688 INFO: 172.21.0.1:34740 - "POST /node/69af01b1ce46d8ec186ac688 HTTP/1.0" 202 Accepted 2026-03-09 17:24:22,466 - lava_callback - INFO - Starting async job submit for node 69af01b1ce46d8ec186ac688 2026-03-09 17:24:22,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:22,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01b1ce46d8ec186ac688 HTTP/1.1" 200 None 2026-03-09 17:24:22,552 - lava_callback - DEBUG - Processing node 69af01b1ce46d8ec186ac688: baseline-nfs-arm64 2026-03-09 17:24:22,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:22,577 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a9ce46d8ec186ac670/log.txt.gz 2026-03-09 17:24:23,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:23,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:23,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:23,952 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01b1ce46d8ec186ac688/log.txt.gz 2026-03-09 17:24:24,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:24,151 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a9ce46d8ec186ac670/lava-logs.yaml 2026-03-09 17:24:24,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:24,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:24,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:24,711 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a9ce46d8ec186ac670/lava_callback.json.gz 2026-03-09 17:24:24,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 DEBUG: Setup failed for 69af01a9ce46d8ec186ac670. Transitting job node result: incomplete -> fail 2026-03-09 17:24:24,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:24,839 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01b1ce46d8ec186ac688/lava-logs.yaml 2026-03-09 17:24:24,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:24,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a9ce46d8ec186ac670 HTTP/1.1" 200 None 2026-03-09 17:24:24,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:24,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:24,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:24,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a9ce46d8ec186ac670 HTTP/1.1" 200 None 2026-03-09 17:24:24,998 - lava_callback - INFO - Completed processing callback for node 69af01a9ce46d8ec186ac670 2026-03-09 17:24:25,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:25,099 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01b1ce46d8ec186ac688/lava_callback.json.gz DEBUG: Setup failed for 69af01b1ce46d8ec186ac688. Transitting job node result: incomplete -> fail 2026-03-09 17:24:25,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:25,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01b1ce46d8ec186ac688 HTTP/1.1" 200 None 2026-03-09 17:24:25,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:25,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:25,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:25,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01b1ce46d8ec186ac688 HTTP/1.1" 200 None 2026-03-09 17:24:25,471 - lava_callback - INFO - Completed processing callback for node 69af01b1ce46d8ec186ac688 2026-03-09 17:24:26,247 - lava_callback - INFO - Received callback for node 69ac5f4646f7ae8ff3365a91 INFO: 172.21.0.1:34748 - "POST /node/69ac5f4646f7ae8ff3365a91 HTTP/1.0" 202 Accepted 2026-03-09 17:24:26,256 - lava_callback - INFO - Starting async job submit for node 69ac5f4646f7ae8ff3365a91 2026-03-09 17:24:26,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:26,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5f4646f7ae8ff3365a91 HTTP/1.1" 200 None 2026-03-09 17:24:26,286 - lava_callback - DEBUG - Processing node 69ac5f4646f7ae8ff3365a91: baseline-arm64-baylibre 2026-03-09 17:24:26,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5f4646f7ae8ff3365a91 2026-03-09 17:24:26,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:26,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5f4646f7ae8ff3365a91 HTTP/1.1" 200 None 2026-03-09 17:24:26,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:26,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a64e31b554803095c1a HTTP/1.1" 200 None 2026-03-09 17:24:26,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:26,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5f4646f7ae8ff3365a91 HTTP/1.1" 200 None 2026-03-09 17:24:26,377 - lava_callback - ERROR - Error processing callback for node 69ac5f4646f7ae8ff3365a91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34224 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:24:44,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:44,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58812 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:25:03,393 - lava_callback - INFO - Received callback for node 69af01c1ce46d8ec186ac6ca INFO: 172.21.0.1:60404 - "POST /node/69af01c1ce46d8ec186ac6ca HTTP/1.0" 202 Accepted 2026-03-09 17:25:03,400 - lava_callback - INFO - Starting async job submit for node 69af01c1ce46d8ec186ac6ca 2026-03-09 17:25:03,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:03,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c1ce46d8ec186ac6ca HTTP/1.1" 200 None 2026-03-09 17:25:03,434 - lava_callback - DEBUG - Processing node 69af01c1ce46d8ec186ac6ca: baseline-arm64-broonie 2026-03-09 17:25:03,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:03,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:03,918 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c1ce46d8ec186ac6ca/log.txt.gz 2026-03-09 17:25:04,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:04,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:04,330 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c1ce46d8ec186ac6ca/lava-logs.yaml 2026-03-09 17:25:04,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:04,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:04,486 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c1ce46d8ec186ac6ca/lava_callback.json.gz 2026-03-09 17:25:04,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:04,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c1ce46d8ec186ac6ca HTTP/1.1" 200 None 2026-03-09 17:25:04,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:04,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:25:04,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:04,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c1ce46d8ec186ac6ca HTTP/1.1" 200 None 2026-03-09 17:25:04,616 - lava_callback - INFO - Completed processing callback for node 69af01c1ce46d8ec186ac6ca INFO: 172.21.0.1:60410 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:25:09,059 - lava_callback - INFO - Received callback for node 69af0190ce46d8ec186ac616 INFO: 172.21.0.1:60412 - "POST /node/69af0190ce46d8ec186ac616 HTTP/1.0" 202 Accepted 2026-03-09 17:25:09,065 - lava_callback - INFO - Starting async job submit for node 69af0190ce46d8ec186ac616 2026-03-09 17:25:09,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:09,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0190ce46d8ec186ac616 HTTP/1.1" 200 None 2026-03-09 17:25:09,111 - lava_callback - DEBUG - Processing node 69af0190ce46d8ec186ac616: baseline-arm64 2026-03-09 17:25:09,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:09,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:09,650 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0190ce46d8ec186ac616/log.txt.gz 2026-03-09 17:25:09,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:10,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:10,066 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0190ce46d8ec186ac616/lava-logs.yaml 2026-03-09 17:25:10,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:10,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:10,206 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0190ce46d8ec186ac616/lava_callback.json.gz DEBUG: Setup failed for 69af0190ce46d8ec186ac616. Transitting job node result: incomplete -> fail 2026-03-09 17:25:10,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:10,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0190ce46d8ec186ac616 HTTP/1.1" 200 None 2026-03-09 17:25:10,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:10,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:25:10,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:10,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0190ce46d8ec186ac616 HTTP/1.1" 200 None 2026-03-09 17:25:10,386 - lava_callback - INFO - Completed processing callback for node 69af0190ce46d8ec186ac616 2026-03-09 17:25:11,596 - lava_callback - INFO - Received callback for node 69af01afce46d8ec186ac681 INFO: 172.21.0.1:35462 - "POST /node/69af01afce46d8ec186ac681 HTTP/1.0" 202 Accepted 2026-03-09 17:25:11,602 - lava_callback - INFO - Starting async job submit for node 69af01afce46d8ec186ac681 2026-03-09 17:25:11,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:11,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01afce46d8ec186ac681 HTTP/1.1" 200 None 2026-03-09 17:25:11,653 - lava_callback - DEBUG - Processing node 69af01afce46d8ec186ac681: baseline-nfs-arm64 2026-03-09 17:25:12,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:12,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:12,176 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01afce46d8ec186ac681/log.txt.gz 2026-03-09 17:25:12,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:12,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:12,625 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01afce46d8ec186ac681/lava-logs.yaml 2026-03-09 17:25:12,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:12,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:12,773 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01afce46d8ec186ac681/lava_callback.json.gz DEBUG: Setup failed for 69af01afce46d8ec186ac681. Transitting job node result: incomplete -> fail 2026-03-09 17:25:12,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:12,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01afce46d8ec186ac681 HTTP/1.1" 200 None 2026-03-09 17:25:12,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:12,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:25:12,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:12,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01afce46d8ec186ac681 HTTP/1.1" 200 None 2026-03-09 17:25:12,960 - lava_callback - INFO - Completed processing callback for node 69af01afce46d8ec186ac681 2026-03-09 17:25:14,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:14,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35486 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:25:26,230 - lava_callback - INFO - Received callback for node 69ac5f8246f7ae8ff3365a9e INFO: 172.21.0.1:38966 - "POST /node/69ac5f8246f7ae8ff3365a9e HTTP/1.0" 202 Accepted 2026-03-09 17:25:26,235 - lava_callback - INFO - Starting async job submit for node 69ac5f8246f7ae8ff3365a9e 2026-03-09 17:25:26,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:26,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5f8246f7ae8ff3365a9e HTTP/1.1" 200 None 2026-03-09 17:25:26,251 - lava_callback - DEBUG - Processing node 69ac5f8246f7ae8ff3365a9e: baseline-arm-baylibre 2026-03-09 17:25:26,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5f8246f7ae8ff3365a9e 2026-03-09 17:25:26,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:26,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5f8246f7ae8ff3365a9e HTTP/1.1" 200 None 2026-03-09 17:25:26,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:26,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None 2026-03-09 17:25:26,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:26,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5f8246f7ae8ff3365a9e HTTP/1.1" 200 None 2026-03-09 17:25:26,337 - lava_callback - ERROR - Error processing callback for node 69ac5f8246f7ae8ff3365a9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:25:45,399 - lava_callback - INFO - Received callback for node 69af01a3ce46d8ec186ac65f INFO: 172.21.0.1:55494 - "POST /node/69af01a3ce46d8ec186ac65f HTTP/1.0" 202 Accepted 2026-03-09 17:25:45,406 - lava_callback - INFO - Starting async job submit for node 69af01a3ce46d8ec186ac65f 2026-03-09 17:25:45,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:45,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a3ce46d8ec186ac65f HTTP/1.1" 200 None 2026-03-09 17:25:45,456 - lava_callback - DEBUG - Processing node 69af01a3ce46d8ec186ac65f: baseline-nfs-arm64 2026-03-09 17:25:45,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:45,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:45,970 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a3ce46d8ec186ac65f/log.txt.gz 2026-03-09 17:25:46,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:46,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:46,431 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a3ce46d8ec186ac65f/lava-logs.yaml 2026-03-09 17:25:46,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:55498 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:25:46,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:46,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a3ce46d8ec186ac65f/lava_callback.json.gz DEBUG: Setup failed for 69af01a3ce46d8ec186ac65f. Transitting job node result: incomplete -> fail 2026-03-09 17:25:46,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:46,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a3ce46d8ec186ac65f HTTP/1.1" 200 None 2026-03-09 17:25:46,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:46,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:25:46,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:46,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a3ce46d8ec186ac65f HTTP/1.1" 200 None 2026-03-09 17:25:46,835 - lava_callback - INFO - Completed processing callback for node 69af01a3ce46d8ec186ac65f INFO: 172.21.0.1:39296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50804 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:26:14,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:14,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50940 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:26:20,703 - lava_callback - INFO - Received callback for node 69af019fce46d8ec186ac655 INFO: 172.21.0.1:40100 - "POST /node/69af019fce46d8ec186ac655 HTTP/1.0" 202 Accepted 2026-03-09 17:26:20,709 - lava_callback - INFO - Starting async job submit for node 69af019fce46d8ec186ac655 2026-03-09 17:26:20,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:20,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019fce46d8ec186ac655 HTTP/1.1" 200 None 2026-03-09 17:26:20,800 - lava_callback - DEBUG - Processing node 69af019fce46d8ec186ac655: baseline-arm64 2026-03-09 17:26:21,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:26:21,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:26:21,264 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019fce46d8ec186ac655/log.txt.gz 2026-03-09 17:26:21,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:26:21,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:26:21,657 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019fce46d8ec186ac655/lava-logs.yaml 2026-03-09 17:26:21,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:26:21,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:26:21,827 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019fce46d8ec186ac655/lava_callback.json.gz DEBUG: Setup failed for 69af019fce46d8ec186ac655. Transitting job node result: incomplete -> fail 2026-03-09 17:26:21,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:21,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019fce46d8ec186ac655 HTTP/1.1" 200 None 2026-03-09 17:26:21,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:21,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:26:21,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:21,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af019fce46d8ec186ac655 HTTP/1.1" 200 None 2026-03-09 17:26:22,029 - lava_callback - INFO - Completed processing callback for node 69af019fce46d8ec186ac655 INFO: 172.21.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51816 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:26:44,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:44,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51124 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:28:45,121 - lava_callback - INFO - Received callback for node 69af01bfce46d8ec186ac6c5 INFO: 172.21.0.1:39584 - "POST /node/69af01bfce46d8ec186ac6c5 HTTP/1.0" 202 Accepted 2026-03-09 17:28:45,136 - lava_callback - INFO - Starting async job submit for node 69af01bfce46d8ec186ac6c5 2026-03-09 17:28:45,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:45,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01bfce46d8ec186ac6c5 HTTP/1.1" 200 None 2026-03-09 17:28:45,241 - lava_callback - DEBUG - Processing node 69af01bfce46d8ec186ac6c5: baseline-arm64-broonie 2026-03-09 17:28:45,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:28:45,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:28:45,767 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bfce46d8ec186ac6c5/log.txt.gz 2026-03-09 17:28:45,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:28:46,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:28:46,223 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bfce46d8ec186ac6c5/lava-logs.yaml 2026-03-09 17:28:46,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:28:46,426 - lava_callback - INFO - Received callback for node 69ac603a46f7ae8ff3365aed INFO: 172.21.0.1:39592 - "POST /node/69ac603a46f7ae8ff3365aed HTTP/1.0" 202 Accepted 2026-03-09 17:28:46,432 - lava_callback - INFO - Starting async job submit for node 69ac603a46f7ae8ff3365aed 2026-03-09 17:28:46,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:28:46,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,436 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bfce46d8ec186ac6c5/lava_callback.json.gz 2026-03-09 17:28:46,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac603a46f7ae8ff3365aed HTTP/1.1" 200 None DEBUG: Setup failed for 69af01bfce46d8ec186ac6c5. Transitting job node result: incomplete -> fail 2026-03-09 17:28:46,518 - lava_callback - DEBUG - Processing node 69ac603a46f7ae8ff3365aed: baseline-arm-baylibre 2026-03-09 17:28:46,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac603a46f7ae8ff3365aed 2026-03-09 17:28:46,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:39602 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:28:46,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01bfce46d8ec186ac6c5 HTTP/1.1" 200 None 2026-03-09 17:28:46,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac603a46f7ae8ff3365aed HTTP/1.1" 200 None 2026-03-09 17:28:46,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:28:46,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None 2026-03-09 17:28:46,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01bfce46d8ec186ac6c5 HTTP/1.1" 200 None 2026-03-09 17:28:46,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac603a46f7ae8ff3365aed HTTP/1.1" 200 None 2026-03-09 17:28:46,646 - lava_callback - ERROR - Error processing callback for node 69ac603a46f7ae8ff3365aed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 17:28:46,774 - lava_callback - INFO - Completed processing callback for node 69af01bfce46d8ec186ac6c5 INFO: 172.21.0.1:49706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49722 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:29:06,409 - lava_callback - INFO - Received callback for node 69ac606046f7ae8ff3365b05 INFO: 172.21.0.1:39574 - "POST /node/69ac606046f7ae8ff3365b05 HTTP/1.0" 202 Accepted 2026-03-09 17:29:06,415 - lava_callback - INFO - Starting async job submit for node 69ac606046f7ae8ff3365b05 2026-03-09 17:29:06,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:29:06,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac606046f7ae8ff3365b05 HTTP/1.1" 200 None 2026-03-09 17:29:06,440 - lava_callback - DEBUG - Processing node 69ac606046f7ae8ff3365b05: baseline-arm-baylibre 2026-03-09 17:29:06,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac606046f7ae8ff3365b05 2026-03-09 17:29:06,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:29:06,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac606046f7ae8ff3365b05 HTTP/1.1" 200 None 2026-03-09 17:29:06,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:29:06,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None INFO: 172.21.0.1:39588 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:29:06,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:29:06,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac606046f7ae8ff3365b05 HTTP/1.1" 200 None 2026-03-09 17:29:06,542 - lava_callback - ERROR - Error processing callback for node 69ac606046f7ae8ff3365b05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 17:29:14,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:29:14,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44186 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:30:26,551 - lava_callback - INFO - Received callback for node 69ac60af46f7ae8ff3365b35 INFO: 172.21.0.1:44190 - "POST /node/69ac60af46f7ae8ff3365b35 HTTP/1.0" 202 Accepted 2026-03-09 17:30:26,556 - lava_callback - INFO - Starting async job submit for node 69ac60af46f7ae8ff3365b35 2026-03-09 17:30:26,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:26,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac60af46f7ae8ff3365b35 HTTP/1.1" 200 None 2026-03-09 17:30:26,572 - lava_callback - DEBUG - Processing node 69ac60af46f7ae8ff3365b35: baseline-arm-baylibre 2026-03-09 17:30:26,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac60af46f7ae8ff3365b35 2026-03-09 17:30:26,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:26,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac60af46f7ae8ff3365b35 HTTP/1.1" 200 None 2026-03-09 17:30:26,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:26,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None 2026-03-09 17:30:26,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:26,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac60af46f7ae8ff3365b35 HTTP/1.1" 200 None 2026-03-09 17:30:26,638 - lava_callback - ERROR - Error processing callback for node 69ac60af46f7ae8ff3365b35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50450 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:30:43,827 - lava_callback - INFO - Received callback for node 69af01c5ce46d8ec186ac6d3 INFO: 172.21.0.1:35374 - "POST /node/69af01c5ce46d8ec186ac6d3 HTTP/1.0" 202 Accepted 2026-03-09 17:30:43,833 - lava_callback - INFO - Starting async job submit for node 69af01c5ce46d8ec186ac6d3 2026-03-09 17:30:43,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:43,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c5ce46d8ec186ac6d3 HTTP/1.1" 200 None 2026-03-09 17:30:43,888 - lava_callback - DEBUG - Processing node 69af01c5ce46d8ec186ac6d3: baseline-arm64-broonie 2026-03-09 17:30:44,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:30:44,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:30:44,373 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c5ce46d8ec186ac6d3/log.txt.gz 2026-03-09 17:30:44,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:30:44,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:30:44,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c5ce46d8ec186ac6d3/lava-logs.yaml 2026-03-09 17:30:44,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:30:44,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:30:44,984 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c5ce46d8ec186ac6d3/lava_callback.json.gz DEBUG: Setup failed for 69af01c5ce46d8ec186ac6d3. Transitting job node result: incomplete -> fail 2026-03-09 17:30:45,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:45,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c5ce46d8ec186ac6d3 HTTP/1.1" 200 None 2026-03-09 17:30:45,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:45,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:30:45,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:45,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c5ce46d8ec186ac6d3 HTTP/1.1" 200 None 2026-03-09 17:30:45,208 - lava_callback - INFO - Completed processing callback for node 69af01c5ce46d8ec186ac6d3 INFO: 172.21.0.1:35378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52606 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:31:14,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:14,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50910 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:31:32,506 - lava_callback - INFO - Received callback for node 69af03eace46d8ec186ac862 INFO: 172.21.0.1:34926 - "POST /node/69af03eace46d8ec186ac862 HTTP/1.0" 202 Accepted 2026-03-09 17:31:32,512 - lava_callback - INFO - Starting async job submit for node 69af03eace46d8ec186ac862 2026-03-09 17:31:32,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:32,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af03eace46d8ec186ac862 HTTP/1.1" 200 None 2026-03-09 17:31:32,531 - lava_callback - DEBUG - Processing node 69af03eace46d8ec186ac862: baseline-arm64 2026-03-09 17:31:32,536 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:31:32,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:31:32,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:31:32,693 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af03eace46d8ec186ac862/lava-logs.yaml 2026-03-09 17:31:32,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:31:32,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:31:32,805 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af03eace46d8ec186ac862/lava_callback.json.gz 2026-03-09 17:31:32,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:32,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af03eace46d8ec186ac862 HTTP/1.1" 200 None 2026-03-09 17:31:32,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:32,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:31:32,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:32,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af03eace46d8ec186ac862 HTTP/1.1" 200 None 2026-03-09 17:31:32,882 - lava_callback - INFO - Completed processing callback for node 69af03eace46d8ec186ac862 INFO: 172.21.0.1:34928 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:31:44,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:44,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34792 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:32:26,667 - lava_callback - INFO - Received callback for node 69ac612046f7ae8ff3365b68 INFO: 172.21.0.1:34806 - "POST /node/69ac612046f7ae8ff3365b68 HTTP/1.0" 202 Accepted 2026-03-09 17:32:26,673 - lava_callback - INFO - Starting async job submit for node 69ac612046f7ae8ff3365b68 2026-03-09 17:32:26,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:26,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac612046f7ae8ff3365b68 HTTP/1.1" 200 None 2026-03-09 17:32:26,688 - lava_callback - DEBUG - Processing node 69ac612046f7ae8ff3365b68: baseline-arm-baylibre 2026-03-09 17:32:26,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac612046f7ae8ff3365b68 2026-03-09 17:32:26,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:26,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac612046f7ae8ff3365b68 HTTP/1.1" 200 None 2026-03-09 17:32:26,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:26,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None 2026-03-09 17:32:26,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:26,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac612046f7ae8ff3365b68 HTTP/1.1" 200 None 2026-03-09 17:32:26,764 - lava_callback - ERROR - Error processing callback for node 69ac612046f7ae8ff3365b68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 17:32:30,101 - lava_callback - INFO - Received callback for node 69af0195ce46d8ec186ac622 INFO: 172.21.0.1:45448 - "POST /node/69af0195ce46d8ec186ac622 HTTP/1.0" 202 Accepted 2026-03-09 17:32:30,114 - lava_callback - INFO - Starting async job submit for node 69af0195ce46d8ec186ac622 2026-03-09 17:32:30,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:30,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0195ce46d8ec186ac622 HTTP/1.1" 200 None 2026-03-09 17:32:30,173 - lava_callback - DEBUG - Processing node 69af0195ce46d8ec186ac622: baseline-arm64 2026-03-09 17:32:30,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:30,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:30,629 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0195ce46d8ec186ac622/log.txt.gz 2026-03-09 17:32:30,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:31,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:31,052 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0195ce46d8ec186ac622/lava-logs.yaml 2026-03-09 17:32:31,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:31,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:31,294 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0195ce46d8ec186ac622/lava_callback.json.gz 2026-03-09 17:32:31,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:31,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0195ce46d8ec186ac622 HTTP/1.1" 200 None 2026-03-09 17:32:31,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:31,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:32:31,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:31,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0195ce46d8ec186ac622 HTTP/1.1" 200 None 2026-03-09 17:32:31,452 - lava_callback - INFO - Completed processing callback for node 69af0195ce46d8ec186ac622 INFO: 172.21.0.1:45456 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:32:38,198 - lava_callback - INFO - Received callback for node 69af019cce46d8ec186ac634 INFO: 172.21.0.1:45470 - "POST /node/69af019cce46d8ec186ac634 HTTP/1.0" 202 Accepted 2026-03-09 17:32:38,204 - lava_callback - INFO - Starting async job submit for node 69af019cce46d8ec186ac634 2026-03-09 17:32:38,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:38,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019cce46d8ec186ac634 HTTP/1.1" 200 None 2026-03-09 17:32:38,243 - lava_callback - DEBUG - Processing node 69af019cce46d8ec186ac634: baseline-arm64 2026-03-09 17:32:38,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:38,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:38,934 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019cce46d8ec186ac634/log.txt.gz 2026-03-09 17:32:39,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:39,426 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:39,428 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019cce46d8ec186ac634/lava-logs.yaml 2026-03-09 17:32:39,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:39,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:39,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019cce46d8ec186ac634/lava_callback.json.gz 2026-03-09 17:32:39,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:39,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019cce46d8ec186ac634 HTTP/1.1" 200 None 2026-03-09 17:32:39,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:39,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:32:39,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:39,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af019cce46d8ec186ac634 HTTP/1.1" 200 None 2026-03-09 17:32:39,754 - lava_callback - INFO - Completed processing callback for node 69af019cce46d8ec186ac634 2026-03-09 17:32:44,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:44,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:32:46,778 - lava_callback - INFO - Received callback for node 69af01adce46d8ec186ac679 INFO: 172.21.0.1:43968 - "POST /node/69af01adce46d8ec186ac679 HTTP/1.0" 202 Accepted 2026-03-09 17:32:46,785 - lava_callback - INFO - Starting async job submit for node 69af01adce46d8ec186ac679 2026-03-09 17:32:46,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:46,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01adce46d8ec186ac679 HTTP/1.1" 200 None 2026-03-09 17:32:46,825 - lava_callback - DEBUG - Processing node 69af01adce46d8ec186ac679: baseline-nfs-arm64 2026-03-09 17:32:47,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:47,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:47,453 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01adce46d8ec186ac679/log.txt.gz 2026-03-09 17:32:47,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:47,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:47,922 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01adce46d8ec186ac679/lava-logs.yaml 2026-03-09 17:32:47,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:48,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:48,100 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01adce46d8ec186ac679/lava_callback.json.gz 2026-03-09 17:32:48,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:48,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01adce46d8ec186ac679 HTTP/1.1" 200 None 2026-03-09 17:32:48,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:48,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:32:48,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:48,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01adce46d8ec186ac679 HTTP/1.1" 200 None 2026-03-09 17:32:48,250 - lava_callback - INFO - Completed processing callback for node 69af01adce46d8ec186ac679 INFO: 172.21.0.1:50366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50374 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:33:01,182 - lava_callback - INFO - Received callback for node 69af0437ce46d8ec186ac876 INFO: 172.21.0.1:39358 - "POST /node/69af0437ce46d8ec186ac876 HTTP/1.0" 202 Accepted 2026-03-09 17:33:01,188 - lava_callback - INFO - Starting async job submit for node 69af0437ce46d8ec186ac876 2026-03-09 17:33:01,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:01,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0437ce46d8ec186ac876 HTTP/1.1" 200 None 2026-03-09 17:33:01,206 - lava_callback - DEBUG - Processing node 69af0437ce46d8ec186ac876: baseline-arm64 2026-03-09 17:33:01,210 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:33:01,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:01,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:01,340 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0437ce46d8ec186ac876/lava-logs.yaml 2026-03-09 17:33:01,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:01,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:01,453 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0437ce46d8ec186ac876/lava_callback.json.gz 2026-03-09 17:33:01,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:01,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0437ce46d8ec186ac876 HTTP/1.1" 200 None 2026-03-09 17:33:01,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:01,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:33:01,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:01,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0437ce46d8ec186ac876 HTTP/1.1" 200 None 2026-03-09 17:33:01,546 - lava_callback - INFO - Completed processing callback for node 69af0437ce46d8ec186ac876 2026-03-09 17:33:04,111 - lava_callback - INFO - Received callback for node 69af01a5ce46d8ec186ac665 INFO: 172.21.0.1:39368 - "POST /node/69af01a5ce46d8ec186ac665 HTTP/1.0" 202 Accepted 2026-03-09 17:33:04,118 - lava_callback - INFO - Starting async job submit for node 69af01a5ce46d8ec186ac665 2026-03-09 17:33:04,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:04,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a5ce46d8ec186ac665 HTTP/1.1" 200 None 2026-03-09 17:33:04,170 - lava_callback - DEBUG - Processing node 69af01a5ce46d8ec186ac665: baseline-nfs-arm64 2026-03-09 17:33:04,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:04,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:04,598 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a5ce46d8ec186ac665/log.txt.gz 2026-03-09 17:33:04,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:04,967 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:04,969 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a5ce46d8ec186ac665/lava-logs.yaml 2026-03-09 17:33:04,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:05,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:05,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a5ce46d8ec186ac665/lava_callback.json.gz 2026-03-09 17:33:05,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:05,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a5ce46d8ec186ac665 HTTP/1.1" 200 None 2026-03-09 17:33:05,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:05,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:33:05,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:05,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a5ce46d8ec186ac665 HTTP/1.1" 200 None 2026-03-09 17:33:05,313 - lava_callback - INFO - Completed processing callback for node 69af01a5ce46d8ec186ac665 INFO: 172.21.0.1:39372 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:33:10,058 - lava_callback - INFO - Received callback for node 69af003fce46d8ec186ac4ca INFO: 172.21.0.1:53218 - "POST /node/69af003fce46d8ec186ac4ca HTTP/1.0" 202 Accepted 2026-03-09 17:33:10,065 - lava_callback - INFO - Starting async job submit for node 69af003fce46d8ec186ac4ca 2026-03-09 17:33:10,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:10,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af003fce46d8ec186ac4ca HTTP/1.1" 200 None 2026-03-09 17:33:10,139 - lava_callback - DEBUG - Processing node 69af003fce46d8ec186ac4ca: ltp-smoke 2026-03-09 17:33:10,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:10,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:10,775 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af003fce46d8ec186ac4ca/log.txt.gz 2026-03-09 17:33:11,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:11,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:11,266 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af003fce46d8ec186ac4ca/lava-logs.yaml 2026-03-09 17:33:11,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:11,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:11,438 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af003fce46d8ec186ac4ca/lava_callback.json.gz 2026-03-09 17:33:11,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:11,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af003fce46d8ec186ac4ca HTTP/1.1" 200 None 2026-03-09 17:33:11,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:11,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef42ace46d8ec186ab814 HTTP/1.1" 200 None 2026-03-09 17:33:11,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:11,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af003fce46d8ec186ac4ca HTTP/1.1" 200 None 2026-03-09 17:33:11,737 - lava_callback - INFO - Completed processing callback for node 69af003fce46d8ec186ac4ca 2026-03-09 17:33:14,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:14,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51822 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:34:26,735 - lava_callback - INFO - Received callback for node 69ac619146f7ae8ff3365b93 INFO: 172.21.0.1:51834 - "POST /node/69ac619146f7ae8ff3365b93 HTTP/1.0" 202 Accepted 2026-03-09 17:34:26,741 - lava_callback - INFO - Starting async job submit for node 69ac619146f7ae8ff3365b93 2026-03-09 17:34:26,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:26,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac619146f7ae8ff3365b93 HTTP/1.1" 200 None 2026-03-09 17:34:26,756 - lava_callback - DEBUG - Processing node 69ac619146f7ae8ff3365b93: baseline-arm-baylibre 2026-03-09 17:34:26,757 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac619146f7ae8ff3365b93 2026-03-09 17:34:26,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:26,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac619146f7ae8ff3365b93 HTTP/1.1" 200 None 2026-03-09 17:34:26,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:26,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None 2026-03-09 17:34:26,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:26,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac619146f7ae8ff3365b93 HTTP/1.1" 200 None 2026-03-09 17:34:26,843 - lava_callback - ERROR - Error processing callback for node 69ac619146f7ae8ff3365b93: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60420 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:34:50,384 - lava_callback - INFO - Received callback for node 69af01cace46d8ec186ac6e5 INFO: 172.21.0.1:36684 - "POST /node/69af01cace46d8ec186ac6e5 HTTP/1.0" 202 Accepted 2026-03-09 17:34:50,390 - lava_callback - INFO - Starting async job submit for node 69af01cace46d8ec186ac6e5 2026-03-09 17:34:50,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:50,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01cace46d8ec186ac6e5 HTTP/1.1" 200 None 2026-03-09 17:34:50,408 - lava_callback - DEBUG - Processing node 69af01cace46d8ec186ac6e5: baseline-arm64-broonie 2026-03-09 17:34:50,413 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:34:50,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:34:50,566 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:34:50,568 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01cace46d8ec186ac6e5/lava-logs.yaml 2026-03-09 17:34:50,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:34:50,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:34:50,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01cace46d8ec186ac6e5/lava_callback.json.gz 2026-03-09 17:34:50,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:50,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01cace46d8ec186ac6e5 HTTP/1.1" 200 None 2026-03-09 17:34:50,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:50,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:34:50,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:50,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01cace46d8ec186ac6e5 HTTP/1.1" 200 None 2026-03-09 17:34:50,749 - lava_callback - INFO - Completed processing callback for node 69af01cace46d8ec186ac6e5 INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:35:14,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:35:14,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44376 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:36:50,335 - lava_callback - INFO - Received callback for node 69af01c4ce46d8ec186ac6d0 INFO: 172.21.0.1:60544 - "POST /node/69af01c4ce46d8ec186ac6d0 HTTP/1.0" 202 Accepted 2026-03-09 17:36:50,341 - lava_callback - INFO - Starting async job submit for node 69af01c4ce46d8ec186ac6d0 2026-03-09 17:36:50,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:36:50,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c4ce46d8ec186ac6d0 HTTP/1.1" 200 None 2026-03-09 17:36:50,393 - lava_callback - DEBUG - Processing node 69af01c4ce46d8ec186ac6d0: baseline-arm64-broonie 2026-03-09 17:36:50,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:36:51,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:36:51,006 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c4ce46d8ec186ac6d0/log.txt.gz 2026-03-09 17:36:51,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:36:51,486 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:36:51,488 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c4ce46d8ec186ac6d0/lava-logs.yaml 2026-03-09 17:36:51,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:36:51,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:36:51,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c4ce46d8ec186ac6d0/lava_callback.json.gz DEBUG: Setup failed for 69af01c4ce46d8ec186ac6d0. Transitting job node result: incomplete -> fail 2026-03-09 17:36:51,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:36:51,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c4ce46d8ec186ac6d0 HTTP/1.1" 200 None 2026-03-09 17:36:51,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:36:51,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:36:51,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:36:51,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c4ce46d8ec186ac6d0 HTTP/1.1" 200 None 2026-03-09 17:36:51,885 - lava_callback - INFO - Completed processing callback for node 69af01c4ce46d8ec186ac6d0 INFO: 172.21.0.1:60560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60560 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:37:13,886 - lava_callback - INFO - Received callback for node 69af0528ce46d8ec186ac8b3 INFO: 172.21.0.1:56490 - "POST /node/69af0528ce46d8ec186ac8b3 HTTP/1.0" 202 Accepted 2026-03-09 17:37:13,891 - lava_callback - INFO - Starting async job submit for node 69af0528ce46d8ec186ac8b3 2026-03-09 17:37:13,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:37:13,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0528ce46d8ec186ac8b3 HTTP/1.1" 200 None 2026-03-09 17:37:13,909 - lava_callback - DEBUG - Processing node 69af0528ce46d8ec186ac8b3: baseline-arm64 2026-03-09 17:37:13,914 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:37:13,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:37:14,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:37:14,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0528ce46d8ec186ac8b3/lava-logs.yaml 2026-03-09 17:37:14,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:37:14,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:37:14,214 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0528ce46d8ec186ac8b3/lava_callback.json.gz 2026-03-09 17:37:14,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:37:14,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0528ce46d8ec186ac8b3 HTTP/1.1" 200 None 2026-03-09 17:37:14,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:37:14,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:37:14,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:37:14,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0528ce46d8ec186ac8b3 HTTP/1.1" 200 None 2026-03-09 17:37:14,283 - lava_callback - INFO - Completed processing callback for node 69af0528ce46d8ec186ac8b3 2026-03-09 17:37:14,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:37:14,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52258 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:38:22,664 - lava_callback - INFO - Received callback for node 69af0156ce46d8ec186ac5c9 INFO: 172.21.0.1:51472 - "POST /node/69af0156ce46d8ec186ac5c9 HTTP/1.0" 202 Accepted 2026-03-09 17:38:22,670 - lava_callback - INFO - Starting async job submit for node 69af0156ce46d8ec186ac5c9 2026-03-09 17:38:22,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:22,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0156ce46d8ec186ac5c9 HTTP/1.1" 200 None 2026-03-09 17:38:22,738 - lava_callback - DEBUG - Processing node 69af0156ce46d8ec186ac5c9: baseline-arm 2026-03-09 17:38:23,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:23,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:23,484 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af0156ce46d8ec186ac5c9/log.txt.gz 2026-03-09 17:38:23,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:24,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:24,084 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af0156ce46d8ec186ac5c9/lava-logs.yaml 2026-03-09 17:38:24,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:24,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:24,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af0156ce46d8ec186ac5c9/lava_callback.json.gz DEBUG: Setup failed for 69af0156ce46d8ec186ac5c9. Transitting job node result: incomplete -> fail 2026-03-09 17:38:24,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:24,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0156ce46d8ec186ac5c9 HTTP/1.1" 200 None 2026-03-09 17:38:24,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:24,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:38:24,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:24,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0156ce46d8ec186ac5c9 HTTP/1.1" 200 None 2026-03-09 17:38:24,616 - lava_callback - INFO - Completed processing callback for node 69af0156ce46d8ec186ac5c9 INFO: 172.21.0.1:51484 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:38:30,821 - lava_callback - INFO - Received callback for node 69af0194ce46d8ec186ac61f INFO: 172.21.0.1:52838 - "POST /node/69af0194ce46d8ec186ac61f HTTP/1.0" 202 Accepted 2026-03-09 17:38:30,827 - lava_callback - INFO - Starting async job submit for node 69af0194ce46d8ec186ac61f 2026-03-09 17:38:30,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:30,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0194ce46d8ec186ac61f HTTP/1.1" 200 None 2026-03-09 17:38:30,858 - lava_callback - DEBUG - Processing node 69af0194ce46d8ec186ac61f: baseline-arm64 2026-03-09 17:38:31,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:31,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:31,341 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0194ce46d8ec186ac61f/log.txt.gz 2026-03-09 17:38:31,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:31,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:31,754 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0194ce46d8ec186ac61f/lava-logs.yaml 2026-03-09 17:38:31,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:31,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:31,917 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0194ce46d8ec186ac61f/lava_callback.json.gz 2026-03-09 17:38:31,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:31,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0194ce46d8ec186ac61f HTTP/1.1" 200 None 2026-03-09 17:38:31,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:31,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:38:31,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:31,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0194ce46d8ec186ac61f HTTP/1.1" 200 None 2026-03-09 17:38:31,998 - lava_callback - INFO - Completed processing callback for node 69af0194ce46d8ec186ac61f INFO: 172.21.0.1:52850 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:38:44,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:44,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59968 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:39:38,272 - lava_callback - INFO - Received callback for node 69af05bdce46d8ec186ac8de INFO: 172.21.0.1:59970 - "POST /node/69af05bdce46d8ec186ac8de HTTP/1.0" 202 Accepted 2026-03-09 17:39:38,278 - lava_callback - INFO - Starting async job submit for node 69af05bdce46d8ec186ac8de 2026-03-09 17:39:38,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:39:38,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af05bdce46d8ec186ac8de HTTP/1.1" 200 None 2026-03-09 17:39:38,317 - lava_callback - DEBUG - Processing node 69af05bdce46d8ec186ac8de: baseline-arm64 2026-03-09 17:39:38,366 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:39:38,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:39:38,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:39:38,573 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af05bdce46d8ec186ac8de/lava-logs.yaml 2026-03-09 17:39:38,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:39:38,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:39:38,709 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af05bdce46d8ec186ac8de/lava_callback.json.gz 2026-03-09 17:39:38,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:39:38,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af05bdce46d8ec186ac8de HTTP/1.1" 200 None 2026-03-09 17:39:38,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:39:38,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:39:38,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:39:38,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af05bdce46d8ec186ac8de HTTP/1.1" 200 None 2026-03-09 17:39:38,844 - lava_callback - INFO - Completed processing callback for node 69af05bdce46d8ec186ac8de 2026-03-09 17:39:44,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:39:44,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54722 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:40:08,386 - lava_callback - INFO - Received callback for node 69af01a4ce46d8ec186ac662 INFO: 172.21.0.1:54726 - "POST /node/69af01a4ce46d8ec186ac662 HTTP/1.0" 202 Accepted 2026-03-09 17:40:08,392 - lava_callback - INFO - Starting async job submit for node 69af01a4ce46d8ec186ac662 2026-03-09 17:40:08,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:08,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a4ce46d8ec186ac662 HTTP/1.1" 200 None 2026-03-09 17:40:08,430 - lava_callback - DEBUG - Processing node 69af01a4ce46d8ec186ac662: baseline-nfs-arm64 2026-03-09 17:40:08,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:40:09,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:40:09,019 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a4ce46d8ec186ac662/log.txt.gz 2026-03-09 17:40:09,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:40:09,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:40:09,466 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a4ce46d8ec186ac662/lava-logs.yaml 2026-03-09 17:40:09,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:40:09,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:40:09,638 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a4ce46d8ec186ac662/lava_callback.json.gz 2026-03-09 17:40:09,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:09,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a4ce46d8ec186ac662 HTTP/1.1" 200 None 2026-03-09 17:40:09,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:09,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:40:09,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:09,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a4ce46d8ec186ac662 HTTP/1.1" 200 None 2026-03-09 17:40:09,714 - lava_callback - INFO - Completed processing callback for node 69af01a4ce46d8ec186ac662 2026-03-09 17:40:14,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:14,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50184 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:40:55,328 - lava_callback - INFO - Received callback for node 69af0614ce46d8ec186ac8f3 INFO: 172.21.0.1:50582 - "POST /node/69af0614ce46d8ec186ac8f3 HTTP/1.0" 202 Accepted 2026-03-09 17:40:55,334 - lava_callback - INFO - Starting async job submit for node 69af0614ce46d8ec186ac8f3 2026-03-09 17:40:55,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:55,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0614ce46d8ec186ac8f3 HTTP/1.1" 200 None 2026-03-09 17:40:55,368 - lava_callback - DEBUG - Processing node 69af0614ce46d8ec186ac8f3: baseline-nfs-arm64 2026-03-09 17:40:55,369 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:40:55,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:40:55,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:40:55,519 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0614ce46d8ec186ac8f3/lava-logs.yaml 2026-03-09 17:40:55,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:40:55,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:40:55,666 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0614ce46d8ec186ac8f3/lava_callback.json.gz 2026-03-09 17:40:55,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:55,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0614ce46d8ec186ac8f3 HTTP/1.1" 200 None 2026-03-09 17:40:55,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:55,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:40:55,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:55,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0614ce46d8ec186ac8f3 HTTP/1.1" 200 None 2026-03-09 17:40:55,734 - lava_callback - INFO - Completed processing callback for node 69af0614ce46d8ec186ac8f3 INFO: 172.21.0.1:50586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55740 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:41:14,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:41:14,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41236 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:42:26,304 - lava_callback - INFO - Received callback for node 69af065ece46d8ec186ac8fb INFO: 172.21.0.1:58138 - "POST /node/69af065ece46d8ec186ac8fb HTTP/1.0" 202 Accepted 2026-03-09 17:42:26,310 - lava_callback - INFO - Starting async job submit for node 69af065ece46d8ec186ac8fb 2026-03-09 17:42:26,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:42:26,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af065ece46d8ec186ac8fb HTTP/1.1" 200 None 2026-03-09 17:42:26,347 - lava_callback - DEBUG - Processing node 69af065ece46d8ec186ac8fb: baseline-arm64 2026-03-09 17:42:26,398 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:42:26,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:58144 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:42:26,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:42:26,601 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af065ece46d8ec186ac8fb/lava-logs.yaml 2026-03-09 17:42:26,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:42:26,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:42:26,745 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af065ece46d8ec186ac8fb/lava_callback.json.gz 2026-03-09 17:42:26,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:42:26,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af065ece46d8ec186ac8fb HTTP/1.1" 200 None 2026-03-09 17:42:26,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:42:26,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:42:26,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:42:26,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af065ece46d8ec186ac8fb HTTP/1.1" 200 None 2026-03-09 17:42:26,863 - lava_callback - INFO - Completed processing callback for node 69af065ece46d8ec186ac8fb INFO: 172.21.0.1:37042 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:42:44,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:42:44,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36764 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:43:39,526 - lava_callback - INFO - Received callback for node 69af06a7ce46d8ec186ac906 INFO: 172.21.0.1:36766 - "POST /node/69af06a7ce46d8ec186ac906 HTTP/1.0" 202 Accepted 2026-03-09 17:43:39,533 - lava_callback - INFO - Starting async job submit for node 69af06a7ce46d8ec186ac906 2026-03-09 17:43:39,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:43:39,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af06a7ce46d8ec186ac906 HTTP/1.1" 200 None 2026-03-09 17:43:39,565 - lava_callback - DEBUG - Processing node 69af06a7ce46d8ec186ac906: baseline-arm64 2026-03-09 17:43:39,613 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:43:39,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:43:39,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:43:39,852 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af06a7ce46d8ec186ac906/lava-logs.yaml 2026-03-09 17:43:39,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:43:39,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:43:39,995 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af06a7ce46d8ec186ac906/lava_callback.json.gz 2026-03-09 17:43:40,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:43:40,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af06a7ce46d8ec186ac906 HTTP/1.1" 200 None 2026-03-09 17:43:40,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:43:40,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:43:40,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:43:40,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af06a7ce46d8ec186ac906 HTTP/1.1" 200 None 2026-03-09 17:43:40,146 - lava_callback - INFO - Completed processing callback for node 69af06a7ce46d8ec186ac906 2026-03-09 17:43:44,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:43:44,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60784 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:44:07,287 - lava_callback - INFO - Received callback for node 69ac63dc46f7ae8ff3365c96 INFO: 172.21.0.1:60792 - "POST /node/69ac63dc46f7ae8ff3365c96 HTTP/1.0" 202 Accepted 2026-03-09 17:44:07,293 - lava_callback - INFO - Starting async job submit for node 69ac63dc46f7ae8ff3365c96 2026-03-09 17:44:07,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:07,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac63dc46f7ae8ff3365c96 HTTP/1.1" 200 None 2026-03-09 17:44:07,308 - lava_callback - DEBUG - Processing node 69ac63dc46f7ae8ff3365c96: baseline-arm-baylibre 2026-03-09 17:44:07,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac63dc46f7ae8ff3365c96 2026-03-09 17:44:07,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:07,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac63dc46f7ae8ff3365c96 HTTP/1.1" 200 None 2026-03-09 17:44:07,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:07,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:44:07,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:07,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac63dc46f7ae8ff3365c96 HTTP/1.1" 200 None 2026-03-09 17:44:07,386 - lava_callback - ERROR - Error processing callback for node 69ac63dc46f7ae8ff3365c96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42798 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:44:47,264 - lava_callback - INFO - Received callback for node 69ac640046f7ae8ff3365ca8 INFO: 172.21.0.1:42800 - "POST /node/69ac640046f7ae8ff3365ca8 HTTP/1.0" 202 Accepted 2026-03-09 17:44:47,269 - lava_callback - INFO - Starting async job submit for node 69ac640046f7ae8ff3365ca8 2026-03-09 17:44:47,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:47,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac640046f7ae8ff3365ca8 HTTP/1.1" 200 None 2026-03-09 17:44:47,286 - lava_callback - DEBUG - Processing node 69ac640046f7ae8ff3365ca8: baseline-arm-baylibre 2026-03-09 17:44:47,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac640046f7ae8ff3365ca8 2026-03-09 17:44:47,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:47,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac640046f7ae8ff3365ca8 HTTP/1.1" 200 None 2026-03-09 17:44:47,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:47,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:44:47,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:47,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac640046f7ae8ff3365ca8 HTTP/1.1" 200 None 2026-03-09 17:44:47,358 - lava_callback - ERROR - Error processing callback for node 69ac640046f7ae8ff3365ca8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 17:44:54,986 - lava_callback - INFO - Received callback for node 69af06fcce46d8ec186ac913 INFO: 172.21.0.1:41768 - "POST /node/69af06fcce46d8ec186ac913 HTTP/1.0" 202 Accepted 2026-03-09 17:44:54,992 - lava_callback - INFO - Starting async job submit for node 69af06fcce46d8ec186ac913 2026-03-09 17:44:54,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:55,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af06fcce46d8ec186ac913 HTTP/1.1" 200 None 2026-03-09 17:44:55,010 - lava_callback - DEBUG - Processing node 69af06fcce46d8ec186ac913: baseline-nfs-arm64 2026-03-09 17:44:55,016 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:44:55,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:44:55,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:44:55,218 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af06fcce46d8ec186ac913/lava-logs.yaml 2026-03-09 17:44:55,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:44:55,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:44:55,351 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af06fcce46d8ec186ac913/lava_callback.json.gz 2026-03-09 17:44:55,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:55,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af06fcce46d8ec186ac913 HTTP/1.1" 200 None 2026-03-09 17:44:55,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:55,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:44:55,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:55,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af06fcce46d8ec186ac913 HTTP/1.1" 200 None 2026-03-09 17:44:55,440 - lava_callback - INFO - Completed processing callback for node 69af06fcce46d8ec186ac913 INFO: 172.21.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40234 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:45:14,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:15,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33970 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:45:27,377 - lava_callback - INFO - Received callback for node 69ac642546f7ae8ff3365cb9 INFO: 172.21.0.1:33980 - "POST /node/69ac642546f7ae8ff3365cb9 HTTP/1.0" 202 Accepted 2026-03-09 17:45:27,383 - lava_callback - INFO - Starting async job submit for node 69ac642546f7ae8ff3365cb9 2026-03-09 17:45:27,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:27,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac642546f7ae8ff3365cb9 HTTP/1.1" 200 None 2026-03-09 17:45:27,399 - lava_callback - DEBUG - Processing node 69ac642546f7ae8ff3365cb9: baseline-arm-baylibre 2026-03-09 17:45:27,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac642546f7ae8ff3365cb9 2026-03-09 17:45:27,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:27,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac642546f7ae8ff3365cb9 HTTP/1.1" 200 None 2026-03-09 17:45:27,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:27,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:45:27,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:27,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac642546f7ae8ff3365cb9 HTTP/1.1" 200 None 2026-03-09 17:45:27,489 - lava_callback - ERROR - Error processing callback for node 69ac642546f7ae8ff3365cb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37840 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:45:47,354 - lava_callback - INFO - Received callback for node 69ac644846f7ae8ff3365cc7 INFO: 172.21.0.1:37854 - "POST /node/69ac644846f7ae8ff3365cc7 HTTP/1.0" 202 Accepted 2026-03-09 17:45:47,359 - lava_callback - INFO - Starting async job submit for node 69ac644846f7ae8ff3365cc7 2026-03-09 17:45:47,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:47,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac644846f7ae8ff3365cc7 HTTP/1.1" 200 None 2026-03-09 17:45:47,380 - lava_callback - DEBUG - Processing node 69ac644846f7ae8ff3365cc7: baseline-arm-baylibre 2026-03-09 17:45:47,380 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac644846f7ae8ff3365cc7 2026-03-09 17:45:47,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:47,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac644846f7ae8ff3365cc7 HTTP/1.1" 200 None 2026-03-09 17:45:47,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:47,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:45:47,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:47,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac644846f7ae8ff3365cc7 HTTP/1.1" 200 None 2026-03-09 17:45:47,465 - lava_callback - ERROR - Error processing callback for node 69ac644846f7ae8ff3365cc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50780 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:46:11,694 - lava_callback - INFO - Received callback for node 69af0757ce46d8ec186ac928 INFO: 172.21.0.1:33788 - "POST /node/69af0757ce46d8ec186ac928 HTTP/1.0" 202 Accepted 2026-03-09 17:46:11,700 - lava_callback - INFO - Starting async job submit for node 69af0757ce46d8ec186ac928 2026-03-09 17:46:11,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:11,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0757ce46d8ec186ac928 HTTP/1.1" 200 None 2026-03-09 17:46:11,753 - lava_callback - DEBUG - Processing node 69af0757ce46d8ec186ac928: baseline-arm64-foundriesio 2026-03-09 17:46:11,804 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:46:11,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:46:12,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:46:12,002 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af0757ce46d8ec186ac928/lava-logs.yaml 2026-03-09 17:46:12,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:46:12,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:46:12,129 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af0757ce46d8ec186ac928/lava_callback.json.gz 2026-03-09 17:46:12,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:12,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0757ce46d8ec186ac928 HTTP/1.1" 200 None 2026-03-09 17:46:12,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:12,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:46:12,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:12,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0757ce46d8ec186ac928 HTTP/1.1" 200 None 2026-03-09 17:46:12,251 - lava_callback - INFO - Completed processing callback for node 69af0757ce46d8ec186ac928 2026-03-09 17:46:15,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:15,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33800 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:46:23,831 - lava_callback - INFO - Received callback for node 69af01c6ce46d8ec186ac6db INFO: 172.21.0.1:45726 - "POST /node/69af01c6ce46d8ec186ac6db HTTP/1.0" 202 Accepted 2026-03-09 17:46:23,837 - lava_callback - INFO - Starting async job submit for node 69af01c6ce46d8ec186ac6db 2026-03-09 17:46:23,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:23,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c6ce46d8ec186ac6db HTTP/1.1" 200 None 2026-03-09 17:46:23,890 - lava_callback - DEBUG - Processing node 69af01c6ce46d8ec186ac6db: baseline-arm64-broonie 2026-03-09 17:46:24,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:46:24,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:46:24,319 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c6ce46d8ec186ac6db/log.txt.gz 2026-03-09 17:46:24,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:46:24,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:46:24,689 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c6ce46d8ec186ac6db/lava-logs.yaml 2026-03-09 17:46:24,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:46:24,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:46:24,844 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c6ce46d8ec186ac6db/lava_callback.json.gz DEBUG: Setup failed for 69af01c6ce46d8ec186ac6db. Transitting job node result: incomplete -> fail 2026-03-09 17:46:24,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:24,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c6ce46d8ec186ac6db HTTP/1.1" 200 None 2026-03-09 17:46:24,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:24,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:46:24,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:24,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c6ce46d8ec186ac6db HTTP/1.1" 200 None 2026-03-09 17:46:25,079 - lava_callback - INFO - Completed processing callback for node 69af01c6ce46d8ec186ac6db INFO: 172.21.0.1:45732 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:46:27,381 - lava_callback - INFO - Received callback for node 69ac646d46f7ae8ff3365cd5 INFO: 172.21.0.1:45736 - "POST /node/69ac646d46f7ae8ff3365cd5 HTTP/1.0" 202 Accepted 2026-03-09 17:46:27,386 - lava_callback - INFO - Starting async job submit for node 69ac646d46f7ae8ff3365cd5 2026-03-09 17:46:27,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:27,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac646d46f7ae8ff3365cd5 HTTP/1.1" 200 None 2026-03-09 17:46:27,415 - lava_callback - DEBUG - Processing node 69ac646d46f7ae8ff3365cd5: baseline-arm-baylibre 2026-03-09 17:46:27,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac646d46f7ae8ff3365cd5 2026-03-09 17:46:27,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:27,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac646d46f7ae8ff3365cd5 HTTP/1.1" 200 None 2026-03-09 17:46:27,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:27,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:46:27,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:27,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac646d46f7ae8ff3365cd5 HTTP/1.1" 200 None 2026-03-09 17:46:27,491 - lava_callback - ERROR - Error processing callback for node 69ac646d46f7ae8ff3365cd5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44738 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:46:45,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:45,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43914 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:47:07,426 - lava_callback - INFO - Received callback for node 69ac649046f7ae8ff3365ceb INFO: 172.21.0.1:43922 - "POST /node/69ac649046f7ae8ff3365ceb HTTP/1.0" 202 Accepted 2026-03-09 17:47:07,432 - lava_callback - INFO - Starting async job submit for node 69ac649046f7ae8ff3365ceb 2026-03-09 17:47:07,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:07,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac649046f7ae8ff3365ceb HTTP/1.1" 200 None 2026-03-09 17:47:07,448 - lava_callback - DEBUG - Processing node 69ac649046f7ae8ff3365ceb: baseline-arm-baylibre 2026-03-09 17:47:07,448 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac649046f7ae8ff3365ceb 2026-03-09 17:47:07,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:07,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac649046f7ae8ff3365ceb HTTP/1.1" 200 None 2026-03-09 17:47:07,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:07,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:47:07,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:07,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac649046f7ae8ff3365ceb HTTP/1.1" 200 None 2026-03-09 17:47:07,509 - lava_callback - ERROR - Error processing callback for node 69ac649046f7ae8ff3365ceb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57642 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:47:24,145 - lava_callback - INFO - Received callback for node 69af01bece46d8ec186ac6c2 INFO: 172.21.0.1:33466 - "POST /node/69af01bece46d8ec186ac6c2 HTTP/1.0" 202 Accepted 2026-03-09 17:47:24,151 - lava_callback - INFO - Starting async job submit for node 69af01bece46d8ec186ac6c2 2026-03-09 17:47:24,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:24,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01bece46d8ec186ac6c2 HTTP/1.1" 200 None 2026-03-09 17:47:24,202 - lava_callback - DEBUG - Processing node 69af01bece46d8ec186ac6c2: baseline-arm64-broonie 2026-03-09 17:47:24,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:47:24,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:47:24,573 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bece46d8ec186ac6c2/log.txt.gz 2026-03-09 17:47:24,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:47:24,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:47:24,915 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bece46d8ec186ac6c2/lava-logs.yaml 2026-03-09 17:47:24,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:47:25,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:47:25,060 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bece46d8ec186ac6c2/lava_callback.json.gz DEBUG: Setup failed for 69af01bece46d8ec186ac6c2. Transitting job node result: incomplete -> fail 2026-03-09 17:47:25,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:25,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01bece46d8ec186ac6c2 HTTP/1.1" 200 None 2026-03-09 17:47:25,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:25,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:47:25,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:25,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01bece46d8ec186ac6c2 HTTP/1.1" 200 None 2026-03-09 17:47:25,292 - lava_callback - INFO - Completed processing callback for node 69af01bece46d8ec186ac6c2 INFO: 172.21.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44062 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:47:37,673 - lava_callback - INFO - Received callback for node 69af079fce46d8ec186ac93f INFO: 172.21.0.1:44070 - "POST /node/69af079fce46d8ec186ac93f HTTP/1.0" 202 Accepted 2026-03-09 17:47:37,678 - lava_callback - INFO - Starting async job submit for node 69af079fce46d8ec186ac93f 2026-03-09 17:47:37,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:37,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af079fce46d8ec186ac93f HTTP/1.1" 200 None 2026-03-09 17:47:37,704 - lava_callback - DEBUG - Processing node 69af079fce46d8ec186ac93f: baseline-arm64-clabbe 2026-03-09 17:47:37,708 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:47:37,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:47:37,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:47:37,848 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af079fce46d8ec186ac93f/lava-logs.yaml 2026-03-09 17:47:37,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:47:37,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:47:37,992 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af079fce46d8ec186ac93f/lava_callback.json.gz 2026-03-09 17:47:37,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:38,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af079fce46d8ec186ac93f HTTP/1.1" 200 None 2026-03-09 17:47:38,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:38,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:47:38,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:38,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af079fce46d8ec186ac93f HTTP/1.1" 200 None 2026-03-09 17:47:38,056 - lava_callback - INFO - Completed processing callback for node 69af079fce46d8ec186ac93f 2026-03-09 17:47:45,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:45,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39526 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:49:03,502 - lava_callback - INFO - Received callback for node 69af07edce46d8ec186ac94d INFO: 172.21.0.1:59434 - "POST /node/69af07edce46d8ec186ac94d HTTP/1.0" 202 Accepted 2026-03-09 17:49:03,508 - lava_callback - INFO - Starting async job submit for node 69af07edce46d8ec186ac94d 2026-03-09 17:49:03,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:49:03,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af07edce46d8ec186ac94d HTTP/1.1" 200 None 2026-03-09 17:49:03,526 - lava_callback - DEBUG - Processing node 69af07edce46d8ec186ac94d: baseline-arm64-broonie 2026-03-09 17:49:03,532 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:49:03,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:49:03,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:49:03,834 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af07edce46d8ec186ac94d/lava-logs.yaml 2026-03-09 17:49:03,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:49:03,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:49:03,983 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af07edce46d8ec186ac94d/lava_callback.json.gz 2026-03-09 17:49:03,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:49:04,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af07edce46d8ec186ac94d HTTP/1.1" 200 None 2026-03-09 17:49:04,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:49:04,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:49:04,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:49:04,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af07edce46d8ec186ac94d HTTP/1.1" 200 None 2026-03-09 17:49:04,053 - lava_callback - INFO - Completed processing callback for node 69af07edce46d8ec186ac94d INFO: 172.21.0.1:59442 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:49:15,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:49:15,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40486 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:51:01,408 - lava_callback - INFO - Received callback for node 69aefa1fce46d8ec186abf59 INFO: 172.21.0.1:33998 - "POST /node/69aefa1fce46d8ec186abf59 HTTP/1.0" 202 Accepted 2026-03-09 17:51:01,457 - lava_callback - INFO - Starting async job submit for node 69aefa1fce46d8ec186abf59 2026-03-09 17:51:01,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:01,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1fce46d8ec186abf59 HTTP/1.1" 200 None 2026-03-09 17:51:01,496 - lava_callback - DEBUG - Processing node 69aefa1fce46d8ec186abf59: baseline-arm-pengutronix 2026-03-09 17:51:01,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:01,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:01,847 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1fce46d8ec186abf59/log.txt.gz 2026-03-09 17:51:01,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:02,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:02,103 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1fce46d8ec186abf59/lava-logs.yaml 2026-03-09 17:51:02,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:02,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:02,251 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1fce46d8ec186abf59/lava_callback.json.gz DEBUG: Setup failed for 69aefa1fce46d8ec186abf59. Transitting job node result: incomplete -> fail 2026-03-09 17:51:02,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:02,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1fce46d8ec186abf59 HTTP/1.1" 200 None 2026-03-09 17:51:02,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:02,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:51:02,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:02,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa1fce46d8ec186abf59 HTTP/1.1" 200 None 2026-03-09 17:51:02,429 - lava_callback - INFO - Completed processing callback for node 69aefa1fce46d8ec186abf59 INFO: 172.21.0.1:34006 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:51:14,798 - lava_callback - INFO - Received callback for node 69af0827ce46d8ec186ac95c INFO: 172.21.0.1:49858 - "POST /node/69af0827ce46d8ec186ac95c HTTP/1.0" 202 Accepted 2026-03-09 17:51:14,804 - lava_callback - INFO - Starting async job submit for node 69af0827ce46d8ec186ac95c 2026-03-09 17:51:14,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:14,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0827ce46d8ec186ac95c HTTP/1.1" 200 None 2026-03-09 17:51:14,871 - lava_callback - DEBUG - Processing node 69af0827ce46d8ec186ac95c: baseline-arm64 2026-03-09 17:51:15,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:49866 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:51:15,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 17:51:15,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:15,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:15,463 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0827ce46d8ec186ac95c/log.txt.gz 2026-03-09 17:51:15,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:15,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:15,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0827ce46d8ec186ac95c/lava-logs.yaml 2026-03-09 17:51:15,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:16,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:16,134 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0827ce46d8ec186ac95c/lava_callback.json.gz DEBUG: Setup failed for 69af0827ce46d8ec186ac95c. Transitting job node result: incomplete -> fail 2026-03-09 17:51:16,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:16,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0827ce46d8ec186ac95c HTTP/1.1" 200 None 2026-03-09 17:51:16,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:16,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:51:16,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:16,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0827ce46d8ec186ac95c HTTP/1.1" 200 None 2026-03-09 17:51:16,335 - lava_callback - INFO - Completed processing callback for node 69af0827ce46d8ec186ac95c INFO: 172.21.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54864 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:51:45,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:45,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54056 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:52:30,929 - lava_callback - INFO - Received callback for node 69af0874ce46d8ec186ac967 INFO: 172.21.0.1:51992 - "POST /node/69af0874ce46d8ec186ac967 HTTP/1.0" 202 Accepted 2026-03-09 17:52:30,936 - lava_callback - INFO - Starting async job submit for node 69af0874ce46d8ec186ac967 2026-03-09 17:52:30,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:52:30,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0874ce46d8ec186ac967 HTTP/1.1" 200 None 2026-03-09 17:52:30,992 - lava_callback - DEBUG - Processing node 69af0874ce46d8ec186ac967: baseline-arm64 2026-03-09 17:52:31,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:52:31,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:52:31,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0874ce46d8ec186ac967/log.txt.gz 2026-03-09 17:52:32,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:52:32,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:52:32,977 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0874ce46d8ec186ac967/lava-logs.yaml 2026-03-09 17:52:32,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:52:33,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:52:33,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0874ce46d8ec186ac967/lava_callback.json.gz DEBUG: Setup failed for 69af0874ce46d8ec186ac967. Transitting job node result: incomplete -> fail 2026-03-09 17:52:33,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:52:33,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0874ce46d8ec186ac967 HTTP/1.1" 200 None 2026-03-09 17:52:33,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:52:33,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:52:33,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:52:33,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0874ce46d8ec186ac967 HTTP/1.1" 200 None 2026-03-09 17:52:33,402 - lava_callback - INFO - Completed processing callback for node 69af0874ce46d8ec186ac967 INFO: 172.21.0.1:52004 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:52:45,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:52:45,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:53:47,849 - lava_callback - INFO - Received callback for node 69ac661f46f7ae8ff3365d72 INFO: 172.21.0.1:46876 - "POST /node/69ac661f46f7ae8ff3365d72 HTTP/1.0" 202 Accepted 2026-03-09 17:53:47,854 - lava_callback - INFO - Starting async job submit for node 69ac661f46f7ae8ff3365d72 2026-03-09 17:53:47,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:53:47,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac661f46f7ae8ff3365d72 HTTP/1.1" 200 None 2026-03-09 17:53:47,870 - lava_callback - DEBUG - Processing node 69ac661f46f7ae8ff3365d72: baseline-arm64-baylibre 2026-03-09 17:53:47,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac661f46f7ae8ff3365d72 2026-03-09 17:53:47,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:53:47,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac661f46f7ae8ff3365d72 HTTP/1.1" 200 None 2026-03-09 17:53:47,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:53:47,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 17:53:47,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:53:47,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac661f46f7ae8ff3365d72 HTTP/1.1" 200 None 2026-03-09 17:53:47,971 - lava_callback - ERROR - Error processing callback for node 69ac661f46f7ae8ff3365d72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40476 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:54:14,523 - lava_callback - INFO - Received callback for node 69af08c7ce46d8ec186ac97d INFO: 172.21.0.1:41634 - "POST /node/69af08c7ce46d8ec186ac97d HTTP/1.0" 202 Accepted 2026-03-09 17:54:14,529 - lava_callback - INFO - Starting async job submit for node 69af08c7ce46d8ec186ac97d 2026-03-09 17:54:14,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:54:14,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af08c7ce46d8ec186ac97d HTTP/1.1" 200 None 2026-03-09 17:54:14,579 - lava_callback - DEBUG - Processing node 69af08c7ce46d8ec186ac97d: baseline-arm64 2026-03-09 17:54:14,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:54:15,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:54:15,102 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af08c7ce46d8ec186ac97d/log.txt.gz INFO: 172.21.0.1:41640 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:54:15,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:54:15,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:54:15,580 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af08c7ce46d8ec186ac97d/lava-logs.yaml 2026-03-09 17:54:15,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:54:15,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:54:15,747 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af08c7ce46d8ec186ac97d/lava_callback.json.gz DEBUG: Setup failed for 69af08c7ce46d8ec186ac97d. Transitting job node result: incomplete -> fail 2026-03-09 17:54:15,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:54:15,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af08c7ce46d8ec186ac97d HTTP/1.1" 200 None 2026-03-09 17:54:15,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:54:15,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:54:15,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:54:15,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af08c7ce46d8ec186ac97d HTTP/1.1" 200 None 2026-03-09 17:54:15,943 - lava_callback - INFO - Completed processing callback for node 69af08c7ce46d8ec186ac97d INFO: 172.21.0.1:41654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56534 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:54:45,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:54:45,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56950 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:55:13,015 - lava_callback - INFO - Received callback for node 69af0913ce46d8ec186ac988 INFO: 172.21.0.1:58140 - "POST /node/69af0913ce46d8ec186ac988 HTTP/1.0" 202 Accepted 2026-03-09 17:55:13,022 - lava_callback - INFO - Starting async job submit for node 69af0913ce46d8ec186ac988 2026-03-09 17:55:13,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:13,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0913ce46d8ec186ac988 HTTP/1.1" 200 None 2026-03-09 17:55:13,079 - lava_callback - DEBUG - Processing node 69af0913ce46d8ec186ac988: baseline-arm64 2026-03-09 17:55:13,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:55:13,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:55:13,536 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0913ce46d8ec186ac988/log.txt.gz 2026-03-09 17:55:13,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:55:13,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:55:13,909 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0913ce46d8ec186ac988/lava-logs.yaml 2026-03-09 17:55:13,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:55:14,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:55:14,072 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0913ce46d8ec186ac988/lava_callback.json.gz DEBUG: Setup failed for 69af0913ce46d8ec186ac988. Transitting job node result: incomplete -> fail 2026-03-09 17:55:14,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:14,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0913ce46d8ec186ac988 HTTP/1.1" 200 None 2026-03-09 17:55:14,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:14,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:55:14,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:14,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0913ce46d8ec186ac988 HTTP/1.1" 200 None 2026-03-09 17:55:14,270 - lava_callback - INFO - Completed processing callback for node 69af0913ce46d8ec186ac988 INFO: 172.21.0.1:58154 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:55:15,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:15,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45934 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:55:27,883 - lava_callback - INFO - Received callback for node 69ac668d46f7ae8ff3365dae INFO: 172.21.0.1:45946 - "POST /node/69ac668d46f7ae8ff3365dae HTTP/1.0" 202 Accepted 2026-03-09 17:55:27,888 - lava_callback - INFO - Starting async job submit for node 69ac668d46f7ae8ff3365dae 2026-03-09 17:55:27,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:27,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac668d46f7ae8ff3365dae HTTP/1.1" 200 None 2026-03-09 17:55:27,904 - lava_callback - DEBUG - Processing node 69ac668d46f7ae8ff3365dae: baseline-arm64-baylibre 2026-03-09 17:55:27,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac668d46f7ae8ff3365dae 2026-03-09 17:55:27,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:27,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac668d46f7ae8ff3365dae HTTP/1.1" 200 None 2026-03-09 17:55:27,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:27,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 17:55:27,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:27,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac668d46f7ae8ff3365dae HTTP/1.1" 200 None 2026-03-09 17:55:27,963 - lava_callback - ERROR - Error processing callback for node 69ac668d46f7ae8ff3365dae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51280 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:56:11,362 - lava_callback - INFO - Received callback for node 69af095ece46d8ec186ac997 INFO: 172.21.0.1:58028 - "POST /node/69af095ece46d8ec186ac997 HTTP/1.0" 202 Accepted 2026-03-09 17:56:11,376 - lava_callback - INFO - Starting async job submit for node 69af095ece46d8ec186ac997 2026-03-09 17:56:11,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:56:11,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af095ece46d8ec186ac997 HTTP/1.1" 200 None 2026-03-09 17:56:11,464 - lava_callback - DEBUG - Processing node 69af095ece46d8ec186ac997: baseline-arm64 2026-03-09 17:56:12,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:56:12,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:56:12,312 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af095ece46d8ec186ac997/log.txt.gz 2026-03-09 17:56:12,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:56:12,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:56:12,945 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af095ece46d8ec186ac997/lava-logs.yaml 2026-03-09 17:56:12,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:56:13,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:56:13,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af095ece46d8ec186ac997/lava_callback.json.gz DEBUG: Setup failed for 69af095ece46d8ec186ac997. Transitting job node result: incomplete -> fail 2026-03-09 17:56:13,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:56:13,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af095ece46d8ec186ac997 HTTP/1.1" 200 None 2026-03-09 17:56:13,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:56:13,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:56:13,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:56:13,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af095ece46d8ec186ac997 HTTP/1.1" 200 None 2026-03-09 17:56:13,384 - lava_callback - INFO - Completed processing callback for node 69af095ece46d8ec186ac997 INFO: 172.21.0.1:58036 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:56:15,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:56:15,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37740 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:57:42,325 - lava_callback - INFO - Received callback for node 69af09b0ce46d8ec186ac9a6 INFO: 172.21.0.1:53520 - "POST /node/69af09b0ce46d8ec186ac9a6 HTTP/1.0" 202 Accepted 2026-03-09 17:57:42,335 - lava_callback - INFO - Starting async job submit for node 69af09b0ce46d8ec186ac9a6 2026-03-09 17:57:42,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:57:42,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af09b0ce46d8ec186ac9a6 HTTP/1.1" 200 None 2026-03-09 17:57:42,398 - lava_callback - DEBUG - Processing node 69af09b0ce46d8ec186ac9a6: baseline-nfs-arm64 2026-03-09 17:57:42,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:57:43,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:57:43,226 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af09b0ce46d8ec186ac9a6/log.txt.gz 2026-03-09 17:57:43,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:57:43,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:57:43,731 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af09b0ce46d8ec186ac9a6/lava-logs.yaml 2026-03-09 17:57:43,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:57:43,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:57:43,917 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af09b0ce46d8ec186ac9a6/lava_callback.json.gz DEBUG: Setup failed for 69af09b0ce46d8ec186ac9a6. Transitting job node result: incomplete -> fail 2026-03-09 17:57:43,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:57:44,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af09b0ce46d8ec186ac9a6 HTTP/1.1" 200 None 2026-03-09 17:57:44,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:57:44,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:57:44,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:57:44,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af09b0ce46d8ec186ac9a6 HTTP/1.1" 200 None 2026-03-09 17:57:44,234 - lava_callback - INFO - Completed processing callback for node 69af09b0ce46d8ec186ac9a6 2026-03-09 17:57:45,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:57:45,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37682 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:58:59,919 - lava_callback - INFO - Received callback for node 69af0a0fce46d8ec186ac9b8 INFO: 172.21.0.1:59310 - "POST /node/69af0a0fce46d8ec186ac9b8 HTTP/1.0" 202 Accepted 2026-03-09 17:58:59,925 - lava_callback - INFO - Starting async job submit for node 69af0a0fce46d8ec186ac9b8 2026-03-09 17:58:59,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:58:59,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a0fce46d8ec186ac9b8 HTTP/1.1" 200 None 2026-03-09 17:58:59,982 - lava_callback - DEBUG - Processing node 69af0a0fce46d8ec186ac9b8: baseline-arm64-foundriesio 2026-03-09 17:59:00,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:59:00,362 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:59:00,364 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af0a0fce46d8ec186ac9b8/log.txt.gz 2026-03-09 17:59:00,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:59:00,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:59:00,753 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af0a0fce46d8ec186ac9b8/lava-logs.yaml 2026-03-09 17:59:00,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:59:00,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:59:00,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af0a0fce46d8ec186ac9b8/lava_callback.json.gz DEBUG: Setup failed for 69af0a0fce46d8ec186ac9b8. Transitting job node result: incomplete -> fail 2026-03-09 17:59:00,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:00,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a0fce46d8ec186ac9b8 HTTP/1.1" 200 None 2026-03-09 17:59:00,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:01,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:59:01,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:01,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0a0fce46d8ec186ac9b8 HTTP/1.1" 200 None 2026-03-09 17:59:01,114 - lava_callback - INFO - Completed processing callback for node 69af0a0fce46d8ec186ac9b8 INFO: 172.21.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:59:15,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:15,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34270 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:59:28,131 - lava_callback - INFO - Received callback for node 69ac677c46f7ae8ff3365df0 INFO: 172.21.0.1:34274 - "POST /node/69ac677c46f7ae8ff3365df0 HTTP/1.0" 202 Accepted 2026-03-09 17:59:28,137 - lava_callback - INFO - Starting async job submit for node 69ac677c46f7ae8ff3365df0 2026-03-09 17:59:28,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:28,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac677c46f7ae8ff3365df0 HTTP/1.1" 200 None 2026-03-09 17:59:28,153 - lava_callback - DEBUG - Processing node 69ac677c46f7ae8ff3365df0: baseline-arm64-baylibre 2026-03-09 17:59:28,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac677c46f7ae8ff3365df0 2026-03-09 17:59:28,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:28,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac677c46f7ae8ff3365df0 HTTP/1.1" 200 None 2026-03-09 17:59:28,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:28,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 17:59:28,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:28,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac677c46f7ae8ff3365df0 HTTP/1.1" 200 None 2026-03-09 17:59:28,209 - lava_callback - ERROR - Error processing callback for node 69ac677c46f7ae8ff3365df0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34604 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:00:15,608 - lava_callback - INFO - Received callback for node 69af0a4dce46d8ec186ac9c7 INFO: 172.21.0.1:34618 - "POST /node/69af0a4dce46d8ec186ac9c7 HTTP/1.0" 202 Accepted 2026-03-09 18:00:15,614 - lava_callback - INFO - Starting async job submit for node 69af0a4dce46d8ec186ac9c7 2026-03-09 18:00:15,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:00:15,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a4dce46d8ec186ac9c7 HTTP/1.1" 200 None 2026-03-09 18:00:15,661 - lava_callback - DEBUG - Processing node 69af0a4dce46d8ec186ac9c7: baseline-nfs-arm64 2026-03-09 18:00:15,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:00:16,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:00:16,141 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a4dce46d8ec186ac9c7/log.txt.gz 2026-03-09 18:00:16,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:34622 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:00:16,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:00:16,550 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a4dce46d8ec186ac9c7/lava-logs.yaml 2026-03-09 18:00:16,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:00:16,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:00:16,717 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a4dce46d8ec186ac9c7/lava_callback.json.gz DEBUG: Setup failed for 69af0a4dce46d8ec186ac9c7. Transitting job node result: incomplete -> fail 2026-03-09 18:00:16,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:00:16,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a4dce46d8ec186ac9c7 HTTP/1.1" 200 None 2026-03-09 18:00:16,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:00:16,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:00:16,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:00:16,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0a4dce46d8ec186ac9c7 HTTP/1.1" 200 None 2026-03-09 18:00:16,913 - lava_callback - INFO - Completed processing callback for node 69af0a4dce46d8ec186ac9c7 INFO: 172.21.0.1:44194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60556 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:00:45,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:00:45,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56594 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:01:36,915 - lava_callback - INFO - Received callback for node 69af0a97ce46d8ec186ac9d4 INFO: 172.21.0.1:56598 - "POST /node/69af0a97ce46d8ec186ac9d4 HTTP/1.0" 202 Accepted 2026-03-09 18:01:36,922 - lava_callback - INFO - Starting async job submit for node 69af0a97ce46d8ec186ac9d4 2026-03-09 18:01:36,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:01:36,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a97ce46d8ec186ac9d4 HTTP/1.1" 200 None 2026-03-09 18:01:36,971 - lava_callback - DEBUG - Processing node 69af0a97ce46d8ec186ac9d4: baseline-nfs-arm64 2026-03-09 18:01:37,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:01:37,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:01:37,972 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a97ce46d8ec186ac9d4/log.txt.gz 2026-03-09 18:01:38,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:01:38,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:01:38,719 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a97ce46d8ec186ac9d4/lava-logs.yaml 2026-03-09 18:01:38,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:01:38,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:01:38,951 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a97ce46d8ec186ac9d4/lava_callback.json.gz DEBUG: Setup failed for 69af0a97ce46d8ec186ac9d4. Transitting job node result: incomplete -> fail 2026-03-09 18:01:39,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:01:39,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a97ce46d8ec186ac9d4 HTTP/1.1" 200 None 2026-03-09 18:01:39,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:01:39,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:01:39,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:01:39,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0a97ce46d8ec186ac9d4 HTTP/1.1" 200 None 2026-03-09 18:01:39,141 - lava_callback - INFO - Completed processing callback for node 69af0a97ce46d8ec186ac9d4 2026-03-09 18:01:45,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:01:45,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45600 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:02:28,268 - lava_callback - INFO - Received callback for node 69ac683146f7ae8ff3365e21 INFO: 172.21.0.1:45602 - "POST /node/69ac683146f7ae8ff3365e21 HTTP/1.0" 202 Accepted 2026-03-09 18:02:28,274 - lava_callback - INFO - Starting async job submit for node 69ac683146f7ae8ff3365e21 2026-03-09 18:02:28,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:02:28,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac683146f7ae8ff3365e21 HTTP/1.1" 200 None 2026-03-09 18:02:28,295 - lava_callback - DEBUG - Processing node 69ac683146f7ae8ff3365e21: baseline-arm64-baylibre 2026-03-09 18:02:28,295 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac683146f7ae8ff3365e21 2026-03-09 18:02:28,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:02:28,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac683146f7ae8ff3365e21 HTTP/1.1" 200 None 2026-03-09 18:02:28,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:02:28,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 18:02:28,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:02:28,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac683146f7ae8ff3365e21 HTTP/1.1" 200 None 2026-03-09 18:02:28,376 - lava_callback - ERROR - Error processing callback for node 69ac683146f7ae8ff3365e21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47332 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:03:12,861 - lava_callback - INFO - Received callback for node 69af0ae4ce46d8ec186ac9e4 INFO: 172.21.0.1:45684 - "POST /node/69af0ae4ce46d8ec186ac9e4 HTTP/1.0" 202 Accepted 2026-03-09 18:03:12,868 - lava_callback - INFO - Starting async job submit for node 69af0ae4ce46d8ec186ac9e4 2026-03-09 18:03:12,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:03:12,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0ae4ce46d8ec186ac9e4 HTTP/1.1" 200 None 2026-03-09 18:03:12,915 - lava_callback - DEBUG - Processing node 69af0ae4ce46d8ec186ac9e4: baseline-nfs-arm64 2026-03-09 18:03:13,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:03:13,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:03:13,663 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0ae4ce46d8ec186ac9e4/log.txt.gz 2026-03-09 18:03:13,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:03:14,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:03:14,217 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0ae4ce46d8ec186ac9e4/lava-logs.yaml 2026-03-09 18:03:14,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:03:14,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:03:14,404 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0ae4ce46d8ec186ac9e4/lava_callback.json.gz DEBUG: Setup failed for 69af0ae4ce46d8ec186ac9e4. Transitting job node result: incomplete -> fail 2026-03-09 18:03:14,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:03:14,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0ae4ce46d8ec186ac9e4 HTTP/1.1" 200 None 2026-03-09 18:03:14,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:03:14,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:03:14,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:03:14,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0ae4ce46d8ec186ac9e4 HTTP/1.1" 200 None 2026-03-09 18:03:14,629 - lava_callback - INFO - Completed processing callback for node 69af0ae4ce46d8ec186ac9e4 INFO: 172.21.0.1:45696 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:03:15,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:03:15,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57054 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:04:24,781 - lava_callback - INFO - Received callback for node 69af0b32ce46d8ec186ac9ef INFO: 172.21.0.1:54618 - "POST /node/69af0b32ce46d8ec186ac9ef HTTP/1.0" 202 Accepted 2026-03-09 18:04:24,787 - lava_callback - INFO - Starting async job submit for node 69af0b32ce46d8ec186ac9ef 2026-03-09 18:04:24,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:04:24,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0b32ce46d8ec186ac9ef HTTP/1.1" 200 None 2026-03-09 18:04:24,839 - lava_callback - DEBUG - Processing node 69af0b32ce46d8ec186ac9ef: baseline-nfs-arm64 2026-03-09 18:04:25,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:04:25,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:04:25,361 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0b32ce46d8ec186ac9ef/log.txt.gz 2026-03-09 18:04:25,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:04:25,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:04:25,821 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0b32ce46d8ec186ac9ef/lava-logs.yaml 2026-03-09 18:04:25,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:04:25,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:04:25,998 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0b32ce46d8ec186ac9ef/lava_callback.json.gz DEBUG: Setup failed for 69af0b32ce46d8ec186ac9ef. Transitting job node result: incomplete -> fail 2026-03-09 18:04:26,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:04:26,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0b32ce46d8ec186ac9ef HTTP/1.1" 200 None 2026-03-09 18:04:26,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:04:26,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:04:26,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:04:26,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0b32ce46d8ec186ac9ef HTTP/1.1" 200 None 2026-03-09 18:04:26,262 - lava_callback - INFO - Completed processing callback for node 69af0b32ce46d8ec186ac9ef INFO: 172.21.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42814 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:04:45,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:04:45,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58048 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:06:48,725 - lava_callback - INFO - Received callback for node 69ac693646f7ae8ff3365e77 INFO: 172.21.0.1:58058 - "POST /node/69ac693646f7ae8ff3365e77 HTTP/1.0" 202 Accepted 2026-03-09 18:06:48,731 - lava_callback - INFO - Starting async job submit for node 69ac693646f7ae8ff3365e77 2026-03-09 18:06:48,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:06:48,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac693646f7ae8ff3365e77 HTTP/1.1" 200 None 2026-03-09 18:06:48,747 - lava_callback - DEBUG - Processing node 69ac693646f7ae8ff3365e77: baseline-arm64-baylibre 2026-03-09 18:06:48,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac693646f7ae8ff3365e77 2026-03-09 18:06:48,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:06:48,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac693646f7ae8ff3365e77 HTTP/1.1" 200 None 2026-03-09 18:06:48,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:06:48,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 18:06:48,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:06:48,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac693646f7ae8ff3365e77 HTTP/1.1" 200 None 2026-03-09 18:06:48,798 - lava_callback - ERROR - Error processing callback for node 69ac693646f7ae8ff3365e77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58592 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:09:16,633 - lava_callback - INFO - Received callback for node 69af0c63ce46d8ec186aca1e INFO: 172.21.0.1:58596 - "POST /node/69af0c63ce46d8ec186aca1e HTTP/1.0" 202 Accepted 2026-03-09 18:09:16,643 - lava_callback - INFO - Starting async job submit for node 69af0c63ce46d8ec186aca1e 2026-03-09 18:09:16,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:16,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0c63ce46d8ec186aca1e HTTP/1.1" 200 None 2026-03-09 18:09:16,713 - lava_callback - DEBUG - Processing node 69af0c63ce46d8ec186aca1e: baseline-nfs-arm64 2026-03-09 18:09:16,911 - lava_callback - INFO - Received callback for node 69af0c0ece46d8ec186aca16 INFO: 172.21.0.1:58600 - "POST /node/69af0c0ece46d8ec186aca16 HTTP/1.0" 202 Accepted 2026-03-09 18:09:16,933 - lava_callback - INFO - Starting async job submit for node 69af0c0ece46d8ec186aca16 2026-03-09 18:09:17,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:17,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0c0ece46d8ec186aca16 HTTP/1.1" 200 None 2026-03-09 18:09:17,165 - lava_callback - DEBUG - Processing node 69af0c0ece46d8ec186aca16: baseline-arm64 2026-03-09 18:09:17,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:17,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:17,649 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0c63ce46d8ec186aca1e/log.txt.gz 2026-03-09 18:09:17,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:17,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:17,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:17,983 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0c0ece46d8ec186aca16/log.txt.gz 2026-03-09 18:09:18,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:18,153 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0c63ce46d8ec186aca1e/lava-logs.yaml 2026-03-09 18:09:18,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:18,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:18,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:18,393 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0c63ce46d8ec186aca1e/lava_callback.json.gz DEBUG: Setup failed for 69af0c63ce46d8ec186aca1e. Transitting job node result: incomplete -> fail 2026-03-09 18:09:18,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:18,465 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0c0ece46d8ec186aca16/lava-logs.yaml 2026-03-09 18:09:18,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:18,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0c63ce46d8ec186aca1e HTTP/1.1" 200 None 2026-03-09 18:09:18,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:09:18,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0c63ce46d8ec186aca1e HTTP/1.1" 200 None 2026-03-09 18:09:18,632 - lava_callback - INFO - Completed processing callback for node 69af0c63ce46d8ec186aca1e 2026-03-09 18:09:18,663 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:18,665 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0c0ece46d8ec186aca16/lava_callback.json.gz DEBUG: Setup failed for 69af0c0ece46d8ec186aca16. Transitting job node result: incomplete -> fail 2026-03-09 18:09:18,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0c0ece46d8ec186aca16 HTTP/1.1" 200 None 2026-03-09 18:09:18,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:09:18,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0c0ece46d8ec186aca16 HTTP/1.1" 200 None 2026-03-09 18:09:18,891 - lava_callback - INFO - Completed processing callback for node 69af0c0ece46d8ec186aca16 INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53558 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:09:45,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:45,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56848 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:11:28,746 - lava_callback - INFO - Received callback for node 69ac6a4f46f7ae8ff3365ed1 INFO: 172.21.0.1:56850 - "POST /node/69ac6a4f46f7ae8ff3365ed1 HTTP/1.0" 202 Accepted 2026-03-09 18:11:28,752 - lava_callback - INFO - Starting async job submit for node 69ac6a4f46f7ae8ff3365ed1 2026-03-09 18:11:28,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:11:28,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac6a4f46f7ae8ff3365ed1 HTTP/1.1" 200 None 2026-03-09 18:11:28,768 - lava_callback - DEBUG - Processing node 69ac6a4f46f7ae8ff3365ed1: baseline-arm64-baylibre 2026-03-09 18:11:28,768 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac6a4f46f7ae8ff3365ed1 2026-03-09 18:11:28,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:11:28,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac6a4f46f7ae8ff3365ed1 HTTP/1.1" 200 None 2026-03-09 18:11:28,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:11:28,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 18:11:28,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:11:28,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac6a4f46f7ae8ff3365ed1 HTTP/1.1" 200 None 2026-03-09 18:11:28,820 - lava_callback - ERROR - Error processing callback for node 69ac6a4f46f7ae8ff3365ed1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43808 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:12:11,323 - lava_callback - INFO - Received callback for node 69af0cabce46d8ec186aca28 INFO: 172.21.0.1:45342 - "POST /node/69af0cabce46d8ec186aca28 HTTP/1.0" 202 Accepted 2026-03-09 18:12:11,330 - lava_callback - INFO - Starting async job submit for node 69af0cabce46d8ec186aca28 2026-03-09 18:12:11,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:11,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0cabce46d8ec186aca28 HTTP/1.1" 200 None 2026-03-09 18:12:11,387 - lava_callback - DEBUG - Processing node 69af0cabce46d8ec186aca28: baseline-nfs-arm64 2026-03-09 18:12:11,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:11,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:11,952 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0cabce46d8ec186aca28/log.txt.gz 2026-03-09 18:12:12,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:12,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:12,440 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0cabce46d8ec186aca28/lava-logs.yaml 2026-03-09 18:12:12,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:12,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:12,586 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0cabce46d8ec186aca28/lava_callback.json.gz DEBUG: Setup failed for 69af0cabce46d8ec186aca28. Transitting job node result: incomplete -> fail 2026-03-09 18:12:12,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:12,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0cabce46d8ec186aca28 HTTP/1.1" 200 None 2026-03-09 18:12:12,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:12,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:12:12,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:12,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0cabce46d8ec186aca28 HTTP/1.1" 200 None 2026-03-09 18:12:12,796 - lava_callback - INFO - Completed processing callback for node 69af0cabce46d8ec186aca28 INFO: 172.21.0.1:45352 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:12:15,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:15,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51698 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:12:36,545 - lava_callback - INFO - Received callback for node 69af0d38ce46d8ec186aca41 INFO: 172.21.0.1:51704 - "POST /node/69af0d38ce46d8ec186aca41 HTTP/1.0" 202 Accepted 2026-03-09 18:12:36,551 - lava_callback - INFO - Starting async job submit for node 69af0d38ce46d8ec186aca41 2026-03-09 18:12:36,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:36,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0d38ce46d8ec186aca41 HTTP/1.1" 200 None 2026-03-09 18:12:36,598 - lava_callback - DEBUG - Processing node 69af0d38ce46d8ec186aca41: baseline-arm64 2026-03-09 18:12:36,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:37,103 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:37,105 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0d38ce46d8ec186aca41/log.txt.gz 2026-03-09 18:12:37,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:37,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:37,551 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0d38ce46d8ec186aca41/lava-logs.yaml 2026-03-09 18:12:37,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:37,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:37,726 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0d38ce46d8ec186aca41/lava_callback.json.gz DEBUG: Setup failed for 69af0d38ce46d8ec186aca41. Transitting job node result: incomplete -> fail 2026-03-09 18:12:37,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:37,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0d38ce46d8ec186aca41 HTTP/1.1" 200 None 2026-03-09 18:12:37,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:37,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:12:37,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:37,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0d38ce46d8ec186aca41 HTTP/1.1" 200 None 2026-03-09 18:12:37,932 - lava_callback - INFO - Completed processing callback for node 69af0d38ce46d8ec186aca41 2026-03-09 18:12:45,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:45,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38018 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:13:32,231 - lava_callback - INFO - Received callback for node 69aefc20ce46d8ec186ac280 INFO: 172.21.0.1:43410 - "POST /node/69aefc20ce46d8ec186ac280 HTTP/1.0" 202 Accepted 2026-03-09 18:13:32,238 - lava_callback - INFO - Starting async job submit for node 69aefc20ce46d8ec186ac280 2026-03-09 18:13:32,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:13:32,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc20ce46d8ec186ac280 HTTP/1.1" 200 None 2026-03-09 18:13:32,293 - lava_callback - DEBUG - Processing node 69aefc20ce46d8ec186ac280: baseline-arm64-broonie 2026-03-09 18:13:32,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:13:32,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:13:32,963 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aefc20ce46d8ec186ac280/log.txt.gz 2026-03-09 18:13:33,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:13:33,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:13:33,482 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aefc20ce46d8ec186ac280/lava-logs.yaml 2026-03-09 18:13:33,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:13:33,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:13:33,663 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aefc20ce46d8ec186ac280/lava_callback.json.gz DEBUG: Setup failed for 69aefc20ce46d8ec186ac280. Transitting job node result: incomplete -> fail 2026-03-09 18:13:33,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:13:33,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc20ce46d8ec186ac280 HTTP/1.1" 200 None 2026-03-09 18:13:33,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:13:33,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-09 18:13:33,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:13:33,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefc20ce46d8ec186ac280 HTTP/1.1" 200 None 2026-03-09 18:13:33,868 - lava_callback - INFO - Completed processing callback for node 69aefc20ce46d8ec186ac280 INFO: 172.21.0.1:43426 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:13:45,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:13:45,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51526 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:14:55,145 - lava_callback - INFO - Received callback for node 69af0e18ce46d8ec186aca70 INFO: 172.21.0.1:50726 - "POST /node/69af0e18ce46d8ec186aca70 HTTP/1.0" 202 Accepted 2026-03-09 18:14:55,151 - lava_callback - INFO - Starting async job submit for node 69af0e18ce46d8ec186aca70 2026-03-09 18:14:55,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:14:55,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0e18ce46d8ec186aca70 HTTP/1.1" 200 None 2026-03-09 18:14:55,169 - lava_callback - DEBUG - Processing node 69af0e18ce46d8ec186aca70: baseline-arm64 2026-03-09 18:14:55,173 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:14:55,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:14:55,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:14:55,356 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0e18ce46d8ec186aca70/lava-logs.yaml 2026-03-09 18:14:55,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:14:55,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:14:55,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0e18ce46d8ec186aca70/lava_callback.json.gz 2026-03-09 18:14:55,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:14:55,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0e18ce46d8ec186aca70 HTTP/1.1" 200 None 2026-03-09 18:14:55,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:14:55,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:14:55,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:14:55,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0e18ce46d8ec186aca70 HTTP/1.1" 200 None 2026-03-09 18:14:55,617 - lava_callback - INFO - Completed processing callback for node 69af0e18ce46d8ec186aca70 INFO: 172.21.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51016 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:15:15,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:15:15,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52018 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:17:25,397 - lava_callback - INFO - Received callback for node 69af0d91ce46d8ec186aca54 INFO: 172.21.0.1:51536 - "POST /node/69af0d91ce46d8ec186aca54 HTTP/1.0" 202 Accepted 2026-03-09 18:17:25,403 - lava_callback - INFO - Starting async job submit for node 69af0d91ce46d8ec186aca54 2026-03-09 18:17:25,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:17:25,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0d91ce46d8ec186aca54 HTTP/1.1" 200 None 2026-03-09 18:17:25,453 - lava_callback - DEBUG - Processing node 69af0d91ce46d8ec186aca54: baseline-arm64-broonie 2026-03-09 18:17:25,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:17:25,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:17:25,869 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0d91ce46d8ec186aca54/log.txt.gz 2026-03-09 18:17:26,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:17:26,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:17:26,228 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0d91ce46d8ec186aca54/lava-logs.yaml 2026-03-09 18:17:26,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:17:26,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:17:26,403 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0d91ce46d8ec186aca54/lava_callback.json.gz DEBUG: Setup failed for 69af0d91ce46d8ec186aca54. Transitting job node result: incomplete -> fail 2026-03-09 18:17:26,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:17:26,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0d91ce46d8ec186aca54 HTTP/1.1" 200 None 2026-03-09 18:17:26,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:51552 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:17:26,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:17:26,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:17:26,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0d91ce46d8ec186aca54 HTTP/1.1" 200 None 2026-03-09 18:17:26,624 - lava_callback - INFO - Completed processing callback for node 69af0d91ce46d8ec186aca54 INFO: 172.21.0.1:50524 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:17:45,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:17:45,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57648 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:19:54,581 - lava_callback - INFO - Received callback for node 69af0f43ce46d8ec186aca9f INFO: 172.21.0.1:45900 - "POST /node/69af0f43ce46d8ec186aca9f HTTP/1.0" 202 Accepted 2026-03-09 18:19:54,587 - lava_callback - INFO - Starting async job submit for node 69af0f43ce46d8ec186aca9f 2026-03-09 18:19:54,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:19:54,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f43ce46d8ec186aca9f HTTP/1.1" 200 None 2026-03-09 18:19:54,610 - lava_callback - DEBUG - Processing node 69af0f43ce46d8ec186aca9f: baseline-arm64 2026-03-09 18:19:54,614 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:19:54,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:19:54,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:19:54,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0f43ce46d8ec186aca9f/lava-logs.yaml 2026-03-09 18:19:54,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:19:54,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:19:54,927 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0f43ce46d8ec186aca9f/lava_callback.json.gz 2026-03-09 18:19:54,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:19:54,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f43ce46d8ec186aca9f HTTP/1.1" 200 None 2026-03-09 18:19:54,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:19:54,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:19:54,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:19:54,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0f43ce46d8ec186aca9f HTTP/1.1" 200 None 2026-03-09 18:19:55,004 - lava_callback - INFO - Completed processing callback for node 69af0f43ce46d8ec186aca9f INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45262 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:20:11,961 - lava_callback - INFO - Received callback for node 69af01c8ce46d8ec186ac6e0 INFO: 172.21.0.1:51132 - "POST /node/69af01c8ce46d8ec186ac6e0 HTTP/1.0" 202 Accepted 2026-03-09 18:20:11,967 - lava_callback - INFO - Starting async job submit for node 69af01c8ce46d8ec186ac6e0 2026-03-09 18:20:12,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:20:12,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c8ce46d8ec186ac6e0 HTTP/1.1" 200 None 2026-03-09 18:20:12,017 - lava_callback - DEBUG - Processing node 69af01c8ce46d8ec186ac6e0: baseline-arm64-broonie 2026-03-09 18:20:12,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:20:12,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:20:12,530 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c8ce46d8ec186ac6e0/log.txt.gz 2026-03-09 18:20:12,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:20:12,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:20:12,988 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c8ce46d8ec186ac6e0/lava-logs.yaml 2026-03-09 18:20:12,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:20:13,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:20:13,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c8ce46d8ec186ac6e0/lava_callback.json.gz DEBUG: Setup failed for 69af01c8ce46d8ec186ac6e0. Transitting job node result: incomplete -> fail 2026-03-09 18:20:13,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:20:13,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c8ce46d8ec186ac6e0 HTTP/1.1" 200 None 2026-03-09 18:20:13,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:20:13,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:20:13,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:20:13,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c8ce46d8ec186ac6e0 HTTP/1.1" 200 None 2026-03-09 18:20:13,360 - lava_callback - INFO - Completed processing callback for node 69af01c8ce46d8ec186ac6e0 INFO: 172.21.0.1:51146 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:20:15,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:20:15,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59086 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:22:25,665 - lava_callback - INFO - Received callback for node 69af0de1ce46d8ec186aca6a INFO: 172.21.0.1:52566 - "POST /node/69af0de1ce46d8ec186aca6a HTTP/1.0" 202 Accepted 2026-03-09 18:22:25,671 - lava_callback - INFO - Starting async job submit for node 69af0de1ce46d8ec186aca6a 2026-03-09 18:22:25,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:22:25,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0de1ce46d8ec186aca6a HTTP/1.1" 200 None 2026-03-09 18:22:25,722 - lava_callback - DEBUG - Processing node 69af0de1ce46d8ec186aca6a: baseline-arm64-broonie 2026-03-09 18:22:26,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:22:26,210 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:22:26,212 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0de1ce46d8ec186aca6a/log.txt.gz 2026-03-09 18:22:26,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:52570 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:22:26,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:22:26,633 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0de1ce46d8ec186aca6a/lava-logs.yaml 2026-03-09 18:22:26,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:22:26,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:22:26,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0de1ce46d8ec186aca6a/lava_callback.json.gz DEBUG: Setup failed for 69af0de1ce46d8ec186aca6a. Transitting job node result: incomplete -> fail 2026-03-09 18:22:26,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:22:26,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0de1ce46d8ec186aca6a HTTP/1.1" 200 None 2026-03-09 18:22:26,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:22:26,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:22:26,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:22:26,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0de1ce46d8ec186aca6a HTTP/1.1" 200 None 2026-03-09 18:22:27,001 - lava_callback - INFO - Completed processing callback for node 69af0de1ce46d8ec186aca6a INFO: 172.21.0.1:42030 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:22:45,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:22:45,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38524 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:25:51,454 - lava_callback - INFO - Received callback for node 69af1080ce46d8ec186acadf INFO: 172.21.0.1:52902 - "POST /node/69af1080ce46d8ec186acadf HTTP/1.0" 202 Accepted 2026-03-09 18:25:51,459 - lava_callback - INFO - Starting async job submit for node 69af1080ce46d8ec186acadf 2026-03-09 18:25:51,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:25:51,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1080ce46d8ec186acadf HTTP/1.1" 200 None 2026-03-09 18:25:51,478 - lava_callback - DEBUG - Processing node 69af1080ce46d8ec186acadf: baseline-arm64-broonie 2026-03-09 18:25:51,483 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:25:51,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:25:51,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:25:51,630 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1080ce46d8ec186acadf/lava-logs.yaml 2026-03-09 18:25:51,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:25:51,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:25:51,772 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1080ce46d8ec186acadf/lava_callback.json.gz 2026-03-09 18:25:51,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:25:51,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1080ce46d8ec186acadf HTTP/1.1" 200 None 2026-03-09 18:25:51,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:25:51,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:25:51,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:25:51,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1080ce46d8ec186acadf HTTP/1.1" 200 None 2026-03-09 18:25:51,853 - lava_callback - INFO - Completed processing callback for node 69af1080ce46d8ec186acadf INFO: 172.21.0.1:52904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47640 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:26:15,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:26:15,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46352 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:27:33,669 - lava_callback - INFO - Received callback for node 69af0e62ce46d8ec186aca7b INFO: 172.21.0.1:36950 - "POST /node/69af0e62ce46d8ec186aca7b HTTP/1.0" 202 Accepted 2026-03-09 18:27:33,675 - lava_callback - INFO - Starting async job submit for node 69af0e62ce46d8ec186aca7b 2026-03-09 18:27:33,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:33,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0e62ce46d8ec186aca7b HTTP/1.1" 200 None 2026-03-09 18:27:33,723 - lava_callback - DEBUG - Processing node 69af0e62ce46d8ec186aca7b: baseline-arm64 2026-03-09 18:27:34,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:34,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:34,209 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0e62ce46d8ec186aca7b/log.txt.gz 2026-03-09 18:27:34,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:34,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:34,620 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0e62ce46d8ec186aca7b/lava-logs.yaml 2026-03-09 18:27:34,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:34,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:34,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0e62ce46d8ec186aca7b/lava_callback.json.gz 2026-03-09 18:27:34,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:34,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0e62ce46d8ec186aca7b HTTP/1.1" 200 None 2026-03-09 18:27:34,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:34,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:27:34,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:34,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0e62ce46d8ec186aca7b HTTP/1.1" 200 None 2026-03-09 18:27:34,972 - lava_callback - INFO - Completed processing callback for node 69af0e62ce46d8ec186aca7b INFO: 172.21.0.1:36966 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:27:40,905 - lava_callback - INFO - Received callback for node 69af0eafce46d8ec186aca83 INFO: 172.21.0.1:56684 - "POST /node/69af0eafce46d8ec186aca83 HTTP/1.0" 202 Accepted 2026-03-09 18:27:40,912 - lava_callback - INFO - Starting async job submit for node 69af0eafce46d8ec186aca83 2026-03-09 18:27:40,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:40,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0eafce46d8ec186aca83 HTTP/1.1" 200 None 2026-03-09 18:27:40,952 - lava_callback - DEBUG - Processing node 69af0eafce46d8ec186aca83: baseline-arm64 2026-03-09 18:27:41,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:41,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:41,778 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0eafce46d8ec186aca83/log.txt.gz 2026-03-09 18:27:42,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:42,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:42,244 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0eafce46d8ec186aca83/lava-logs.yaml 2026-03-09 18:27:42,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:42,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:42,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0eafce46d8ec186aca83/lava_callback.json.gz 2026-03-09 18:27:42,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:42,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0eafce46d8ec186aca83 HTTP/1.1" 200 None 2026-03-09 18:27:42,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:42,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:27:42,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:42,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0eafce46d8ec186aca83 HTTP/1.1" 200 None 2026-03-09 18:27:42,557 - lava_callback - INFO - Completed processing callback for node 69af0eafce46d8ec186aca83 2026-03-09 18:27:45,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:45,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59720 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:30:12,918 - lava_callback - INFO - Received callback for node 69af1198ce46d8ec186acb14 INFO: 172.21.0.1:36238 - "POST /node/69af1198ce46d8ec186acb14 HTTP/1.0" 202 Accepted 2026-03-09 18:30:12,924 - lava_callback - INFO - Starting async job submit for node 69af1198ce46d8ec186acb14 2026-03-09 18:30:12,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:12,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1198ce46d8ec186acb14 HTTP/1.1" 200 None 2026-03-09 18:30:12,954 - lava_callback - DEBUG - Processing node 69af1198ce46d8ec186acb14: baseline-arm64 2026-03-09 18:30:12,959 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:30:12,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:30:13,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:30:13,127 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1198ce46d8ec186acb14/lava-logs.yaml 2026-03-09 18:30:13,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:30:13,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:30:13,288 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1198ce46d8ec186acb14/lava_callback.json.gz 2026-03-09 18:30:13,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:13,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1198ce46d8ec186acb14 HTTP/1.1" 200 None 2026-03-09 18:30:13,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:13,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:30:13,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:13,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1198ce46d8ec186acb14 HTTP/1.1" 200 None 2026-03-09 18:30:13,352 - lava_callback - INFO - Completed processing callback for node 69af1198ce46d8ec186acb14 INFO: 172.21.0.1:36248 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:30:15,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:15,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42848 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:30:59,206 - lava_callback - INFO - Received callback for node 69af0f00ce46d8ec186aca9a INFO: 172.21.0.1:42854 - "POST /node/69af0f00ce46d8ec186aca9a HTTP/1.0" 202 Accepted 2026-03-09 18:30:59,213 - lava_callback - INFO - Starting async job submit for node 69af0f00ce46d8ec186aca9a 2026-03-09 18:30:59,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:59,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f00ce46d8ec186aca9a HTTP/1.1" 200 None 2026-03-09 18:30:59,253 - lava_callback - DEBUG - Processing node 69af0f00ce46d8ec186aca9a: baseline-nfs-arm64 2026-03-09 18:30:59,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:30:59,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:30:59,982 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f00ce46d8ec186aca9a/log.txt.gz 2026-03-09 18:31:00,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:31:00,494 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:31:00,496 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f00ce46d8ec186aca9a/lava-logs.yaml 2026-03-09 18:31:00,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:31:00,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:31:00,651 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f00ce46d8ec186aca9a/lava_callback.json.gz 2026-03-09 18:31:00,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:31:00,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f00ce46d8ec186aca9a HTTP/1.1" 200 None 2026-03-09 18:31:00,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:31:00,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:31:00,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:31:00,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0f00ce46d8ec186aca9a HTTP/1.1" 200 None 2026-03-09 18:31:00,800 - lava_callback - INFO - Completed processing callback for node 69af0f00ce46d8ec186aca9a INFO: 172.21.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33936 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:31:15,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:31:15,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39382 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:32:23,785 - lava_callback - INFO - Received callback for node 69af0f94ce46d8ec186acab3 INFO: 172.21.0.1:57442 - "POST /node/69af0f94ce46d8ec186acab3 HTTP/1.0" 202 Accepted 2026-03-09 18:32:23,792 - lava_callback - INFO - Starting async job submit for node 69af0f94ce46d8ec186acab3 2026-03-09 18:32:23,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:32:23,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f94ce46d8ec186acab3 HTTP/1.1" 200 None 2026-03-09 18:32:23,846 - lava_callback - DEBUG - Processing node 69af0f94ce46d8ec186acab3: baseline-nfs-arm64 2026-03-09 18:32:24,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:32:24,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:32:24,327 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f94ce46d8ec186acab3/log.txt.gz 2026-03-09 18:32:24,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:32:24,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:32:24,798 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f94ce46d8ec186acab3/lava-logs.yaml 2026-03-09 18:32:24,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:32:24,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:32:24,975 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f94ce46d8ec186acab3/lava_callback.json.gz 2026-03-09 18:32:25,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:32:25,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f94ce46d8ec186acab3 HTTP/1.1" 200 None 2026-03-09 18:32:25,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:32:25,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:32:25,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:32:25,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0f94ce46d8ec186acab3 HTTP/1.1" 200 None 2026-03-09 18:32:25,171 - lava_callback - INFO - Completed processing callback for node 69af0f94ce46d8ec186acab3 INFO: 172.21.0.1:57458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56440 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:32:45,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:32:45,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58854 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:33:28,540 - lava_callback - INFO - Received callback for node 69af1255ce46d8ec186acb43 INFO: 172.21.0.1:58870 - "POST /node/69af1255ce46d8ec186acb43 HTTP/1.0" 202 Accepted 2026-03-09 18:33:28,546 - lava_callback - INFO - Starting async job submit for node 69af1255ce46d8ec186acb43 2026-03-09 18:33:28,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:33:28,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1255ce46d8ec186acb43 HTTP/1.1" 200 None 2026-03-09 18:33:28,636 - lava_callback - DEBUG - Processing node 69af1255ce46d8ec186acb43: baseline-arm64 2026-03-09 18:33:28,686 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:33:28,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:33:28,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:33:28,926 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1255ce46d8ec186acb43/lava-logs.yaml 2026-03-09 18:33:28,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:33:29,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:33:29,076 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1255ce46d8ec186acb43/lava_callback.json.gz 2026-03-09 18:33:29,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:33:29,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1255ce46d8ec186acb43 HTTP/1.1" 200 None 2026-03-09 18:33:29,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:33:29,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:33:29,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:33:29,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1255ce46d8ec186acb43 HTTP/1.1" 200 None 2026-03-09 18:33:29,195 - lava_callback - INFO - Completed processing callback for node 69af1255ce46d8ec186acb43 INFO: 172.21.0.1:58896 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:33:45,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:33:45,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39622 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:34:12,702 - lava_callback - INFO - Received callback for node 69af115bce46d8ec186acb0f INFO: 172.21.0.1:39970 - "POST /node/69af115bce46d8ec186acb0f HTTP/1.0" 202 Accepted 2026-03-09 18:34:12,708 - lava_callback - INFO - Starting async job submit for node 69af115bce46d8ec186acb0f 2026-03-09 18:34:12,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:12,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af115bce46d8ec186acb0f HTTP/1.1" 200 None 2026-03-09 18:34:12,768 - lava_callback - DEBUG - Processing node 69af115bce46d8ec186acb0f: baseline-arm64-broonie 2026-03-09 18:34:13,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:34:13,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:34:13,350 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af115bce46d8ec186acb0f/log.txt.gz 2026-03-09 18:34:13,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:34:13,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:34:13,850 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af115bce46d8ec186acb0f/lava-logs.yaml 2026-03-09 18:34:13,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:34:14,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:34:14,050 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af115bce46d8ec186acb0f/lava_callback.json.gz DEBUG: Setup failed for 69af115bce46d8ec186acb0f. Transitting job node result: incomplete -> fail 2026-03-09 18:34:14,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:14,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af115bce46d8ec186acb0f HTTP/1.1" 200 None 2026-03-09 18:34:14,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:14,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:34:14,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:14,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af115bce46d8ec186acb0f HTTP/1.1" 200 None 2026-03-09 18:34:14,285 - lava_callback - INFO - Completed processing callback for node 69af115bce46d8ec186acb0f INFO: 172.21.0.1:39972 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:34:15,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:15,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37622 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:34:37,379 - lava_callback - INFO - Received callback for node 69af12a7ce46d8ec186acb6a INFO: 172.21.0.1:37630 - "POST /node/69af12a7ce46d8ec186acb6a HTTP/1.0" 202 Accepted 2026-03-09 18:34:37,385 - lava_callback - INFO - Starting async job submit for node 69af12a7ce46d8ec186acb6a 2026-03-09 18:34:37,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:37,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af12a7ce46d8ec186acb6a HTTP/1.1" 200 None 2026-03-09 18:34:37,402 - lava_callback - DEBUG - Processing node 69af12a7ce46d8ec186acb6a: baseline-nfs-arm64 2026-03-09 18:34:37,404 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:34:37,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:34:37,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:34:37,569 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af12a7ce46d8ec186acb6a/lava-logs.yaml 2026-03-09 18:34:37,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:34:37,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:34:37,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af12a7ce46d8ec186acb6a/lava_callback.json.gz 2026-03-09 18:34:37,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:37,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af12a7ce46d8ec186acb6a HTTP/1.1" 200 None 2026-03-09 18:34:37,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:37,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:34:37,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:37,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af12a7ce46d8ec186acb6a HTTP/1.1" 200 None 2026-03-09 18:34:37,782 - lava_callback - INFO - Completed processing callback for node 69af12a7ce46d8ec186acb6a 2026-03-09 18:34:45,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:45,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45868 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:36:11,081 - lava_callback - INFO - Received callback for node 69af12eece46d8ec186acb75 INFO: 172.21.0.1:40284 - "POST /node/69af12eece46d8ec186acb75 HTTP/1.0" 202 Accepted 2026-03-09 18:36:11,087 - lava_callback - INFO - Starting async job submit for node 69af12eece46d8ec186acb75 2026-03-09 18:36:11,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:11,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af12eece46d8ec186acb75 HTTP/1.1" 200 None 2026-03-09 18:36:11,119 - lava_callback - DEBUG - Processing node 69af12eece46d8ec186acb75: baseline-arm64 2026-03-09 18:36:11,185 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:36:11,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:36:11,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:36:11,360 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af12eece46d8ec186acb75/lava-logs.yaml 2026-03-09 18:36:11,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:36:11,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:36:11,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af12eece46d8ec186acb75/lava_callback.json.gz 2026-03-09 18:36:11,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:11,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af12eece46d8ec186acb75 HTTP/1.1" 200 None 2026-03-09 18:36:11,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:11,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:36:11,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:11,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af12eece46d8ec186acb75 HTTP/1.1" 200 None 2026-03-09 18:36:11,614 - lava_callback - INFO - Completed processing callback for node 69af12eece46d8ec186acb75 INFO: 172.21.0.1:40294 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:36:15,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:15,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40304 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:36:25,603 - lava_callback - INFO - Received callback for node 69af0bd2ce46d8ec186aca10 INFO: 172.21.0.1:35208 - "POST /node/69af0bd2ce46d8ec186aca10 HTTP/1.0" 202 Accepted 2026-03-09 18:36:25,608 - lava_callback - INFO - Starting async job submit for node 69af0bd2ce46d8ec186aca10 2026-03-09 18:36:25,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:25,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0bd2ce46d8ec186aca10 HTTP/1.1" 200 None 2026-03-09 18:36:25,642 - lava_callback - DEBUG - Processing node 69af0bd2ce46d8ec186aca10: baseline-arm64-broonie 2026-03-09 18:36:25,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:36:25,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:36:25,835 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0bd2ce46d8ec186aca10/log.txt.gz 2026-03-09 18:36:25,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:36:26,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:36:26,044 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0bd2ce46d8ec186aca10/lava-logs.yaml 2026-03-09 18:36:26,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:36:26,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:36:26,193 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0bd2ce46d8ec186aca10/lava_callback.json.gz 2026-03-09 18:36:26,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:26,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0bd2ce46d8ec186aca10 HTTP/1.1" 200 None 2026-03-09 18:36:26,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:26,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:36:26,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:26,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0bd2ce46d8ec186aca10 HTTP/1.1" 200 None 2026-03-09 18:36:26,310 - lava_callback - INFO - Completed processing callback for node 69af0bd2ce46d8ec186aca10 INFO: 172.21.0.1:35214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35316 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:36:45,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:45,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39278 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:37:01,417 - lava_callback - INFO - Received callback for node 69af1334ce46d8ec186acb82 INFO: 172.21.0.1:42470 - "POST /node/69af1334ce46d8ec186acb82 HTTP/1.0" 202 Accepted 2026-03-09 18:37:01,423 - lava_callback - INFO - Starting async job submit for node 69af1334ce46d8ec186acb82 2026-03-09 18:37:01,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:37:01,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1334ce46d8ec186acb82 HTTP/1.1" 200 None 2026-03-09 18:37:01,465 - lava_callback - DEBUG - Processing node 69af1334ce46d8ec186acb82: baseline-arm64 2026-03-09 18:37:01,513 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:37:01,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:37:01,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:37:01,710 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1334ce46d8ec186acb82/lava-logs.yaml 2026-03-09 18:37:01,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:37:01,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:37:01,834 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1334ce46d8ec186acb82/lava_callback.json.gz 2026-03-09 18:37:01,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:37:01,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1334ce46d8ec186acb82 HTTP/1.1" 200 None 2026-03-09 18:37:01,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:37:01,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:37:01,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:37:01,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1334ce46d8ec186acb82 HTTP/1.1" 200 None 2026-03-09 18:37:01,949 - lava_callback - INFO - Completed processing callback for node 69af1334ce46d8ec186acb82 INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35720 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:37:15,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:37:15,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59694 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:39:26,955 - lava_callback - INFO - Received callback for node 69af0fd6ce46d8ec186acab9 INFO: 172.21.0.1:59710 - "POST /node/69af0fd6ce46d8ec186acab9 HTTP/1.0" 202 Accepted 2026-03-09 18:39:26,961 - lava_callback - INFO - Starting async job submit for node 69af0fd6ce46d8ec186acab9 2026-03-09 18:39:27,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:27,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0fd6ce46d8ec186acab9 HTTP/1.1" 200 None 2026-03-09 18:39:27,049 - lava_callback - DEBUG - Processing node 69af0fd6ce46d8ec186acab9: ltp-smoke 2026-03-09 18:39:27,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:39:27,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:39:27,732 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af0fd6ce46d8ec186acab9/log.txt.gz 2026-03-09 18:39:28,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:39:28,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:39:28,849 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af0fd6ce46d8ec186acab9/lava-logs.yaml 2026-03-09 18:39:28,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:39:29,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:39:29,055 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af0fd6ce46d8ec186acab9/lava_callback.json.gz 2026-03-09 18:39:29,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:29,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0fd6ce46d8ec186acab9 HTTP/1.1" 200 None 2026-03-09 18:39:29,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:29,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef42ace46d8ec186ab814 HTTP/1.1" 200 None 2026-03-09 18:39:29,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:29,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0fd6ce46d8ec186acab9 HTTP/1.1" 200 None 2026-03-09 18:39:29,352 - lava_callback - INFO - Completed processing callback for node 69af0fd6ce46d8ec186acab9 INFO: 172.21.0.1:40544 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:39:45,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:45,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36990 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:39:56,905 - lava_callback - INFO - Received callback for node 69af13dace46d8ec186acbaa INFO: 172.21.0.1:36994 - "POST /node/69af13dace46d8ec186acbaa HTTP/1.0" 202 Accepted 2026-03-09 18:39:56,911 - lava_callback - INFO - Starting async job submit for node 69af13dace46d8ec186acbaa 2026-03-09 18:39:56,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:56,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af13dace46d8ec186acbaa HTTP/1.1" 200 None 2026-03-09 18:39:56,929 - lava_callback - DEBUG - Processing node 69af13dace46d8ec186acbaa: baseline-nfs-arm64 2026-03-09 18:39:56,934 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:39:56,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:39:57,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:39:57,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af13dace46d8ec186acbaa/lava-logs.yaml 2026-03-09 18:39:57,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:39:57,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:39:57,211 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af13dace46d8ec186acbaa/lava_callback.json.gz 2026-03-09 18:39:57,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:57,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af13dace46d8ec186acbaa HTTP/1.1" 200 None 2026-03-09 18:39:57,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:57,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:39:57,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:57,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af13dace46d8ec186acbaa HTTP/1.1" 200 None 2026-03-09 18:39:57,277 - lava_callback - INFO - Completed processing callback for node 69af13dace46d8ec186acbaa INFO: 172.21.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40820 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:40:15,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:40:16,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47432 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:41:50,351 - lava_callback - INFO - Received callback for node 69ac716946f7ae8ff3366031 INFO: 172.21.0.1:38506 - "POST /node/69ac716946f7ae8ff3366031 HTTP/1.0" 202 Accepted 2026-03-09 18:41:50,356 - lava_callback - INFO - Starting async job submit for node 69ac716946f7ae8ff3366031 2026-03-09 18:41:50,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:41:50,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac716946f7ae8ff3366031 HTTP/1.1" 200 None 2026-03-09 18:41:50,372 - lava_callback - DEBUG - Processing node 69ac716946f7ae8ff3366031: baseline-arm64-baylibre 2026-03-09 18:41:50,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac716946f7ae8ff3366031 2026-03-09 18:41:50,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:41:50,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac716946f7ae8ff3366031 HTTP/1.1" 200 None 2026-03-09 18:41:50,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:41:50,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:41:50,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:41:50,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac716946f7ae8ff3366031 HTTP/1.1" 200 None 2026-03-09 18:41:50,427 - lava_callback - ERROR - Error processing callback for node 69ac716946f7ae8ff3366031: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47222 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:42:24,976 - lava_callback - INFO - Received callback for node 69af1485ce46d8ec186acbd4 INFO: 172.21.0.1:40040 - "POST /node/69af1485ce46d8ec186acbd4 HTTP/1.0" 202 Accepted 2026-03-09 18:42:24,982 - lava_callback - INFO - Starting async job submit for node 69af1485ce46d8ec186acbd4 2026-03-09 18:42:24,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:42:25,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1485ce46d8ec186acbd4 HTTP/1.1" 200 None 2026-03-09 18:42:25,014 - lava_callback - DEBUG - Processing node 69af1485ce46d8ec186acbd4: baseline-arm64-foundriesio 2026-03-09 18:42:25,064 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:42:25,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:42:25,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:42:25,291 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af1485ce46d8ec186acbd4/lava-logs.yaml 2026-03-09 18:42:25,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:42:25,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:42:25,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af1485ce46d8ec186acbd4/lava_callback.json.gz 2026-03-09 18:42:25,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:42:25,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1485ce46d8ec186acbd4 HTTP/1.1" 200 None 2026-03-09 18:42:25,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:42:25,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:42:25,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:42:25,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1485ce46d8ec186acbd4 HTTP/1.1" 200 None 2026-03-09 18:42:25,560 - lava_callback - INFO - Completed processing callback for node 69af1485ce46d8ec186acbd4 INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35032 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:42:46,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:42:46,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59382 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:43:10,376 - lava_callback - INFO - Received callback for node 69ac71b946f7ae8ff3366057 INFO: 172.21.0.1:35034 - "POST /node/69ac71b946f7ae8ff3366057 HTTP/1.0" 202 Accepted 2026-03-09 18:43:10,382 - lava_callback - INFO - Starting async job submit for node 69ac71b946f7ae8ff3366057 2026-03-09 18:43:10,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:43:10,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac71b946f7ae8ff3366057 HTTP/1.1" 200 None 2026-03-09 18:43:10,398 - lava_callback - DEBUG - Processing node 69ac71b946f7ae8ff3366057: baseline-arm64-baylibre 2026-03-09 18:43:10,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac71b946f7ae8ff3366057 2026-03-09 18:43:10,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:43:10,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac71b946f7ae8ff3366057 HTTP/1.1" 200 None 2026-03-09 18:43:10,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:43:10,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:43:10,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:43:10,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac71b946f7ae8ff3366057 HTTP/1.1" 200 None 2026-03-09 18:43:10,480 - lava_callback - ERROR - Error processing callback for node 69ac71b946f7ae8ff3366057: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:44:30,446 - lava_callback - INFO - Received callback for node 69ac720a46f7ae8ff3366074 INFO: 172.21.0.1:38584 - "POST /node/69ac720a46f7ae8ff3366074 HTTP/1.0" 202 Accepted 2026-03-09 18:44:30,452 - lava_callback - INFO - Starting async job submit for node 69ac720a46f7ae8ff3366074 2026-03-09 18:44:30,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:44:30,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac720a46f7ae8ff3366074 HTTP/1.1" 200 None 2026-03-09 18:44:30,468 - lava_callback - DEBUG - Processing node 69ac720a46f7ae8ff3366074: baseline-arm64-baylibre 2026-03-09 18:44:30,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac720a46f7ae8ff3366074 2026-03-09 18:44:30,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:44:30,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac720a46f7ae8ff3366074 HTTP/1.1" 200 None 2026-03-09 18:44:30,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:44:30,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:44:30,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:44:30,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac720a46f7ae8ff3366074 HTTP/1.1" 200 None 2026-03-09 18:44:30,520 - lava_callback - ERROR - Error processing callback for node 69ac720a46f7ae8ff3366074: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58284 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:46:07,312 - lava_callback - INFO - Received callback for node 69af14cace46d8ec186acbe3 INFO: 172.21.0.1:58300 - "POST /node/69af14cace46d8ec186acbe3 HTTP/1.0" 202 Accepted 2026-03-09 18:46:07,318 - lava_callback - INFO - Starting async job submit for node 69af14cace46d8ec186acbe3 2026-03-09 18:46:07,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:07,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af14cace46d8ec186acbe3 HTTP/1.1" 200 None 2026-03-09 18:46:07,374 - lava_callback - DEBUG - Processing node 69af14cace46d8ec186acbe3: baseline-arm64-broonie 2026-03-09 18:46:07,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:46:07,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:46:07,785 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af14cace46d8ec186acbe3/log.txt.gz 2026-03-09 18:46:07,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:46:08,162 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:46:08,164 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af14cace46d8ec186acbe3/lava-logs.yaml 2026-03-09 18:46:08,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:46:08,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:46:08,332 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af14cace46d8ec186acbe3/lava_callback.json.gz DEBUG: Setup failed for 69af14cace46d8ec186acbe3. Transitting job node result: incomplete -> fail 2026-03-09 18:46:08,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:08,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af14cace46d8ec186acbe3 HTTP/1.1" 200 None 2026-03-09 18:46:08,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:08,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:46:08,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:08,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af14cace46d8ec186acbe3 HTTP/1.1" 200 None 2026-03-09 18:46:08,579 - lava_callback - INFO - Completed processing callback for node 69af14cace46d8ec186acbe3 2026-03-09 18:46:10,562 - lava_callback - INFO - Received callback for node 69ac726f46f7ae8ff336608b INFO: 172.21.0.1:58400 - "POST /node/69ac726f46f7ae8ff336608b HTTP/1.0" 202 Accepted 2026-03-09 18:46:10,567 - lava_callback - INFO - Starting async job submit for node 69ac726f46f7ae8ff336608b 2026-03-09 18:46:10,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:10,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac726f46f7ae8ff336608b HTTP/1.1" 200 None 2026-03-09 18:46:10,584 - lava_callback - DEBUG - Processing node 69ac726f46f7ae8ff336608b: baseline-arm64-baylibre 2026-03-09 18:46:10,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac726f46f7ae8ff336608b 2026-03-09 18:46:10,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:10,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac726f46f7ae8ff336608b HTTP/1.1" 200 None 2026-03-09 18:46:10,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:10,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:46:10,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:10,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac726f46f7ae8ff336608b HTTP/1.1" 200 None 2026-03-09 18:46:10,641 - lava_callback - ERROR - Error processing callback for node 69ac726f46f7ae8ff336608b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58416 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:46:16,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:16,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33414 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:47:30,625 - lava_callback - INFO - Received callback for node 69ac72c046f7ae8ff33660ac INFO: 172.21.0.1:50538 - "POST /node/69ac72c046f7ae8ff33660ac HTTP/1.0" 202 Accepted 2026-03-09 18:47:30,630 - lava_callback - INFO - Starting async job submit for node 69ac72c046f7ae8ff33660ac 2026-03-09 18:47:30,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:30,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac72c046f7ae8ff33660ac HTTP/1.1" 200 None 2026-03-09 18:47:30,647 - lava_callback - DEBUG - Processing node 69ac72c046f7ae8ff33660ac: baseline-arm64-baylibre 2026-03-09 18:47:30,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac72c046f7ae8ff33660ac 2026-03-09 18:47:30,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:30,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac72c046f7ae8ff33660ac HTTP/1.1" 200 None 2026-03-09 18:47:30,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:30,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:47:30,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:30,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac72c046f7ae8ff33660ac HTTP/1.1" 200 None 2026-03-09 18:47:30,731 - lava_callback - ERROR - Error processing callback for node 69ac72c046f7ae8ff33660ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50546 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:47:39,482 - lava_callback - INFO - Received callback for node 69af15b2ce46d8ec186acc1d INFO: 172.21.0.1:50562 - "POST /node/69af15b2ce46d8ec186acc1d HTTP/1.0" 202 Accepted 2026-03-09 18:47:39,487 - lava_callback - INFO - Starting async job submit for node 69af15b2ce46d8ec186acc1d 2026-03-09 18:47:39,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:39,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af15b2ce46d8ec186acc1d HTTP/1.1" 200 None 2026-03-09 18:47:39,506 - lava_callback - DEBUG - Processing node 69af15b2ce46d8ec186acc1d: baseline-arm64-clabbe 2026-03-09 18:47:39,510 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:47:39,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:47:39,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:47:39,685 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af15b2ce46d8ec186acc1d/lava-logs.yaml 2026-03-09 18:47:39,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:47:39,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:47:39,821 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af15b2ce46d8ec186acc1d/lava_callback.json.gz 2026-03-09 18:47:39,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:39,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af15b2ce46d8ec186acc1d HTTP/1.1" 200 None 2026-03-09 18:47:39,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:39,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:47:39,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:39,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af15b2ce46d8ec186acc1d HTTP/1.1" 200 None 2026-03-09 18:47:39,887 - lava_callback - INFO - Completed processing callback for node 69af15b2ce46d8ec186acc1d 2026-03-09 18:47:46,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:46,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 18:47:46,402 - lava_callback - INFO - Received callback for node 69af1562ce46d8ec186acc06 INFO: 172.21.0.1:35560 - "POST /node/69af1562ce46d8ec186acc06 HTTP/1.0" 202 Accepted 2026-03-09 18:47:46,409 - lava_callback - INFO - Starting async job submit for node 69af1562ce46d8ec186acc06 2026-03-09 18:47:46,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:46,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1562ce46d8ec186acc06 HTTP/1.1" 200 None 2026-03-09 18:47:46,469 - lava_callback - DEBUG - Processing node 69af1562ce46d8ec186acc06: baseline-arm64-broonie INFO: 172.21.0.1:35564 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:47:46,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:47:46,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:47:46,815 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1562ce46d8ec186acc06/log.txt.gz 2026-03-09 18:47:46,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:47:47,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:47:47,158 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1562ce46d8ec186acc06/lava-logs.yaml 2026-03-09 18:47:47,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:47:47,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:47:47,313 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1562ce46d8ec186acc06/lava_callback.json.gz DEBUG: Setup failed for 69af1562ce46d8ec186acc06. Transitting job node result: incomplete -> fail 2026-03-09 18:47:47,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:47,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1562ce46d8ec186acc06 HTTP/1.1" 200 None 2026-03-09 18:47:47,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:47,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:47:47,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:47,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1562ce46d8ec186acc06 HTTP/1.1" 200 None 2026-03-09 18:47:47,557 - lava_callback - INFO - Completed processing callback for node 69af1562ce46d8ec186acc06 INFO: 172.21.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54156 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:48:15,409 - lava_callback - INFO - Received callback for node 69af122bce46d8ec186acb2b INFO: 172.21.0.1:54166 - "POST /node/69af122bce46d8ec186acb2b HTTP/1.0" 202 Accepted 2026-03-09 18:48:15,416 - lava_callback - INFO - Starting async job submit for node 69af122bce46d8ec186acb2b 2026-03-09 18:48:15,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:15,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af122bce46d8ec186acb2b HTTP/1.1" 200 None 2026-03-09 18:48:15,490 - lava_callback - DEBUG - Processing node 69af122bce46d8ec186acb2b: baseline-arm 2026-03-09 18:48:16,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:16,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:48:16,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 18:48:16,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:48:16,217 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af122bce46d8ec186acb2b/log.txt.gz INFO: 172.21.0.1:54176 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:48:16,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:48:16,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:48:16,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af122bce46d8ec186acb2b/lava-logs.yaml 2026-03-09 18:48:16,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:48:17,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:48:17,007 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af122bce46d8ec186acb2b/lava_callback.json.gz DEBUG: Setup failed for 69af122bce46d8ec186acb2b. Transitting job node result: incomplete -> fail 2026-03-09 18:48:17,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:17,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af122bce46d8ec186acb2b HTTP/1.1" 200 None 2026-03-09 18:48:17,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:17,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 18:48:17,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:17,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af122bce46d8ec186acb2b HTTP/1.1" 200 None 2026-03-09 18:48:17,293 - lava_callback - INFO - Completed processing callback for node 69af122bce46d8ec186acb2b INFO: 172.21.0.1:39672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60186 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:48:46,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:46,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:49:10,766 - lava_callback - INFO - Received callback for node 69ac732446f7ae8ff33660bf INFO: 172.21.0.1:48448 - "POST /node/69ac732446f7ae8ff33660bf HTTP/1.0" 202 Accepted 2026-03-09 18:49:10,772 - lava_callback - INFO - Starting async job submit for node 69ac732446f7ae8ff33660bf 2026-03-09 18:49:10,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:10,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac732446f7ae8ff33660bf HTTP/1.1" 200 None 2026-03-09 18:49:10,790 - lava_callback - DEBUG - Processing node 69ac732446f7ae8ff33660bf: baseline-arm64-baylibre 2026-03-09 18:49:10,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac732446f7ae8ff33660bf 2026-03-09 18:49:10,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:10,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac732446f7ae8ff33660bf HTTP/1.1" 200 None 2026-03-09 18:49:10,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:10,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:49:10,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:10,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac732446f7ae8ff33660bf HTTP/1.1" 200 None 2026-03-09 18:49:10,851 - lava_callback - ERROR - Error processing callback for node 69ac732446f7ae8ff33660bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 18:49:12,364 - lava_callback - INFO - Received callback for node 69af15ffce46d8ec186acc48 INFO: 172.21.0.1:48452 - "POST /node/69af15ffce46d8ec186acc48 HTTP/1.0" 202 Accepted 2026-03-09 18:49:12,370 - lava_callback - INFO - Starting async job submit for node 69af15ffce46d8ec186acc48 2026-03-09 18:49:12,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:12,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af15ffce46d8ec186acc48 HTTP/1.1" 200 None 2026-03-09 18:49:12,392 - lava_callback - DEBUG - Processing node 69af15ffce46d8ec186acc48: baseline-arm64-broonie 2026-03-09 18:49:12,397 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:49:12,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:49:12,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:49:12,534 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af15ffce46d8ec186acc48/lava-logs.yaml 2026-03-09 18:49:12,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:49:12,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:49:12,677 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af15ffce46d8ec186acc48/lava_callback.json.gz 2026-03-09 18:49:12,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:12,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af15ffce46d8ec186acc48 HTTP/1.1" 200 None 2026-03-09 18:49:12,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:12,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:49:12,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:12,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af15ffce46d8ec186acc48 HTTP/1.1" 200 None 2026-03-09 18:49:12,739 - lava_callback - INFO - Completed processing callback for node 69af15ffce46d8ec186acc48 INFO: 172.21.0.1:48454 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:49:16,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:16,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60130 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:51:47,671 - lava_callback - INFO - Received callback for node 69af165fce46d8ec186acc7d INFO: 172.21.0.1:60146 - "POST /node/69af165fce46d8ec186acc7d HTTP/1.0" 202 Accepted 2026-03-09 18:51:47,677 - lava_callback - INFO - Starting async job submit for node 69af165fce46d8ec186acc7d 2026-03-09 18:51:47,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:51:47,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af165fce46d8ec186acc7d HTTP/1.1" 200 None 2026-03-09 18:51:47,727 - lava_callback - DEBUG - Processing node 69af165fce46d8ec186acc7d: baseline-arm64 2026-03-09 18:51:48,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:51:48,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:51:48,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af165fce46d8ec186acc7d/log.txt.gz 2026-03-09 18:51:48,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:51:48,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:51:48,805 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af165fce46d8ec186acc7d/lava-logs.yaml 2026-03-09 18:51:48,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:51:48,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:51:48,978 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af165fce46d8ec186acc7d/lava_callback.json.gz DEBUG: Setup failed for 69af165fce46d8ec186acc7d. Transitting job node result: incomplete -> fail 2026-03-09 18:51:49,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:51:49,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af165fce46d8ec186acc7d HTTP/1.1" 200 None 2026-03-09 18:51:49,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:51:49,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:51:49,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:51:49,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af165fce46d8ec186acc7d HTTP/1.1" 200 None 2026-03-09 18:51:49,195 - lava_callback - INFO - Completed processing callback for node 69af165fce46d8ec186acc7d INFO: 172.21.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:52:16,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:16,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:52:30,980 - lava_callback - INFO - Received callback for node 69ac73dc46f7ae8ff33660fd INFO: 172.21.0.1:44132 - "POST /node/69ac73dc46f7ae8ff33660fd HTTP/1.0" 202 Accepted 2026-03-09 18:52:30,986 - lava_callback - INFO - Starting async job submit for node 69ac73dc46f7ae8ff33660fd 2026-03-09 18:52:30,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:31,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac73dc46f7ae8ff33660fd HTTP/1.1" 200 None 2026-03-09 18:52:31,025 - lava_callback - DEBUG - Processing node 69ac73dc46f7ae8ff33660fd: baseline-arm-baylibre 2026-03-09 18:52:31,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac73dc46f7ae8ff33660fd 2026-03-09 18:52:31,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:31,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac73dc46f7ae8ff33660fd HTTP/1.1" 200 None 2026-03-09 18:52:31,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:31,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:52:31,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:31,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac73dc46f7ae8ff33660fd HTTP/1.1" 200 None 2026-03-09 18:52:31,080 - lava_callback - ERROR - Error processing callback for node 69ac73dc46f7ae8ff33660fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44134 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:52:45,012 - lava_callback - INFO - Received callback for node 69af16abce46d8ec186acc86 INFO: 172.21.0.1:48728 - "POST /node/69af16abce46d8ec186acc86 HTTP/1.0" 202 Accepted 2026-03-09 18:52:45,019 - lava_callback - INFO - Starting async job submit for node 69af16abce46d8ec186acc86 2026-03-09 18:52:45,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:45,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af16abce46d8ec186acc86 HTTP/1.1" 200 None 2026-03-09 18:52:45,083 - lava_callback - DEBUG - Processing node 69af16abce46d8ec186acc86: baseline-arm64 2026-03-09 18:52:45,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:52:46,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:52:46,085 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16abce46d8ec186acc86/log.txt.gz INFO: 172.21.0.1:48744 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:52:46,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:52:46,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:52:46,795 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16abce46d8ec186acc86/lava-logs.yaml 2026-03-09 18:52:46,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:52:46,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:52:46,957 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16abce46d8ec186acc86/lava_callback.json.gz DEBUG: Setup failed for 69af16abce46d8ec186acc86. Transitting job node result: incomplete -> fail 2026-03-09 18:52:47,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:47,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af16abce46d8ec186acc86 HTTP/1.1" 200 None 2026-03-09 18:52:47,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:47,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:52:47,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:47,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af16abce46d8ec186acc86 HTTP/1.1" 200 None 2026-03-09 18:52:47,178 - lava_callback - INFO - Completed processing callback for node 69af16abce46d8ec186acc86 INFO: 172.21.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33206 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:53:11,004 - lava_callback - INFO - Received callback for node 69ac741046f7ae8ff336610f INFO: 172.21.0.1:49108 - "POST /node/69ac741046f7ae8ff336610f HTTP/1.0" 202 Accepted 2026-03-09 18:53:11,009 - lava_callback - INFO - Starting async job submit for node 69ac741046f7ae8ff336610f 2026-03-09 18:53:11,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:53:11,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac741046f7ae8ff336610f HTTP/1.1" 200 None 2026-03-09 18:53:11,026 - lava_callback - DEBUG - Processing node 69ac741046f7ae8ff336610f: baseline-arm-baylibre 2026-03-09 18:53:11,026 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac741046f7ae8ff336610f 2026-03-09 18:53:11,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:53:11,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac741046f7ae8ff336610f HTTP/1.1" 200 None 2026-03-09 18:53:11,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:53:11,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:53:11,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:53:11,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac741046f7ae8ff336610f HTTP/1.1" 200 None 2026-03-09 18:53:11,078 - lava_callback - ERROR - Error processing callback for node 69ac741046f7ae8ff336610f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49112 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:53:16,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:53:16,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45074 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:54:11,037 - lava_callback - INFO - Received callback for node 69ac744d46f7ae8ff3366131 INFO: 172.21.0.1:44738 - "POST /node/69ac744d46f7ae8ff3366131 HTTP/1.0" 202 Accepted 2026-03-09 18:54:11,042 - lava_callback - INFO - Starting async job submit for node 69ac744d46f7ae8ff3366131 2026-03-09 18:54:11,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:11,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac744d46f7ae8ff3366131 HTTP/1.1" 200 None 2026-03-09 18:54:11,061 - lava_callback - DEBUG - Processing node 69ac744d46f7ae8ff3366131: baseline-arm-baylibre 2026-03-09 18:54:11,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac744d46f7ae8ff3366131 2026-03-09 18:54:11,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:11,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac744d46f7ae8ff3366131 HTTP/1.1" 200 None 2026-03-09 18:54:11,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:11,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:54:11,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:11,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac744d46f7ae8ff3366131 HTTP/1.1" 200 None 2026-03-09 18:54:11,164 - lava_callback - ERROR - Error processing callback for node 69ac744d46f7ae8ff3366131: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 18:54:13,495 - lava_callback - INFO - Received callback for node 69af16f9ce46d8ec186acc97 INFO: 172.21.0.1:44742 - "POST /node/69af16f9ce46d8ec186acc97 HTTP/1.0" 202 Accepted 2026-03-09 18:54:13,501 - lava_callback - INFO - Starting async job submit for node 69af16f9ce46d8ec186acc97 2026-03-09 18:54:13,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:13,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af16f9ce46d8ec186acc97 HTTP/1.1" 200 None 2026-03-09 18:54:13,560 - lava_callback - DEBUG - Processing node 69af16f9ce46d8ec186acc97: baseline-arm64 2026-03-09 18:54:13,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:54:14,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:54:14,047 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16f9ce46d8ec186acc97/log.txt.gz 2026-03-09 18:54:14,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:54:14,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:54:14,479 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16f9ce46d8ec186acc97/lava-logs.yaml 2026-03-09 18:54:14,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:54:14,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:54:14,618 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16f9ce46d8ec186acc97/lava_callback.json.gz DEBUG: Setup failed for 69af16f9ce46d8ec186acc97. Transitting job node result: incomplete -> fail 2026-03-09 18:54:14,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:14,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af16f9ce46d8ec186acc97 HTTP/1.1" 200 None 2026-03-09 18:54:14,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:14,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:54:14,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:14,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af16f9ce46d8ec186acc97 HTTP/1.1" 200 None 2026-03-09 18:54:14,809 - lava_callback - INFO - Completed processing callback for node 69af16f9ce46d8ec186acc97 INFO: 172.21.0.1:44744 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:54:16,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:16,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52874 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:54:51,030 - lava_callback - INFO - Received callback for node 69ac747146f7ae8ff3366145 INFO: 172.21.0.1:43118 - "POST /node/69ac747146f7ae8ff3366145 HTTP/1.0" 202 Accepted 2026-03-09 18:54:51,035 - lava_callback - INFO - Starting async job submit for node 69ac747146f7ae8ff3366145 2026-03-09 18:54:51,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:51,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac747146f7ae8ff3366145 HTTP/1.1" 200 None 2026-03-09 18:54:51,051 - lava_callback - DEBUG - Processing node 69ac747146f7ae8ff3366145: baseline-arm-baylibre 2026-03-09 18:54:51,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac747146f7ae8ff3366145 2026-03-09 18:54:51,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:51,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac747146f7ae8ff3366145 HTTP/1.1" 200 None 2026-03-09 18:54:51,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:51,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:54:51,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:51,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac747146f7ae8ff3366145 HTTP/1.1" 200 None 2026-03-09 18:54:51,159 - lava_callback - ERROR - Error processing callback for node 69ac747146f7ae8ff3366145: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38390 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:55:31,079 - lava_callback - INFO - Received callback for node 69ac749546f7ae8ff3366159 INFO: 172.21.0.1:36890 - "POST /node/69ac749546f7ae8ff3366159 HTTP/1.0" 202 Accepted 2026-03-09 18:55:31,084 - lava_callback - INFO - Starting async job submit for node 69ac749546f7ae8ff3366159 2026-03-09 18:55:31,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:31,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac749546f7ae8ff3366159 HTTP/1.1" 200 None 2026-03-09 18:55:31,101 - lava_callback - DEBUG - Processing node 69ac749546f7ae8ff3366159: baseline-arm-baylibre 2026-03-09 18:55:31,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac749546f7ae8ff3366159 2026-03-09 18:55:31,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:31,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac749546f7ae8ff3366159 HTTP/1.1" 200 None 2026-03-09 18:55:31,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:31,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:55:31,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:31,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac749546f7ae8ff3366159 HTTP/1.1" 200 None 2026-03-09 18:55:31,170 - lava_callback - ERROR - Error processing callback for node 69ac749546f7ae8ff3366159: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 18:55:36,368 - lava_callback - INFO - Received callback for node 69af1742ce46d8ec186acca4 INFO: 172.21.0.1:36902 - "POST /node/69af1742ce46d8ec186acca4 HTTP/1.0" 202 Accepted 2026-03-09 18:55:36,374 - lava_callback - INFO - Starting async job submit for node 69af1742ce46d8ec186acca4 2026-03-09 18:55:36,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:36,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1742ce46d8ec186acca4 HTTP/1.1" 200 None 2026-03-09 18:55:36,420 - lava_callback - DEBUG - Processing node 69af1742ce46d8ec186acca4: baseline-arm64 INFO: 172.21.0.1:36908 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:55:36,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:55:36,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:55:36,911 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1742ce46d8ec186acca4/log.txt.gz 2026-03-09 18:55:37,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:55:37,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:55:37,357 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1742ce46d8ec186acca4/lava-logs.yaml 2026-03-09 18:55:37,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:55:37,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:55:37,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1742ce46d8ec186acca4/lava_callback.json.gz DEBUG: Setup failed for 69af1742ce46d8ec186acca4. Transitting job node result: incomplete -> fail 2026-03-09 18:55:37,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:37,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1742ce46d8ec186acca4 HTTP/1.1" 200 None 2026-03-09 18:55:37,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:37,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:55:37,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:37,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1742ce46d8ec186acca4 HTTP/1.1" 200 None 2026-03-09 18:55:37,722 - lava_callback - INFO - Completed processing callback for node 69af1742ce46d8ec186acca4 2026-03-09 18:55:46,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:46,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45374 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:57:07,449 - lava_callback - INFO - Received callback for node 69af178dce46d8ec186accb6 INFO: 172.21.0.1:45384 - "POST /node/69af178dce46d8ec186accb6 HTTP/1.0" 202 Accepted 2026-03-09 18:57:07,455 - lava_callback - INFO - Starting async job submit for node 69af178dce46d8ec186accb6 2026-03-09 18:57:07,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:07,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af178dce46d8ec186accb6 HTTP/1.1" 200 None 2026-03-09 18:57:07,502 - lava_callback - DEBUG - Processing node 69af178dce46d8ec186accb6: baseline-arm64 2026-03-09 18:57:08,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:57:08,189 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:57:08,191 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af178dce46d8ec186accb6/log.txt.gz 2026-03-09 18:57:08,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:57:08,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:57:08,726 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af178dce46d8ec186accb6/lava-logs.yaml 2026-03-09 18:57:08,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:57:08,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:57:08,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af178dce46d8ec186accb6/lava_callback.json.gz DEBUG: Setup failed for 69af178dce46d8ec186accb6. Transitting job node result: incomplete -> fail 2026-03-09 18:57:08,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:08,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af178dce46d8ec186accb6 HTTP/1.1" 200 None 2026-03-09 18:57:08,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:08,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:57:08,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:09,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af178dce46d8ec186accb6 HTTP/1.1" 200 None 2026-03-09 18:57:09,084 - lava_callback - INFO - Completed processing callback for node 69af178dce46d8ec186accb6 INFO: 172.21.0.1:58524 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:57:16,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:16,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58538 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:57:23,208 - lava_callback - INFO - Received callback for node 69ac750546f7ae8ff3366190 INFO: 172.21.0.1:49804 - "POST /node/69ac750546f7ae8ff3366190 HTTP/1.0" 202 Accepted 2026-03-09 18:57:23,213 - lava_callback - INFO - Starting async job submit for node 69ac750546f7ae8ff3366190 2026-03-09 18:57:23,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:23,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac750546f7ae8ff3366190 HTTP/1.1" 200 None 2026-03-09 18:57:23,255 - lava_callback - DEBUG - Processing node 69ac750546f7ae8ff3366190: baseline-arm-clabbe 2026-03-09 18:57:23,255 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac750546f7ae8ff3366190 2026-03-09 18:57:23,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:23,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac750546f7ae8ff3366190 HTTP/1.1" 200 None 2026-03-09 18:57:23,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:23,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:57:23,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:23,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac750546f7ae8ff3366190 HTTP/1.1" 200 None 2026-03-09 18:57:23,339 - lava_callback - ERROR - Error processing callback for node 69ac750546f7ae8ff3366190: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:58:03,185 - lava_callback - INFO - Received callback for node 69ac752946f7ae8ff336619d INFO: 172.21.0.1:53708 - "POST /node/69ac752946f7ae8ff336619d HTTP/1.0" 202 Accepted 2026-03-09 18:58:03,191 - lava_callback - INFO - Starting async job submit for node 69ac752946f7ae8ff336619d 2026-03-09 18:58:03,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:03,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac752946f7ae8ff336619d HTTP/1.1" 200 None 2026-03-09 18:58:03,211 - lava_callback - DEBUG - Processing node 69ac752946f7ae8ff336619d: baseline-arm-clabbe 2026-03-09 18:58:03,212 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac752946f7ae8ff336619d 2026-03-09 18:58:03,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:03,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac752946f7ae8ff336619d HTTP/1.1" 200 None 2026-03-09 18:58:03,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:03,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:58:03,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:03,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac752946f7ae8ff336619d HTTP/1.1" 200 None 2026-03-09 18:58:03,269 - lava_callback - ERROR - Error processing callback for node 69ac752946f7ae8ff336619d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34954 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:58:23,967 - lava_callback - INFO - Received callback for node 69af17ddce46d8ec186accc6 INFO: 172.21.0.1:58822 - "POST /node/69af17ddce46d8ec186accc6 HTTP/1.0" 202 Accepted 2026-03-09 18:58:23,974 - lava_callback - INFO - Starting async job submit for node 69af17ddce46d8ec186accc6 2026-03-09 18:58:24,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:24,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af17ddce46d8ec186accc6 HTTP/1.1" 200 None 2026-03-09 18:58:24,021 - lava_callback - DEBUG - Processing node 69af17ddce46d8ec186accc6: baseline-nfs-arm64 2026-03-09 18:58:24,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:58:24,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:58:24,583 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af17ddce46d8ec186accc6/log.txt.gz 2026-03-09 18:58:24,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:58:25,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:58:25,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af17ddce46d8ec186accc6/lava-logs.yaml 2026-03-09 18:58:25,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:58:25,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:58:25,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af17ddce46d8ec186accc6/lava_callback.json.gz DEBUG: Setup failed for 69af17ddce46d8ec186accc6. Transitting job node result: incomplete -> fail 2026-03-09 18:58:25,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:25,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af17ddce46d8ec186accc6 HTTP/1.1" 200 None 2026-03-09 18:58:25,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:25,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:58:25,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:25,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af17ddce46d8ec186accc6 HTTP/1.1" 200 None 2026-03-09 18:58:25,473 - lava_callback - INFO - Completed processing callback for node 69af17ddce46d8ec186accc6 INFO: 172.21.0.1:58828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36688 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:58:43,260 - lava_callback - INFO - Received callback for node 69ac754e46f7ae8ff33661b2 INFO: 172.21.0.1:34544 - "POST /node/69ac754e46f7ae8ff33661b2 HTTP/1.0" 202 Accepted 2026-03-09 18:58:43,266 - lava_callback - INFO - Starting async job submit for node 69ac754e46f7ae8ff33661b2 2026-03-09 18:58:43,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:43,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac754e46f7ae8ff33661b2 HTTP/1.1" 200 None 2026-03-09 18:58:43,282 - lava_callback - DEBUG - Processing node 69ac754e46f7ae8ff33661b2: baseline-arm-clabbe 2026-03-09 18:58:43,282 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac754e46f7ae8ff33661b2 2026-03-09 18:58:43,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:43,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac754e46f7ae8ff33661b2 HTTP/1.1" 200 None 2026-03-09 18:58:43,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:43,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:58:43,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:43,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac754e46f7ae8ff33661b2 HTTP/1.1" 200 None 2026-03-09 18:58:43,348 - lava_callback - ERROR - Error processing callback for node 69ac754e46f7ae8ff33661b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 18:58:46,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:46,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45006 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:59:25,157 - lava_callback - INFO - Received callback for node 69af1839ce46d8ec186accde INFO: 172.21.0.1:56436 - "POST /node/69af1839ce46d8ec186accde HTTP/1.0" 202 Accepted 2026-03-09 18:59:25,163 - lava_callback - INFO - Starting async job submit for node 69af1839ce46d8ec186accde 2026-03-09 18:59:25,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:59:25,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1839ce46d8ec186accde HTTP/1.1" 200 None 2026-03-09 18:59:25,210 - lava_callback - DEBUG - Processing node 69af1839ce46d8ec186accde: baseline-arm64-foundriesio 2026-03-09 18:59:25,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:59:25,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:59:25,575 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af1839ce46d8ec186accde/log.txt.gz 2026-03-09 18:59:25,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:59:25,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:59:25,923 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af1839ce46d8ec186accde/lava-logs.yaml 2026-03-09 18:59:25,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:59:26,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:59:26,124 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af1839ce46d8ec186accde/lava_callback.json.gz DEBUG: Setup failed for 69af1839ce46d8ec186accde. Transitting job node result: incomplete -> fail 2026-03-09 18:59:26,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:59:26,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1839ce46d8ec186accde HTTP/1.1" 200 None 2026-03-09 18:59:26,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:59:26,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:59:26,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:59:26,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1839ce46d8ec186accde HTTP/1.1" 200 None 2026-03-09 18:59:26,321 - lava_callback - INFO - Completed processing callback for node 69af1839ce46d8ec186accde INFO: 172.21.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56072 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:59:46,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:59:46,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53828 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:01:01,526 - lava_callback - INFO - Received callback for node 69af1877ce46d8ec186accef INFO: 172.21.0.1:55830 - "POST /node/69af1877ce46d8ec186accef HTTP/1.0" 202 Accepted 2026-03-09 19:01:01,532 - lava_callback - INFO - Starting async job submit for node 69af1877ce46d8ec186accef 2026-03-09 19:01:01,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:01,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1877ce46d8ec186accef HTTP/1.1" 200 None 2026-03-09 19:01:01,587 - lava_callback - DEBUG - Processing node 69af1877ce46d8ec186accef: baseline-nfs-arm64 2026-03-09 19:01:01,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:01:02,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:01:02,053 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1877ce46d8ec186accef/log.txt.gz 2026-03-09 19:01:02,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:01:02,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:01:02,545 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1877ce46d8ec186accef/lava-logs.yaml 2026-03-09 19:01:02,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:01:02,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:01:02,699 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1877ce46d8ec186accef/lava_callback.json.gz DEBUG: Setup failed for 69af1877ce46d8ec186accef. Transitting job node result: incomplete -> fail 2026-03-09 19:01:02,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:02,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1877ce46d8ec186accef HTTP/1.1" 200 None 2026-03-09 19:01:02,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:02,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:01:02,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:02,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1877ce46d8ec186accef HTTP/1.1" 200 None 2026-03-09 19:01:02,904 - lava_callback - INFO - Completed processing callback for node 69af1877ce46d8ec186accef INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52242 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:01:16,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:16,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50372 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:01:59,024 - lava_callback - INFO - Received callback for node 69af18c3ce46d8ec186acd01 INFO: 172.21.0.1:50380 - "POST /node/69af18c3ce46d8ec186acd01 HTTP/1.0" 202 Accepted 2026-03-09 19:01:59,031 - lava_callback - INFO - Starting async job submit for node 69af18c3ce46d8ec186acd01 2026-03-09 19:01:59,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:59,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af18c3ce46d8ec186acd01 HTTP/1.1" 200 None 2026-03-09 19:01:59,079 - lava_callback - DEBUG - Processing node 69af18c3ce46d8ec186acd01: baseline-nfs-arm64 2026-03-09 19:01:59,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:02:00,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:02:00,085 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af18c3ce46d8ec186acd01/log.txt.gz 2026-03-09 19:02:00,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:02:00,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:02:00,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af18c3ce46d8ec186acd01/lava-logs.yaml 2026-03-09 19:02:00,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:02:00,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:02:00,951 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af18c3ce46d8ec186acd01/lava_callback.json.gz DEBUG: Setup failed for 69af18c3ce46d8ec186acd01. Transitting job node result: incomplete -> fail 2026-03-09 19:02:01,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:02:01,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af18c3ce46d8ec186acd01 HTTP/1.1" 200 None 2026-03-09 19:02:01,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:02:01,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:02:01,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:02:01,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af18c3ce46d8ec186acd01 HTTP/1.1" 200 None 2026-03-09 19:02:01,155 - lava_callback - INFO - Completed processing callback for node 69af18c3ce46d8ec186acd01 INFO: 172.21.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33776 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:02:16,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:02:16,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40554 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:03:33,479 - lava_callback - INFO - Received callback for node 69af1912ce46d8ec186acd14 INFO: 172.21.0.1:52594 - "POST /node/69af1912ce46d8ec186acd14 HTTP/1.0" 202 Accepted 2026-03-09 19:03:33,486 - lava_callback - INFO - Starting async job submit for node 69af1912ce46d8ec186acd14 2026-03-09 19:03:33,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:03:33,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1912ce46d8ec186acd14 HTTP/1.1" 200 None 2026-03-09 19:03:33,533 - lava_callback - DEBUG - Processing node 69af1912ce46d8ec186acd14: baseline-nfs-arm64 2026-03-09 19:03:34,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:03:34,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:03:34,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1912ce46d8ec186acd14/log.txt.gz 2026-03-09 19:03:34,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:03:34,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:03:34,750 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1912ce46d8ec186acd14/lava-logs.yaml 2026-03-09 19:03:34,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:03:34,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:03:34,934 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1912ce46d8ec186acd14/lava_callback.json.gz DEBUG: Setup failed for 69af1912ce46d8ec186acd14. Transitting job node result: incomplete -> fail 2026-03-09 19:03:35,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:03:35,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1912ce46d8ec186acd14 HTTP/1.1" 200 None 2026-03-09 19:03:35,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:03:35,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:03:35,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:03:35,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1912ce46d8ec186acd14 HTTP/1.1" 200 None 2026-03-09 19:03:35,124 - lava_callback - INFO - Completed processing callback for node 69af1912ce46d8ec186acd14 INFO: 172.21.0.1:52600 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:03:46,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:03:46,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46280 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:05:02,096 - lava_callback - INFO - Received callback for node 69af1964ce46d8ec186acd23 INFO: 172.21.0.1:48848 - "POST /node/69af1964ce46d8ec186acd23 HTTP/1.0" 202 Accepted 2026-03-09 19:05:02,102 - lava_callback - INFO - Starting async job submit for node 69af1964ce46d8ec186acd23 2026-03-09 19:05:02,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:02,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1964ce46d8ec186acd23 HTTP/1.1" 200 None 2026-03-09 19:05:02,169 - lava_callback - DEBUG - Processing node 69af1964ce46d8ec186acd23: baseline-nfs-arm64 2026-03-09 19:05:02,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:02,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:02,712 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1964ce46d8ec186acd23/log.txt.gz 2026-03-09 19:05:02,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:03,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:03,196 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1964ce46d8ec186acd23/lava-logs.yaml 2026-03-09 19:05:03,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:03,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:03,358 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1964ce46d8ec186acd23/lava_callback.json.gz DEBUG: Setup failed for 69af1964ce46d8ec186acd23. Transitting job node result: incomplete -> fail 2026-03-09 19:05:03,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:03,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1964ce46d8ec186acd23 HTTP/1.1" 200 None 2026-03-09 19:05:03,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:03,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:05:03,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:03,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1964ce46d8ec186acd23 HTTP/1.1" 200 None 2026-03-09 19:05:03,558 - lava_callback - INFO - Completed processing callback for node 69af1964ce46d8ec186acd23 INFO: 172.21.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42898 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:05:16,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:16,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43068 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:05:57,117 - lava_callback - INFO - Received callback for node 69af19b3ce46d8ec186acd35 INFO: 172.21.0.1:43080 - "POST /node/69af19b3ce46d8ec186acd35 HTTP/1.0" 202 Accepted 2026-03-09 19:05:57,124 - lava_callback - INFO - Starting async job submit for node 69af19b3ce46d8ec186acd35 2026-03-09 19:05:57,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:57,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af19b3ce46d8ec186acd35 HTTP/1.1" 200 None 2026-03-09 19:05:57,171 - lava_callback - DEBUG - Processing node 69af19b3ce46d8ec186acd35: baseline-nfs-arm64 2026-03-09 19:05:57,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:57,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:57,725 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af19b3ce46d8ec186acd35/log.txt.gz INFO: 172.21.0.1:43096 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:05:57,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:58,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:58,214 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af19b3ce46d8ec186acd35/lava-logs.yaml 2026-03-09 19:05:58,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:58,469 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:58,471 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af19b3ce46d8ec186acd35/lava_callback.json.gz DEBUG: Setup failed for 69af19b3ce46d8ec186acd35. Transitting job node result: incomplete -> fail 2026-03-09 19:05:58,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:58,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af19b3ce46d8ec186acd35 HTTP/1.1" 200 None 2026-03-09 19:05:58,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:58,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:05:58,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:58,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af19b3ce46d8ec186acd35 HTTP/1.1" 200 None 2026-03-09 19:05:58,674 - lava_callback - INFO - Completed processing callback for node 69af19b3ce46d8ec186acd35 INFO: 172.21.0.1:52036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34840 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:06:16,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:06:16,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60664 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:08:35,381 - lava_callback - INFO - Received callback for node 69af19f7ce46d8ec186acd45 INFO: 172.21.0.1:48554 - "POST /node/69af19f7ce46d8ec186acd45 HTTP/1.0" 202 Accepted 2026-03-09 19:08:35,387 - lava_callback - INFO - Starting async job submit for node 69af19f7ce46d8ec186acd45 2026-03-09 19:08:35,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:08:35,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af19f7ce46d8ec186acd45 HTTP/1.1" 200 None 2026-03-09 19:08:35,433 - lava_callback - DEBUG - Processing node 69af19f7ce46d8ec186acd45: baseline-arm64 2026-03-09 19:08:35,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:08:35,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:08:35,950 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af19f7ce46d8ec186acd45/log.txt.gz 2026-03-09 19:08:36,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:08:36,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:08:36,412 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af19f7ce46d8ec186acd45/lava-logs.yaml 2026-03-09 19:08:36,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:48570 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:08:36,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:08:36,588 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af19f7ce46d8ec186acd45/lava_callback.json.gz DEBUG: Setup failed for 69af19f7ce46d8ec186acd45. Transitting job node result: incomplete -> fail 2026-03-09 19:08:36,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:08:36,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af19f7ce46d8ec186acd45 HTTP/1.1" 200 None 2026-03-09 19:08:36,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:08:36,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:08:36,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:08:36,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af19f7ce46d8ec186acd45 HTTP/1.1" 200 None 2026-03-09 19:08:36,787 - lava_callback - INFO - Completed processing callback for node 69af19f7ce46d8ec186acd45 2026-03-09 19:08:46,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:08:46,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41416 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:09:05,144 - lava_callback - INFO - Received callback for node 69af1640ce46d8ec186acc72 INFO: 172.21.0.1:59796 - "POST /node/69af1640ce46d8ec186acc72 HTTP/1.0" 202 Accepted 2026-03-09 19:09:05,193 - lava_callback - INFO - Starting async job submit for node 69af1640ce46d8ec186acc72 2026-03-09 19:09:05,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:09:05,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1640ce46d8ec186acc72 HTTP/1.1" 200 None 2026-03-09 19:09:05,232 - lava_callback - DEBUG - Processing node 69af1640ce46d8ec186acc72: baseline-arm-pengutronix 2026-03-09 19:09:05,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:09:05,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:09:05,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af1640ce46d8ec186acc72/log.txt.gz 2026-03-09 19:09:05,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:09:05,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:09:05,852 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af1640ce46d8ec186acc72/lava-logs.yaml 2026-03-09 19:09:05,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:09:05,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:09:05,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af1640ce46d8ec186acc72/lava_callback.json.gz DEBUG: Setup failed for 69af1640ce46d8ec186acc72. Transitting job node result: incomplete -> fail 2026-03-09 19:09:06,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:09:06,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1640ce46d8ec186acc72 HTTP/1.1" 200 None 2026-03-09 19:09:06,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:09:06,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 19:09:06,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:09:06,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1640ce46d8ec186acc72 HTTP/1.1" 200 None 2026-03-09 19:09:06,151 - lava_callback - INFO - Completed processing callback for node 69af1640ce46d8ec186acc72 INFO: 172.21.0.1:59802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60808 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:09:16,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:09:16,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58074 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:10:38,839 - lava_callback - INFO - Received callback for node 69af1ad7ce46d8ec186acd73 INFO: 172.21.0.1:58084 - "POST /node/69af1ad7ce46d8ec186acd73 HTTP/1.0" 202 Accepted 2026-03-09 19:10:38,845 - lava_callback - INFO - Starting async job submit for node 69af1ad7ce46d8ec186acd73 2026-03-09 19:10:38,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:38,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1ad7ce46d8ec186acd73 HTTP/1.1" 200 None 2026-03-09 19:10:38,891 - lava_callback - DEBUG - Processing node 69af1ad7ce46d8ec186acd73: baseline-arm64 2026-03-09 19:10:39,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:10:39,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:10:39,431 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1ad7ce46d8ec186acd73/log.txt.gz 2026-03-09 19:10:39,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:10:39,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:10:39,913 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1ad7ce46d8ec186acd73/lava-logs.yaml 2026-03-09 19:10:39,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:10:40,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:10:40,049 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1ad7ce46d8ec186acd73/lava_callback.json.gz DEBUG: Setup failed for 69af1ad7ce46d8ec186acd73. Transitting job node result: incomplete -> fail 2026-03-09 19:10:40,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:40,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1ad7ce46d8ec186acd73 HTTP/1.1" 200 None 2026-03-09 19:10:40,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:40,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:10:40,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:40,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1ad7ce46d8ec186acd73 HTTP/1.1" 200 None 2026-03-09 19:10:40,258 - lava_callback - INFO - Completed processing callback for node 69af1ad7ce46d8ec186acd73 2026-03-09 19:10:46,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:46,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52144 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:10:51,961 - lava_callback - INFO - Received callback for node 69ac783846f7ae8ff336624e INFO: 172.21.0.1:55078 - "POST /node/69ac783846f7ae8ff336624e HTTP/1.0" 202 Accepted 2026-03-09 19:10:51,967 - lava_callback - INFO - Starting async job submit for node 69ac783846f7ae8ff336624e 2026-03-09 19:10:51,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:51,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac783846f7ae8ff336624e HTTP/1.1" 200 None 2026-03-09 19:10:51,983 - lava_callback - DEBUG - Processing node 69ac783846f7ae8ff336624e: baseline-arm64-baylibre 2026-03-09 19:10:51,983 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac783846f7ae8ff336624e 2026-03-09 19:10:51,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:51,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac783846f7ae8ff336624e HTTP/1.1" 200 None 2026-03-09 19:10:51,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:52,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:10:52,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:52,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac783846f7ae8ff336624e HTTP/1.1" 200 None 2026-03-09 19:10:52,041 - lava_callback - ERROR - Error processing callback for node 69ac783846f7ae8ff336624e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42678 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:11:29,062 - lava_callback - INFO - Received callback for node 69af1a89ce46d8ec186acd5a INFO: 172.21.0.1:42692 - "POST /node/69af1a89ce46d8ec186acd5a HTTP/1.0" 202 Accepted 2026-03-09 19:11:29,068 - lava_callback - INFO - Starting async job submit for node 69af1a89ce46d8ec186acd5a 2026-03-09 19:11:29,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:29,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1a89ce46d8ec186acd5a HTTP/1.1" 200 None 2026-03-09 19:11:29,119 - lava_callback - DEBUG - Processing node 69af1a89ce46d8ec186acd5a: baseline-nfs-arm64 2026-03-09 19:11:29,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:11:29,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:11:29,680 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1a89ce46d8ec186acd5a/log.txt.gz 2026-03-09 19:11:29,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:11:30,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:11:30,102 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1a89ce46d8ec186acd5a/lava-logs.yaml 2026-03-09 19:11:30,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:11:30,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:11:30,248 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1a89ce46d8ec186acd5a/lava_callback.json.gz DEBUG: Setup failed for 69af1a89ce46d8ec186acd5a. Transitting job node result: incomplete -> fail 2026-03-09 19:11:30,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:30,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1a89ce46d8ec186acd5a HTTP/1.1" 200 None 2026-03-09 19:11:30,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:30,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:11:30,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:30,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1a89ce46d8ec186acd5a HTTP/1.1" 200 None 2026-03-09 19:11:30,504 - lava_callback - INFO - Completed processing callback for node 69af1a89ce46d8ec186acd5a INFO: 172.21.0.1:41160 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:11:46,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:46,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35420 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:11:57,474 - lava_callback - INFO - Received callback for node 69af1b6dce46d8ec186acd91 INFO: 172.21.0.1:35424 - "POST /node/69af1b6dce46d8ec186acd91 HTTP/1.0" 202 Accepted 2026-03-09 19:11:57,480 - lava_callback - INFO - Starting async job submit for node 69af1b6dce46d8ec186acd91 2026-03-09 19:11:57,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:57,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1b6dce46d8ec186acd91 HTTP/1.1" 200 None 2026-03-09 19:11:57,497 - lava_callback - DEBUG - Processing node 69af1b6dce46d8ec186acd91: baseline-arm64 2026-03-09 19:11:57,501 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:11:57,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:11:57,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:11:57,632 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1b6dce46d8ec186acd91/lava-logs.yaml 2026-03-09 19:11:57,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:11:57,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:11:57,765 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1b6dce46d8ec186acd91/lava_callback.json.gz 2026-03-09 19:11:57,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:57,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1b6dce46d8ec186acd91 HTTP/1.1" 200 None 2026-03-09 19:11:57,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:57,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:11:57,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:57,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1b6dce46d8ec186acd91 HTTP/1.1" 200 None 2026-03-09 19:11:57,844 - lava_callback - INFO - Completed processing callback for node 69af1b6dce46d8ec186acd91 INFO: 172.21.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53068 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:12:11,981 - lava_callback - INFO - Received callback for node 69ac788746f7ae8ff3366271 INFO: 172.21.0.1:56678 - "POST /node/69ac788746f7ae8ff3366271 HTTP/1.0" 202 Accepted 2026-03-09 19:12:11,987 - lava_callback - INFO - Starting async job submit for node 69ac788746f7ae8ff3366271 2026-03-09 19:12:11,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:12:12,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac788746f7ae8ff3366271 HTTP/1.1" 200 None 2026-03-09 19:12:12,009 - lava_callback - DEBUG - Processing node 69ac788746f7ae8ff3366271: baseline-arm64-baylibre 2026-03-09 19:12:12,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac788746f7ae8ff3366271 2026-03-09 19:12:12,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:12:12,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac788746f7ae8ff3366271 HTTP/1.1" 200 None 2026-03-09 19:12:12,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:12:12,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:12:12,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:12:12,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac788746f7ae8ff3366271 HTTP/1.1" 200 None 2026-03-09 19:12:12,062 - lava_callback - ERROR - Error processing callback for node 69ac788746f7ae8ff3366271: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56690 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:12:16,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:12:16,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57088 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:13:32,090 - lava_callback - INFO - Received callback for node 69ac78d246f7ae8ff336628a INFO: 172.21.0.1:37932 - "POST /node/69ac78d246f7ae8ff336628a HTTP/1.0" 202 Accepted 2026-03-09 19:13:32,096 - lava_callback - INFO - Starting async job submit for node 69ac78d246f7ae8ff336628a 2026-03-09 19:13:32,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:13:32,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac78d246f7ae8ff336628a HTTP/1.1" 200 None 2026-03-09 19:13:32,112 - lava_callback - DEBUG - Processing node 69ac78d246f7ae8ff336628a: baseline-arm64-baylibre 2026-03-09 19:13:32,113 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac78d246f7ae8ff336628a 2026-03-09 19:13:32,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:13:32,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac78d246f7ae8ff336628a HTTP/1.1" 200 None 2026-03-09 19:13:32,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:13:32,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:13:32,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:13:32,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac78d246f7ae8ff336628a HTTP/1.1" 200 None 2026-03-09 19:13:32,171 - lava_callback - ERROR - Error processing callback for node 69ac78d246f7ae8ff336628a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:14:37,902 - lava_callback - INFO - Received callback for node 69af1c07ce46d8ec186acdab INFO: 172.21.0.1:40514 - "POST /node/69af1c07ce46d8ec186acdab HTTP/1.0" 202 Accepted 2026-03-09 19:14:37,908 - lava_callback - INFO - Starting async job submit for node 69af1c07ce46d8ec186acdab 2026-03-09 19:14:37,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:37,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1c07ce46d8ec186acdab HTTP/1.1" 200 None 2026-03-09 19:14:37,926 - lava_callback - DEBUG - Processing node 69af1c07ce46d8ec186acdab: baseline-arm64 2026-03-09 19:14:37,931 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:14:37,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:14:38,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:14:38,085 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1c07ce46d8ec186acdab/lava-logs.yaml 2026-03-09 19:14:38,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:14:38,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:14:38,228 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1c07ce46d8ec186acdab/lava_callback.json.gz 2026-03-09 19:14:38,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:38,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1c07ce46d8ec186acdab HTTP/1.1" 200 None 2026-03-09 19:14:38,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:38,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:14:38,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:38,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1c07ce46d8ec186acdab HTTP/1.1" 200 None 2026-03-09 19:14:38,299 - lava_callback - INFO - Completed processing callback for node 69af1c07ce46d8ec186acdab 2026-03-09 19:14:46,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:46,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38144 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:14:52,145 - lava_callback - INFO - Received callback for node 69ac791c46f7ae8ff33662b2 INFO: 172.21.0.1:45296 - "POST /node/69ac791c46f7ae8ff33662b2 HTTP/1.0" 202 Accepted 2026-03-09 19:14:52,151 - lava_callback - INFO - Starting async job submit for node 69ac791c46f7ae8ff33662b2 2026-03-09 19:14:52,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:52,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac791c46f7ae8ff33662b2 HTTP/1.1" 200 None 2026-03-09 19:14:52,167 - lava_callback - DEBUG - Processing node 69ac791c46f7ae8ff33662b2: baseline-arm64-baylibre 2026-03-09 19:14:52,167 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac791c46f7ae8ff33662b2 2026-03-09 19:14:52,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:52,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac791c46f7ae8ff33662b2 HTTP/1.1" 200 None 2026-03-09 19:14:52,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:52,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:14:52,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:52,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac791c46f7ae8ff33662b2 HTTP/1.1" 200 None 2026-03-09 19:14:52,238 - lava_callback - ERROR - Error processing callback for node 69ac791c46f7ae8ff33662b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39242 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:15:36,114 - lava_callback - INFO - Received callback for node 69af1b34ce46d8ec186acd85 INFO: 172.21.0.1:47192 - "POST /node/69af1b34ce46d8ec186acd85 HTTP/1.0" 202 Accepted 2026-03-09 19:15:36,121 - lava_callback - INFO - Starting async job submit for node 69af1b34ce46d8ec186acd85 2026-03-09 19:15:36,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:36,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1b34ce46d8ec186acd85 HTTP/1.1" 200 None 2026-03-09 19:15:36,175 - lava_callback - DEBUG - Processing node 69af1b34ce46d8ec186acd85: baseline-arm64-broonie INFO: 172.21.0.1:47194 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:15:36,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:36,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:36,804 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1b34ce46d8ec186acd85/log.txt.gz 2026-03-09 19:15:37,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:37,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:37,247 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1b34ce46d8ec186acd85/lava-logs.yaml 2026-03-09 19:15:37,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:37,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:37,450 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1b34ce46d8ec186acd85/lava_callback.json.gz DEBUG: Setup failed for 69af1b34ce46d8ec186acd85. Transitting job node result: incomplete -> fail 2026-03-09 19:15:37,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:37,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1b34ce46d8ec186acd85 HTTP/1.1" 200 None 2026-03-09 19:15:37,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:37,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-09 19:15:37,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:37,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1b34ce46d8ec186acd85 HTTP/1.1" 200 None 2026-03-09 19:15:37,679 - lava_callback - INFO - Completed processing callback for node 69af1b34ce46d8ec186acd85 2026-03-09 19:15:45,104 - lava_callback - INFO - Received callback for node 69ae6fb08ef4424e487b5ebc INFO: 172.21.0.1:38936 - "POST /node/69ae6fb08ef4424e487b5ebc HTTP/1.0" 202 Accepted 2026-03-09 19:15:45,109 - lava_callback - INFO - Starting async job submit for node 69ae6fb08ef4424e487b5ebc 2026-03-09 19:15:45,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:45,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6fb08ef4424e487b5ebc HTTP/1.1" 200 None 2026-03-09 19:15:45,142 - lava_callback - DEBUG - Processing node 69ae6fb08ef4424e487b5ebc: baseline-x86 2026-03-09 19:15:45,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:45,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:45,629 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ae6fb08ef4424e487b5ebc/log.txt.gz 2026-03-09 19:15:45,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:46,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:46,039 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ae6fb08ef4424e487b5ebc/lava-logs.yaml 2026-03-09 19:15:46,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:46,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:46,196 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ae6fb08ef4424e487b5ebc/lava_callback.json.gz 2026-03-09 19:15:46,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:46,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6fb08ef4424e487b5ebc HTTP/1.1" 200 None 2026-03-09 19:15:46,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:46,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 19:15:46,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:46,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6fb08ef4424e487b5ebc HTTP/1.1" 200 None 2026-03-09 19:15:46,292 - lava_callback - INFO - Completed processing callback for node 69ae6fb08ef4424e487b5ebc 2026-03-09 19:15:46,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:38950 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:15:46,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:15:46,885 - lava_callback - INFO - Received callback for node 69ae1a5b8ef4424e487b3966 INFO: 172.21.0.1:38952 - "POST /node/69ae1a5b8ef4424e487b3966 HTTP/1.0" 202 Accepted 2026-03-09 19:15:46,891 - lava_callback - INFO - Starting async job submit for node 69ae1a5b8ef4424e487b3966 2026-03-09 19:15:46,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:46,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a5b8ef4424e487b3966 HTTP/1.1" 200 None 2026-03-09 19:15:46,926 - lava_callback - DEBUG - Processing node 69ae1a5b8ef4424e487b3966: baseline-x86 2026-03-09 19:15:47,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:47,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:47,379 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a5b8ef4424e487b3966/log.txt.gz 2026-03-09 19:15:47,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:47,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:47,755 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a5b8ef4424e487b3966/lava-logs.yaml 2026-03-09 19:15:47,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:47,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:47,920 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a5b8ef4424e487b3966/lava_callback.json.gz 2026-03-09 19:15:47,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:47,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a5b8ef4424e487b3966 HTTP/1.1" 200 None 2026-03-09 19:15:47,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:47,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 19:15:47,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:47,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1a5b8ef4424e487b3966 HTTP/1.1" 200 None 2026-03-09 19:15:47,994 - lava_callback - INFO - Completed processing callback for node 69ae1a5b8ef4424e487b3966 2026-03-09 19:15:52,157 - lava_callback - INFO - Received callback for node 69ac796546f7ae8ff33662bf INFO: 172.21.0.1:55440 - "POST /node/69ac796546f7ae8ff33662bf HTTP/1.0" 202 Accepted 2026-03-09 19:15:52,163 - lava_callback - INFO - Starting async job submit for node 69ac796546f7ae8ff33662bf 2026-03-09 19:15:52,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:52,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac796546f7ae8ff33662bf HTTP/1.1" 200 None 2026-03-09 19:15:52,180 - lava_callback - DEBUG - Processing node 69ac796546f7ae8ff33662bf: baseline-arm64-baylibre 2026-03-09 19:15:52,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac796546f7ae8ff33662bf 2026-03-09 19:15:52,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:52,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac796546f7ae8ff33662bf HTTP/1.1" 200 None 2026-03-09 19:15:52,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:52,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:15:52,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:52,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac796546f7ae8ff33662bf HTTP/1.1" 200 None 2026-03-09 19:15:52,244 - lava_callback - ERROR - Error processing callback for node 69ac796546f7ae8ff33662bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47976 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:16:16,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:47982 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:16:16,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34218 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:17:12,287 - lava_callback - INFO - Received callback for node 69ac79b046f7ae8ff33662e0 INFO: 172.21.0.1:59504 - "POST /node/69ac79b046f7ae8ff33662e0 HTTP/1.0" 202 Accepted 2026-03-09 19:17:12,292 - lava_callback - INFO - Starting async job submit for node 69ac79b046f7ae8ff33662e0 2026-03-09 19:17:12,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:12,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac79b046f7ae8ff33662e0 HTTP/1.1" 200 None 2026-03-09 19:17:12,309 - lava_callback - DEBUG - Processing node 69ac79b046f7ae8ff33662e0: baseline-arm64-baylibre 2026-03-09 19:17:12,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac79b046f7ae8ff33662e0 2026-03-09 19:17:12,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:12,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac79b046f7ae8ff33662e0 HTTP/1.1" 200 None 2026-03-09 19:17:12,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:12,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:17:12,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:12,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac79b046f7ae8ff33662e0 HTTP/1.1" 200 None 2026-03-09 19:17:12,365 - lava_callback - ERROR - Error processing callback for node 69ac79b046f7ae8ff33662e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56976 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:17:26,664 - lava_callback - INFO - Received callback for node 69ae875fc660ead8a389adf6 INFO: 172.21.0.1:56978 - "POST /node/69ae875fc660ead8a389adf6 HTTP/1.0" 202 Accepted 2026-03-09 19:17:26,670 - lava_callback - INFO - Starting async job submit for node 69ae875fc660ead8a389adf6 2026-03-09 19:17:26,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:26,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae875fc660ead8a389adf6 HTTP/1.1" 200 None 2026-03-09 19:17:26,704 - lava_callback - DEBUG - Processing node 69ae875fc660ead8a389adf6: baseline-x86 2026-03-09 19:17:27,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:17:27,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:17:27,159 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ae875fc660ead8a389adf6/log.txt.gz 2026-03-09 19:17:27,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:17:27,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:17:27,559 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ae875fc660ead8a389adf6/lava-logs.yaml 2026-03-09 19:17:27,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:17:27,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:17:27,714 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ae875fc660ead8a389adf6/lava_callback.json.gz 2026-03-09 19:17:27,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:27,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae875fc660ead8a389adf6 HTTP/1.1" 200 None 2026-03-09 19:17:27,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:27,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8240c660ead8a389ab2d HTTP/1.1" 200 None 2026-03-09 19:17:27,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:27,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae875fc660ead8a389adf6 HTTP/1.1" 200 None 2026-03-09 19:17:27,800 - lava_callback - INFO - Completed processing callback for node 69ae875fc660ead8a389adf6 INFO: 172.21.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60908 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:17:46,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:46,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:18:08,923 - lava_callback - INFO - Received callback for node 69af1bcace46d8ec186acda1 INFO: 172.21.0.1:52314 - "POST /node/69af1bcace46d8ec186acda1 HTTP/1.0" 202 Accepted 2026-03-09 19:18:08,929 - lava_callback - INFO - Starting async job submit for node 69af1bcace46d8ec186acda1 2026-03-09 19:18:08,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:08,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1bcace46d8ec186acda1 HTTP/1.1" 200 None 2026-03-09 19:18:09,000 - lava_callback - DEBUG - Processing node 69af1bcace46d8ec186acda1: baseline-arm64-broonie 2026-03-09 19:18:09,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:18:09,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:18:09,393 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1bcace46d8ec186acda1/log.txt.gz 2026-03-09 19:18:09,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:18:09,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:18:09,713 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1bcace46d8ec186acda1/lava-logs.yaml 2026-03-09 19:18:09,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:18:09,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:18:09,845 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1bcace46d8ec186acda1/lava_callback.json.gz DEBUG: Setup failed for 69af1bcace46d8ec186acda1. Transitting job node result: incomplete -> fail 2026-03-09 19:18:09,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:09,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1bcace46d8ec186acda1 HTTP/1.1" 200 None 2026-03-09 19:18:09,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:09,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:18:09,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:09,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1bcace46d8ec186acda1 HTTP/1.1" 200 None 2026-03-09 19:18:10,058 - lava_callback - INFO - Completed processing callback for node 69af1bcace46d8ec186acda1 INFO: 172.21.0.1:35704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35712 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:18:16,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:16,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44244 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:18:32,355 - lava_callback - INFO - Received callback for node 69ac79fa46f7ae8ff33662f7 INFO: 172.21.0.1:55188 - "POST /node/69ac79fa46f7ae8ff33662f7 HTTP/1.0" 202 Accepted 2026-03-09 19:18:32,360 - lava_callback - INFO - Starting async job submit for node 69ac79fa46f7ae8ff33662f7 2026-03-09 19:18:32,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:32,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac79fa46f7ae8ff33662f7 HTTP/1.1" 200 None 2026-03-09 19:18:32,376 - lava_callback - DEBUG - Processing node 69ac79fa46f7ae8ff33662f7: baseline-arm64-baylibre 2026-03-09 19:18:32,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac79fa46f7ae8ff33662f7 2026-03-09 19:18:32,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:32,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac79fa46f7ae8ff33662f7 HTTP/1.1" 200 None 2026-03-09 19:18:32,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:32,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:18:32,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:32,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac79fa46f7ae8ff33662f7 HTTP/1.1" 200 None 2026-03-09 19:18:32,439 - lava_callback - ERROR - Error processing callback for node 69ac79fa46f7ae8ff33662f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50784 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:19:52,491 - lava_callback - INFO - Received callback for node 69ac7a4346f7ae8ff3366318 INFO: 172.21.0.1:54300 - "POST /node/69ac7a4346f7ae8ff3366318 HTTP/1.0" 202 Accepted 2026-03-09 19:19:52,496 - lava_callback - INFO - Starting async job submit for node 69ac7a4346f7ae8ff3366318 2026-03-09 19:19:52,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:19:52,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7a4346f7ae8ff3366318 HTTP/1.1" 200 None 2026-03-09 19:19:52,512 - lava_callback - DEBUG - Processing node 69ac7a4346f7ae8ff3366318: baseline-arm64-baylibre 2026-03-09 19:19:52,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7a4346f7ae8ff3366318 2026-03-09 19:19:52,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:19:52,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7a4346f7ae8ff3366318 HTTP/1.1" 200 None 2026-03-09 19:19:52,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:19:52,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:19:52,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:19:52,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7a4346f7ae8ff3366318 HTTP/1.1" 200 None 2026-03-09 19:19:52,569 - lava_callback - ERROR - Error processing callback for node 69ac7a4346f7ae8ff3366318: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46276 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:20:37,501 - lava_callback - INFO - Received callback for node 69aec064feb96b28aa34ab95 INFO: 172.21.0.1:46288 - "POST /node/69aec064feb96b28aa34ab95 HTTP/1.0" 202 Accepted 2026-03-09 19:20:37,507 - lava_callback - INFO - Starting async job submit for node 69aec064feb96b28aa34ab95 2026-03-09 19:20:37,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:37,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec064feb96b28aa34ab95 HTTP/1.1" 200 None 2026-03-09 19:20:37,550 - lava_callback - DEBUG - Processing node 69aec064feb96b28aa34ab95: baseline-x86 2026-03-09 19:20:37,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:20:38,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:38,039 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aec064feb96b28aa34ab95/log.txt.gz 2026-03-09 19:20:38,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:20:38,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:38,437 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aec064feb96b28aa34ab95/lava-logs.yaml 2026-03-09 19:20:38,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:20:38,602 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:38,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aec064feb96b28aa34ab95/lava_callback.json.gz 2026-03-09 19:20:38,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:38,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec064feb96b28aa34ab95 HTTP/1.1" 200 None 2026-03-09 19:20:38,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:38,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba54feb96b28aa34a79d HTTP/1.1" 200 None 2026-03-09 19:20:38,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:38,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aec064feb96b28aa34ab95 HTTP/1.1" 200 None 2026-03-09 19:20:38,677 - lava_callback - INFO - Completed processing callback for node 69aec064feb96b28aa34ab95 INFO: 172.21.0.1:40230 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:20:46,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:46,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:20:52,462 - lava_callback - INFO - Received callback for node 69ac7a8e46f7ae8ff3366326 INFO: 172.21.0.1:45514 - "POST /node/69ac7a8e46f7ae8ff3366326 HTTP/1.0" 202 Accepted 2026-03-09 19:20:52,467 - lava_callback - INFO - Starting async job submit for node 69ac7a8e46f7ae8ff3366326 2026-03-09 19:20:52,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:52,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7a8e46f7ae8ff3366326 HTTP/1.1" 200 None 2026-03-09 19:20:52,488 - lava_callback - DEBUG - Processing node 69ac7a8e46f7ae8ff3366326: baseline-arm64-baylibre 2026-03-09 19:20:52,488 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7a8e46f7ae8ff3366326 2026-03-09 19:20:52,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:52,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7a8e46f7ae8ff3366326 HTTP/1.1" 200 None 2026-03-09 19:20:52,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:52,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:20:52,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:52,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7a8e46f7ae8ff3366326 HTTP/1.1" 200 None 2026-03-09 19:20:52,545 - lava_callback - ERROR - Error processing callback for node 69ac7a8e46f7ae8ff3366326: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 19:20:55,704 - lava_callback - INFO - Received callback for node 69aec870feb96b28aa34b61d INFO: 172.21.0.1:45526 - "POST /node/69aec870feb96b28aa34b61d HTTP/1.0" 202 Accepted 2026-03-09 19:20:55,710 - lava_callback - INFO - Starting async job submit for node 69aec870feb96b28aa34b61d 2026-03-09 19:20:55,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:55,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec870feb96b28aa34b61d HTTP/1.1" 200 None 2026-03-09 19:20:55,744 - lava_callback - DEBUG - Processing node 69aec870feb96b28aa34b61d: baseline-x86 2026-03-09 19:20:56,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:20:56,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:56,214 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aec870feb96b28aa34b61d/log.txt.gz 2026-03-09 19:20:56,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:45540 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:20:56,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:56,587 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aec870feb96b28aa34b61d/lava-logs.yaml 2026-03-09 19:20:56,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:20:56,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:56,758 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aec870feb96b28aa34b61d/lava_callback.json.gz 2026-03-09 19:20:56,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:56,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec870feb96b28aa34b61d HTTP/1.1" 200 None 2026-03-09 19:20:56,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:56,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 19:20:56,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:56,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aec870feb96b28aa34b61d HTTP/1.1" 200 None 2026-03-09 19:20:56,868 - lava_callback - INFO - Completed processing callback for node 69aec870feb96b28aa34b61d INFO: 172.21.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45940 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:21:16,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:16,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41932 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:21:37,802 - lava_callback - INFO - Received callback for node 69aef10efeb96b28aa34c4b5 INFO: 172.21.0.1:41944 - "POST /node/69aef10efeb96b28aa34c4b5 HTTP/1.0" 202 Accepted 2026-03-09 19:21:37,808 - lava_callback - INFO - Starting async job submit for node 69aef10efeb96b28aa34c4b5 2026-03-09 19:21:37,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:37,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef10efeb96b28aa34c4b5 HTTP/1.1" 200 None 2026-03-09 19:21:37,844 - lava_callback - DEBUG - Processing node 69aef10efeb96b28aa34c4b5: baseline-x86 2026-03-09 19:21:38,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:38,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:38,251 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aef10efeb96b28aa34c4b5/log.txt.gz 2026-03-09 19:21:38,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:38,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:38,656 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aef10efeb96b28aa34c4b5/lava-logs.yaml 2026-03-09 19:21:38,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:38,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:38,808 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aef10efeb96b28aa34c4b5/lava_callback.json.gz 2026-03-09 19:21:38,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:38,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef10efeb96b28aa34c4b5 HTTP/1.1" 200 None 2026-03-09 19:21:38,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:38,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8240c660ead8a389ab2d HTTP/1.1" 200 None 2026-03-09 19:21:38,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:38,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef10efeb96b28aa34c4b5 HTTP/1.1" 200 None 2026-03-09 19:21:38,896 - lava_callback - INFO - Completed processing callback for node 69aef10efeb96b28aa34c4b5 INFO: 172.21.0.1:43876 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:21:46,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:46,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:21:54,304 - lava_callback - INFO - Received callback for node 69af1c67ce46d8ec186acdcd INFO: 172.21.0.1:57300 - "POST /node/69af1c67ce46d8ec186acdcd HTTP/1.0" 202 Accepted 2026-03-09 19:21:54,311 - lava_callback - INFO - Starting async job submit for node 69af1c67ce46d8ec186acdcd 2026-03-09 19:21:54,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:54,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1c67ce46d8ec186acdcd HTTP/1.1" 200 None 2026-03-09 19:21:54,361 - lava_callback - DEBUG - Processing node 69af1c67ce46d8ec186acdcd: baseline-arm64-broonie 2026-03-09 19:21:54,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:54,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:54,930 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1c67ce46d8ec186acdcd/log.txt.gz 2026-03-09 19:21:55,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:55,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:55,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1c67ce46d8ec186acdcd/lava-logs.yaml 2026-03-09 19:21:55,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:55,572 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:55,574 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1c67ce46d8ec186acdcd/lava_callback.json.gz DEBUG: Setup failed for 69af1c67ce46d8ec186acdcd. Transitting job node result: incomplete -> fail 2026-03-09 19:21:55,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:55,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1c67ce46d8ec186acdcd HTTP/1.1" 200 None 2026-03-09 19:21:55,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:55,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:21:55,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:55,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1c67ce46d8ec186acdcd HTTP/1.1" 200 None 2026-03-09 19:21:55,805 - lava_callback - INFO - Completed processing callback for node 69af1c67ce46d8ec186acdcd INFO: 172.21.0.1:57310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42810 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:22:07,765 - lava_callback - INFO - Received callback for node 69aeffbace46d8ec186ac444 INFO: 172.21.0.1:42824 - "POST /node/69aeffbace46d8ec186ac444 HTTP/1.0" 202 Accepted 2026-03-09 19:22:07,770 - lava_callback - INFO - Starting async job submit for node 69aeffbace46d8ec186ac444 2026-03-09 19:22:07,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:07,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffbace46d8ec186ac444 HTTP/1.1" 200 None 2026-03-09 19:22:07,812 - lava_callback - DEBUG - Processing node 69aeffbace46d8ec186ac444: baseline-x86 2026-03-09 19:22:08,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:22:08,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:22:08,282 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffbace46d8ec186ac444/log.txt.gz 2026-03-09 19:22:08,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:22:08,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:22:08,666 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffbace46d8ec186ac444/lava-logs.yaml 2026-03-09 19:22:08,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:22:08,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:22:08,831 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffbace46d8ec186ac444/lava_callback.json.gz 2026-03-09 19:22:08,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:08,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffbace46d8ec186ac444 HTTP/1.1" 200 None 2026-03-09 19:22:08,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:08,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef449ce46d8ec186ab83b HTTP/1.1" 200 None 2026-03-09 19:22:08,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:08,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aeffbace46d8ec186ac444 HTTP/1.1" 200 None 2026-03-09 19:22:08,920 - lava_callback - INFO - Completed processing callback for node 69aeffbace46d8ec186ac444 2026-03-09 19:22:12,599 - lava_callback - INFO - Received callback for node 69ac7ad746f7ae8ff3366347 INFO: 172.21.0.1:54046 - "POST /node/69ac7ad746f7ae8ff3366347 HTTP/1.0" 202 Accepted 2026-03-09 19:22:12,604 - lava_callback - INFO - Starting async job submit for node 69ac7ad746f7ae8ff3366347 2026-03-09 19:22:12,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:12,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7ad746f7ae8ff3366347 HTTP/1.1" 200 None 2026-03-09 19:22:12,628 - lava_callback - DEBUG - Processing node 69ac7ad746f7ae8ff3366347: baseline-arm64-baylibre 2026-03-09 19:22:12,628 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7ad746f7ae8ff3366347 2026-03-09 19:22:12,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:12,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7ad746f7ae8ff3366347 HTTP/1.1" 200 None 2026-03-09 19:22:12,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:12,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:22:12,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:12,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7ad746f7ae8ff3366347 HTTP/1.1" 200 None 2026-03-09 19:22:12,686 - lava_callback - ERROR - Error processing callback for node 69ac7ad746f7ae8ff3366347: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54058 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:22:16,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:16,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34926 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:24:32,733 - lava_callback - INFO - Received callback for node 69ac7b6c46f7ae8ff336637e INFO: 172.21.0.1:38114 - "POST /node/69ac7b6c46f7ae8ff336637e HTTP/1.0" 202 Accepted 2026-03-09 19:24:32,742 - lava_callback - INFO - Starting async job submit for node 69ac7b6c46f7ae8ff336637e 2026-03-09 19:24:32,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:24:32,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7b6c46f7ae8ff336637e HTTP/1.1" 200 None 2026-03-09 19:24:32,758 - lava_callback - DEBUG - Processing node 69ac7b6c46f7ae8ff336637e: baseline-arm64-baylibre 2026-03-09 19:24:32,758 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7b6c46f7ae8ff336637e 2026-03-09 19:24:32,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:24:32,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7b6c46f7ae8ff336637e HTTP/1.1" 200 None 2026-03-09 19:24:32,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:24:32,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:24:32,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:24:32,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7b6c46f7ae8ff336637e HTTP/1.1" 200 None 2026-03-09 19:24:32,819 - lava_callback - ERROR - Error processing callback for node 69ac7b6c46f7ae8ff336637e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33836 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:25:29,368 - lava_callback - INFO - Received callback for node 69af1cb5ce46d8ec186acddd INFO: 172.21.0.1:33852 - "POST /node/69af1cb5ce46d8ec186acddd HTTP/1.0" 202 Accepted 2026-03-09 19:25:29,374 - lava_callback - INFO - Starting async job submit for node 69af1cb5ce46d8ec186acddd 2026-03-09 19:25:29,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:25:29,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1cb5ce46d8ec186acddd HTTP/1.1" 200 None 2026-03-09 19:25:29,428 - lava_callback - DEBUG - Processing node 69af1cb5ce46d8ec186acddd: baseline-arm64-broonie 2026-03-09 19:25:29,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:25:29,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:25:29,928 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1cb5ce46d8ec186acddd/log.txt.gz 2026-03-09 19:25:30,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:25:30,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:25:30,380 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1cb5ce46d8ec186acddd/lava-logs.yaml 2026-03-09 19:25:30,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:25:30,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:25:30,553 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1cb5ce46d8ec186acddd/lava_callback.json.gz DEBUG: Setup failed for 69af1cb5ce46d8ec186acddd. Transitting job node result: incomplete -> fail 2026-03-09 19:25:30,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:25:30,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1cb5ce46d8ec186acddd HTTP/1.1" 200 None 2026-03-09 19:25:30,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:25:30,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:25:30,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:25:30,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1cb5ce46d8ec186acddd HTTP/1.1" 200 None 2026-03-09 19:25:30,792 - lava_callback - INFO - Completed processing callback for node 69af1cb5ce46d8ec186acddd INFO: 172.21.0.1:43790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51966 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:25:46,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:25:46,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41646 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:31:14,059 - lava_callback - INFO - Received callback for node 69af1e36ce46d8ec186ace7b INFO: 172.21.0.1:41682 - "POST /node/69af1e36ce46d8ec186ace7b HTTP/1.0" 202 Accepted 2026-03-09 19:31:14,064 - lava_callback - INFO - Starting async job submit for node 69af1e36ce46d8ec186ace7b 2026-03-09 19:31:14,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:31:14,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1e36ce46d8ec186ace7b HTTP/1.1" 200 None 2026-03-09 19:31:14,082 - lava_callback - DEBUG - Processing node 69af1e36ce46d8ec186ace7b: baseline-arm64-broonie 2026-03-09 19:31:14,087 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:31:14,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:31:14,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:31:14,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1e36ce46d8ec186ace7b/lava-logs.yaml 2026-03-09 19:31:14,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:31:14,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:31:14,429 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1e36ce46d8ec186ace7b/lava_callback.json.gz 2026-03-09 19:31:14,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:31:14,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1e36ce46d8ec186ace7b HTTP/1.1" 200 None 2026-03-09 19:31:14,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:31:14,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:31:14,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:31:14,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1e36ce46d8ec186ace7b HTTP/1.1" 200 None 2026-03-09 19:31:14,498 - lava_callback - INFO - Completed processing callback for node 69af1e36ce46d8ec186ace7b INFO: 172.21.0.1:41692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41702 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:31:16,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:31:16,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39988 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:32:35,156 - lava_callback - INFO - Received callback for node 69af203dce46d8ec186aced5 INFO: 172.21.0.1:37334 - "POST /node/69af203dce46d8ec186aced5 HTTP/1.0" 202 Accepted 2026-03-09 19:32:35,162 - lava_callback - INFO - Starting async job submit for node 69af203dce46d8ec186aced5 2026-03-09 19:32:35,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:32:35,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af203dce46d8ec186aced5 HTTP/1.1" 200 None 2026-03-09 19:32:35,180 - lava_callback - DEBUG - Processing node 69af203dce46d8ec186aced5: baseline-arm64 2026-03-09 19:32:35,186 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:32:35,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:32:35,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:32:35,348 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af203dce46d8ec186aced5/lava-logs.yaml 2026-03-09 19:32:35,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:32:35,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:32:35,497 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af203dce46d8ec186aced5/lava_callback.json.gz 2026-03-09 19:32:35,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:32:35,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af203dce46d8ec186aced5 HTTP/1.1" 200 None 2026-03-09 19:32:35,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:32:35,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:32:35,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:32:35,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af203dce46d8ec186aced5 HTTP/1.1" 200 None 2026-03-09 19:32:35,563 - lava_callback - INFO - Completed processing callback for node 69af203dce46d8ec186aced5 INFO: 172.21.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:32:46,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:32:46,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60890 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:33:33,291 - lava_callback - INFO - Received callback for node 69ac7d7d46f7ae8ff3366448 INFO: 172.21.0.1:42734 - "POST /node/69ac7d7d46f7ae8ff3366448 HTTP/1.0" 202 Accepted 2026-03-09 19:33:33,296 - lava_callback - INFO - Starting async job submit for node 69ac7d7d46f7ae8ff3366448 2026-03-09 19:33:33,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:33:33,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7d7d46f7ae8ff3366448 HTTP/1.1" 200 None 2026-03-09 19:33:33,313 - lava_callback - DEBUG - Processing node 69ac7d7d46f7ae8ff3366448: baseline-arm64-baylibre 2026-03-09 19:33:33,313 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7d7d46f7ae8ff3366448 2026-03-09 19:33:33,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:33:33,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7d7d46f7ae8ff3366448 HTTP/1.1" 200 None 2026-03-09 19:33:33,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:33:33,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:33:33,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:33:33,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7d7d46f7ae8ff3366448 HTTP/1.1" 200 None 2026-03-09 19:33:33,372 - lava_callback - ERROR - Error processing callback for node 69ac7d7d46f7ae8ff3366448: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41998 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:34:33,244 - lava_callback - INFO - Received callback for node 69ac7dbe46f7ae8ff336645c INFO: 172.21.0.1:46136 - "POST /node/69ac7dbe46f7ae8ff336645c HTTP/1.0" 202 Accepted 2026-03-09 19:34:33,249 - lava_callback - INFO - Starting async job submit for node 69ac7dbe46f7ae8ff336645c 2026-03-09 19:34:33,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:34:33,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7dbe46f7ae8ff336645c HTTP/1.1" 200 None 2026-03-09 19:34:33,266 - lava_callback - DEBUG - Processing node 69ac7dbe46f7ae8ff336645c: baseline-arm-baylibre 2026-03-09 19:34:33,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7dbe46f7ae8ff336645c 2026-03-09 19:34:33,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:34:33,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7dbe46f7ae8ff336645c HTTP/1.1" 200 None 2026-03-09 19:34:33,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:34:33,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:34:33,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:34:33,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7dbe46f7ae8ff336645c HTTP/1.1" 200 None 2026-03-09 19:34:33,331 - lava_callback - ERROR - Error processing callback for node 69ac7dbe46f7ae8ff336645c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40820 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:35:13,321 - lava_callback - INFO - Received callback for node 69ac7dec46f7ae8ff3366479 INFO: 172.21.0.1:48710 - "POST /node/69ac7dec46f7ae8ff3366479 HTTP/1.0" 202 Accepted 2026-03-09 19:35:13,326 - lava_callback - INFO - Starting async job submit for node 69ac7dec46f7ae8ff3366479 2026-03-09 19:35:13,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:35:13,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7dec46f7ae8ff3366479 HTTP/1.1" 200 None 2026-03-09 19:35:13,342 - lava_callback - DEBUG - Processing node 69ac7dec46f7ae8ff3366479: baseline-arm64-baylibre 2026-03-09 19:35:13,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7dec46f7ae8ff3366479 2026-03-09 19:35:13,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:35:13,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7dec46f7ae8ff3366479 HTTP/1.1" 200 None 2026-03-09 19:35:13,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:35:13,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:35:13,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:35:13,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7dec46f7ae8ff3366479 HTTP/1.1" 200 None 2026-03-09 19:35:13,400 - lava_callback - ERROR - Error processing callback for node 69ac7dec46f7ae8ff3366479: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32996 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:36:33,409 - lava_callback - INFO - Received callback for node 69ac7e2c46f7ae8ff3366489 INFO: 172.21.0.1:35236 - "POST /node/69ac7e2c46f7ae8ff3366489 HTTP/1.0" 202 Accepted 2026-03-09 19:36:33,415 - lava_callback - INFO - Starting async job submit for node 69ac7e2c46f7ae8ff3366489 2026-03-09 19:36:33,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:36:33,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e2c46f7ae8ff3366489 HTTP/1.1" 200 None 2026-03-09 19:36:33,430 - lava_callback - DEBUG - Processing node 69ac7e2c46f7ae8ff3366489: baseline-arm-baylibre 2026-03-09 19:36:33,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7e2c46f7ae8ff3366489 2026-03-09 19:36:33,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:36:33,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e2c46f7ae8ff3366489 HTTP/1.1" 200 None 2026-03-09 19:36:33,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:36:33,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:36:33,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:36:33,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7e2c46f7ae8ff3366489 HTTP/1.1" 200 None 2026-03-09 19:36:33,529 - lava_callback - ERROR - Error processing callback for node 69ac7e2c46f7ae8ff3366489: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42130 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:37:13,404 - lava_callback - INFO - Received callback for node 69ac7e5a46f7ae8ff336649b INFO: 172.21.0.1:54368 - "POST /node/69ac7e5a46f7ae8ff336649b HTTP/1.0" 202 Accepted 2026-03-09 19:37:13,409 - lava_callback - INFO - Starting async job submit for node 69ac7e5a46f7ae8ff336649b 2026-03-09 19:37:13,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:37:13,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e5a46f7ae8ff336649b HTTP/1.1" 200 None 2026-03-09 19:37:13,425 - lava_callback - DEBUG - Processing node 69ac7e5a46f7ae8ff336649b: baseline-arm64-baylibre 2026-03-09 19:37:13,425 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7e5a46f7ae8ff336649b 2026-03-09 19:37:13,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:37:13,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e5a46f7ae8ff336649b HTTP/1.1" 200 None 2026-03-09 19:37:13,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:37:13,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:37:13,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:37:13,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7e5a46f7ae8ff336649b HTTP/1.1" 200 None 2026-03-09 19:37:13,488 - lava_callback - ERROR - Error processing callback for node 69ac7e5a46f7ae8ff336649b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50962 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:38:07,594 - lava_callback - INFO - Received callback for node 69af1f0ace46d8ec186acea5 INFO: 172.21.0.1:50972 - "POST /node/69af1f0ace46d8ec186acea5 HTTP/1.0" 202 Accepted 2026-03-09 19:38:07,604 - lava_callback - INFO - Starting async job submit for node 69af1f0ace46d8ec186acea5 2026-03-09 19:38:07,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:07,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1f0ace46d8ec186acea5 HTTP/1.1" 200 None 2026-03-09 19:38:07,661 - lava_callback - DEBUG - Processing node 69af1f0ace46d8ec186acea5: baseline-arm64 2026-03-09 19:38:07,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:38:08,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:38:08,149 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f0ace46d8ec186acea5/log.txt.gz 2026-03-09 19:38:08,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:38:09,705 - lava_callback - INFO - Received callback for node 69af2176ce46d8ec186aceff INFO: 172.21.0.1:50832 - "POST /node/69af2176ce46d8ec186aceff HTTP/1.0" 202 Accepted 2026-03-09 19:38:09,711 - lava_callback - INFO - Starting async job submit for node 69af2176ce46d8ec186aceff 2026-03-09 19:38:09,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:09,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2176ce46d8ec186aceff HTTP/1.1" 200 None 2026-03-09 19:38:09,743 - lava_callback - DEBUG - Processing node 69af2176ce46d8ec186aceff: baseline-arm64 2026-03-09 19:38:09,793 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:38:09,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:38:09,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:38:09,977 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2176ce46d8ec186aceff/lava-logs.yaml 2026-03-09 19:38:09,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:38:10,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:38:10,119 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2176ce46d8ec186aceff/lava_callback.json.gz 2026-03-09 19:38:10,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:10,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2176ce46d8ec186aceff HTTP/1.1" 200 None 2026-03-09 19:38:10,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:10,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:38:10,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:10,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2176ce46d8ec186aceff HTTP/1.1" 200 None 2026-03-09 19:38:10,239 - lava_callback - INFO - Completed processing callback for node 69af2176ce46d8ec186aceff 2026-03-09 19:38:12,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:38:12,541 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f0ace46d8ec186acea5/lava-logs.yaml 2026-03-09 19:38:12,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:38:12,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:38:12,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f0ace46d8ec186acea5/lava_callback.json.gz 2026-03-09 19:38:12,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:12,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1f0ace46d8ec186acea5 HTTP/1.1" 200 None 2026-03-09 19:38:12,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:12,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:38:12,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:12,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1f0ace46d8ec186acea5 HTTP/1.1" 200 None 2026-03-09 19:38:12,847 - lava_callback - INFO - Completed processing callback for node 69af1f0ace46d8ec186acea5 2026-03-09 19:38:13,423 - lava_callback - INFO - Received callback for node 69ac7e9a46f7ae8ff33664b6 INFO: 172.21.0.1:50840 - "POST /node/69ac7e9a46f7ae8ff33664b6 HTTP/1.0" 202 Accepted 2026-03-09 19:38:13,428 - lava_callback - INFO - Starting async job submit for node 69ac7e9a46f7ae8ff33664b6 2026-03-09 19:38:13,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:13,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e9a46f7ae8ff33664b6 HTTP/1.1" 200 None 2026-03-09 19:38:13,445 - lava_callback - DEBUG - Processing node 69ac7e9a46f7ae8ff33664b6: baseline-arm-baylibre 2026-03-09 19:38:13,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7e9a46f7ae8ff33664b6 2026-03-09 19:38:13,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:13,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e9a46f7ae8ff33664b6 HTTP/1.1" 200 None 2026-03-09 19:38:13,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:13,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:38:13,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:13,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7e9a46f7ae8ff33664b6 HTTP/1.1" 200 None 2026-03-09 19:38:13,499 - lava_callback - ERROR - Error processing callback for node 69ac7e9a46f7ae8ff33664b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50866 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:38:16,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:16,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56238 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:39:03,039 - lava_callback - INFO - Received callback for node 69af1f5ace46d8ec186aceaf INFO: 172.21.0.1:60836 - "POST /node/69af1f5ace46d8ec186aceaf HTTP/1.0" 202 Accepted 2026-03-09 19:39:03,052 - lava_callback - INFO - Starting async job submit for node 69af1f5ace46d8ec186aceaf 2026-03-09 19:39:03,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:03,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1f5ace46d8ec186aceaf HTTP/1.1" 200 None 2026-03-09 19:39:03,094 - lava_callback - DEBUG - Processing node 69af1f5ace46d8ec186aceaf: baseline-arm64 2026-03-09 19:39:03,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:39:03,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:39:03,764 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f5ace46d8ec186aceaf/log.txt.gz 2026-03-09 19:39:04,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:39:04,309 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:39:04,311 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f5ace46d8ec186aceaf/lava-logs.yaml 2026-03-09 19:39:04,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:39:04,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:39:04,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f5ace46d8ec186aceaf/lava_callback.json.gz 2026-03-09 19:39:04,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:04,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1f5ace46d8ec186aceaf HTTP/1.1" 200 None 2026-03-09 19:39:04,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:04,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:39:04,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:04,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1f5ace46d8ec186aceaf HTTP/1.1" 200 None 2026-03-09 19:39:04,613 - lava_callback - INFO - Completed processing callback for node 69af1f5ace46d8ec186aceaf INFO: 172.21.0.1:60838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36386 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:39:16,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:16,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54920 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:39:53,585 - lava_callback - INFO - Received callback for node 69ac7f0646f7ae8ff33664db INFO: 172.21.0.1:52116 - "POST /node/69ac7f0646f7ae8ff33664db HTTP/1.0" 202 Accepted 2026-03-09 19:39:53,590 - lava_callback - INFO - Starting async job submit for node 69ac7f0646f7ae8ff33664db 2026-03-09 19:39:53,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:53,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7f0646f7ae8ff33664db HTTP/1.1" 200 None 2026-03-09 19:39:53,607 - lava_callback - DEBUG - Processing node 69ac7f0646f7ae8ff33664db: baseline-arm-baylibre 2026-03-09 19:39:53,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7f0646f7ae8ff33664db 2026-03-09 19:39:53,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:53,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7f0646f7ae8ff33664db HTTP/1.1" 200 None 2026-03-09 19:39:53,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:53,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:39:53,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:53,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7f0646f7ae8ff33664db HTTP/1.1" 200 None 2026-03-09 19:39:53,670 - lava_callback - ERROR - Error processing callback for node 69ac7f0646f7ae8ff33664db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50578 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:40:44,066 - lava_callback - INFO - Received callback for node 69af221bce46d8ec186acf1c INFO: 172.21.0.1:57548 - "POST /node/69af221bce46d8ec186acf1c HTTP/1.0" 202 Accepted 2026-03-09 19:40:44,072 - lava_callback - INFO - Starting async job submit for node 69af221bce46d8ec186acf1c 2026-03-09 19:40:44,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:40:44,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af221bce46d8ec186acf1c HTTP/1.1" 200 None 2026-03-09 19:40:44,089 - lava_callback - DEBUG - Processing node 69af221bce46d8ec186acf1c: baseline-nfs-arm64 2026-03-09 19:40:44,091 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:40:44,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:40:44,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:40:44,283 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af221bce46d8ec186acf1c/lava-logs.yaml 2026-03-09 19:40:44,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:40:44,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:40:44,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af221bce46d8ec186acf1c/lava_callback.json.gz 2026-03-09 19:40:44,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:40:44,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af221bce46d8ec186acf1c HTTP/1.1" 200 None 2026-03-09 19:40:44,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:40:44,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:40:44,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:40:44,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af221bce46d8ec186acf1c HTTP/1.1" 200 None 2026-03-09 19:40:44,489 - lava_callback - INFO - Completed processing callback for node 69af221bce46d8ec186acf1c INFO: 172.21.0.1:57564 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:40:46,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:40:46,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59826 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:41:53,699 - lava_callback - INFO - Received callback for node 69ac7f7346f7ae8ff33664fa INFO: 172.21.0.1:42390 - "POST /node/69ac7f7346f7ae8ff33664fa HTTP/1.0" 202 Accepted 2026-03-09 19:41:53,704 - lava_callback - INFO - Starting async job submit for node 69ac7f7346f7ae8ff33664fa 2026-03-09 19:41:53,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:41:53,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7f7346f7ae8ff33664fa HTTP/1.1" 200 None 2026-03-09 19:41:53,720 - lava_callback - DEBUG - Processing node 69ac7f7346f7ae8ff33664fa: baseline-arm-baylibre 2026-03-09 19:41:53,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7f7346f7ae8ff33664fa 2026-03-09 19:41:53,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:41:53,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7f7346f7ae8ff33664fa HTTP/1.1" 200 None 2026-03-09 19:41:53,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:41:53,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:41:53,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:41:53,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7f7346f7ae8ff33664fa HTTP/1.1" 200 None 2026-03-09 19:41:53,772 - lava_callback - ERROR - Error processing callback for node 69ac7f7346f7ae8ff33664fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42418 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:42:05,873 - lava_callback - INFO - Received callback for node 69af2266ce46d8ec186acf23 INFO: 172.21.0.1:57534 - "POST /node/69af2266ce46d8ec186acf23 HTTP/1.0" 202 Accepted 2026-03-09 19:42:05,880 - lava_callback - INFO - Starting async job submit for node 69af2266ce46d8ec186acf23 2026-03-09 19:42:05,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:05,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2266ce46d8ec186acf23 HTTP/1.1" 200 None 2026-03-09 19:42:05,911 - lava_callback - DEBUG - Processing node 69af2266ce46d8ec186acf23: baseline-arm64 2026-03-09 19:42:05,961 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:42:05,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:42:06,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:42:06,173 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2266ce46d8ec186acf23/lava-logs.yaml 2026-03-09 19:42:06,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:42:06,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:42:06,315 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2266ce46d8ec186acf23/lava_callback.json.gz 2026-03-09 19:42:06,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:06,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2266ce46d8ec186acf23 HTTP/1.1" 200 None 2026-03-09 19:42:06,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:06,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:42:06,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:06,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2266ce46d8ec186acf23 HTTP/1.1" 200 None 2026-03-09 19:42:06,435 - lava_callback - INFO - Completed processing callback for node 69af2266ce46d8ec186acf23 INFO: 172.21.0.1:57546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58646 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:42:16,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:16,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41738 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:42:33,679 - lava_callback - INFO - Received callback for node 69ac7fa146f7ae8ff336651e INFO: 172.21.0.1:42066 - "POST /node/69ac7fa146f7ae8ff336651e HTTP/1.0" 202 Accepted 2026-03-09 19:42:33,684 - lava_callback - INFO - Starting async job submit for node 69ac7fa146f7ae8ff336651e 2026-03-09 19:42:33,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:33,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7fa146f7ae8ff336651e HTTP/1.1" 200 None 2026-03-09 19:42:33,700 - lava_callback - DEBUG - Processing node 69ac7fa146f7ae8ff336651e: baseline-arm64-baylibre 2026-03-09 19:42:33,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7fa146f7ae8ff336651e 2026-03-09 19:42:33,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:33,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7fa146f7ae8ff336651e HTTP/1.1" 200 None 2026-03-09 19:42:33,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:33,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:42:33,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:33,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7fa146f7ae8ff336651e HTTP/1.1" 200 None 2026-03-09 19:42:33,764 - lava_callback - ERROR - Error processing callback for node 69ac7fa146f7ae8ff336651e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39838 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:43:33,744 - lava_callback - INFO - Received callback for node 69ac7fe146f7ae8ff336652b INFO: 172.21.0.1:58332 - "POST /node/69ac7fe146f7ae8ff336652b HTTP/1.0" 202 Accepted 2026-03-09 19:43:33,750 - lava_callback - INFO - Starting async job submit for node 69ac7fe146f7ae8ff336652b 2026-03-09 19:43:33,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:33,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7fe146f7ae8ff336652b HTTP/1.1" 200 None 2026-03-09 19:43:33,777 - lava_callback - DEBUG - Processing node 69ac7fe146f7ae8ff336652b: baseline-arm-baylibre 2026-03-09 19:43:33,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7fe146f7ae8ff336652b 2026-03-09 19:43:33,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:33,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7fe146f7ae8ff336652b HTTP/1.1" 200 None 2026-03-09 19:43:33,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:33,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:43:33,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:33,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7fe146f7ae8ff336652b HTTP/1.1" 200 None 2026-03-09 19:43:33,856 - lava_callback - ERROR - Error processing callback for node 69ac7fe146f7ae8ff336652b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51950 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:43:54,610 - lava_callback - INFO - Received callback for node 69af2093ce46d8ec186acee5 INFO: 172.21.0.1:42578 - "POST /node/69af2093ce46d8ec186acee5 HTTP/1.0" 202 Accepted 2026-03-09 19:43:54,617 - lava_callback - INFO - Starting async job submit for node 69af2093ce46d8ec186acee5 2026-03-09 19:43:54,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:54,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2093ce46d8ec186acee5 HTTP/1.1" 200 None 2026-03-09 19:43:54,661 - lava_callback - DEBUG - Processing node 69af2093ce46d8ec186acee5: baseline-nfs-arm64 2026-03-09 19:43:55,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:43:55,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:43:55,452 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2093ce46d8ec186acee5/log.txt.gz 2026-03-09 19:43:55,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:43:56,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:43:56,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2093ce46d8ec186acee5/lava-logs.yaml 2026-03-09 19:43:56,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:43:56,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:43:56,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2093ce46d8ec186acee5/lava_callback.json.gz 2026-03-09 19:43:56,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:56,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2093ce46d8ec186acee5 HTTP/1.1" 200 None 2026-03-09 19:43:56,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:56,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:43:56,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:56,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2093ce46d8ec186acee5 HTTP/1.1" 200 None 2026-03-09 19:43:56,353 - lava_callback - INFO - Completed processing callback for node 69af2093ce46d8ec186acee5 INFO: 172.21.0.1:42588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40188 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:44:16,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:16,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:44:22,876 - lava_callback - INFO - Received callback for node 69af22face46d8ec186acf38 INFO: 172.21.0.1:34598 - "POST /node/69af22face46d8ec186acf38 HTTP/1.0" 202 Accepted 2026-03-09 19:44:22,882 - lava_callback - INFO - Starting async job submit for node 69af22face46d8ec186acf38 2026-03-09 19:44:22,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:22,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af22face46d8ec186acf38 HTTP/1.1" 200 None 2026-03-09 19:44:22,914 - lava_callback - DEBUG - Processing node 69af22face46d8ec186acf38: baseline-arm64 2026-03-09 19:44:22,966 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:44:22,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:44:23,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:44:23,145 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af22face46d8ec186acf38/lava-logs.yaml 2026-03-09 19:44:23,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:44:23,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:44:23,284 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af22face46d8ec186acf38/lava_callback.json.gz 2026-03-09 19:44:23,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:23,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af22face46d8ec186acf38 HTTP/1.1" 200 None 2026-03-09 19:44:23,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:23,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:44:23,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:23,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af22face46d8ec186acf38 HTTP/1.1" 200 None 2026-03-09 19:44:23,410 - lava_callback - INFO - Completed processing callback for node 69af22face46d8ec186acf38 2026-03-09 19:44:25,315 - lava_callback - INFO - Received callback for node 69ac801446f7ae8ff336653a INFO: 172.21.0.1:34612 - "POST /node/69ac801446f7ae8ff336653a HTTP/1.0" 202 Accepted 2026-03-09 19:44:25,320 - lava_callback - INFO - Starting async job submit for node 69ac801446f7ae8ff336653a 2026-03-09 19:44:25,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:25,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac801446f7ae8ff336653a HTTP/1.1" 200 None 2026-03-09 19:44:25,336 - lava_callback - DEBUG - Processing node 69ac801446f7ae8ff336653a: baseline-arm64-clabbe 2026-03-09 19:44:25,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac801446f7ae8ff336653a 2026-03-09 19:44:25,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:25,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac801446f7ae8ff336653a HTTP/1.1" 200 None 2026-03-09 19:44:25,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:25,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:44:25,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:25,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac801446f7ae8ff336653a HTTP/1.1" 200 None 2026-03-09 19:44:25,393 - lava_callback - ERROR - Error processing callback for node 69ac801446f7ae8ff336653a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56000 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:44:46,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:46,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47158 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:45:33,949 - lava_callback - INFO - Received callback for node 69ac804b46f7ae8ff3366557 INFO: 172.21.0.1:57276 - "POST /node/69ac804b46f7ae8ff3366557 HTTP/1.0" 202 Accepted 2026-03-09 19:45:33,954 - lava_callback - INFO - Starting async job submit for node 69ac804b46f7ae8ff3366557 2026-03-09 19:45:33,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:33,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac804b46f7ae8ff3366557 HTTP/1.1" 200 None 2026-03-09 19:45:33,974 - lava_callback - DEBUG - Processing node 69ac804b46f7ae8ff3366557: baseline-arm-baylibre 2026-03-09 19:45:33,974 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac804b46f7ae8ff3366557 2026-03-09 19:45:33,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:33,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac804b46f7ae8ff3366557 HTTP/1.1" 200 None 2026-03-09 19:45:33,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:34,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:45:34,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:34,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac804b46f7ae8ff3366557 HTTP/1.1" 200 None 2026-03-09 19:45:34,036 - lava_callback - ERROR - Error processing callback for node 69ac804b46f7ae8ff3366557: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57284 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:45:39,428 - lava_callback - INFO - Received callback for node 69af2350ce46d8ec186acf4a INFO: 172.21.0.1:57286 - "POST /node/69af2350ce46d8ec186acf4a HTTP/1.0" 202 Accepted 2026-03-09 19:45:39,434 - lava_callback - INFO - Starting async job submit for node 69af2350ce46d8ec186acf4a 2026-03-09 19:45:39,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:39,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2350ce46d8ec186acf4a HTTP/1.1" 200 None 2026-03-09 19:45:39,453 - lava_callback - DEBUG - Processing node 69af2350ce46d8ec186acf4a: baseline-nfs-arm64 2026-03-09 19:45:39,459 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:45:39,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:45:39,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:45:39,612 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2350ce46d8ec186acf4a/lava-logs.yaml 2026-03-09 19:45:39,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:45:39,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:45:39,753 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2350ce46d8ec186acf4a/lava_callback.json.gz 2026-03-09 19:45:39,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:39,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2350ce46d8ec186acf4a HTTP/1.1" 200 None 2026-03-09 19:45:39,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:39,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:45:39,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:39,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2350ce46d8ec186acf4a HTTP/1.1" 200 None 2026-03-09 19:45:39,819 - lava_callback - INFO - Completed processing callback for node 69af2350ce46d8ec186acf4a INFO: 172.21.0.1:51692 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:45:46,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:46,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:45:53,866 - lava_callback - INFO - Received callback for node 69ac807046f7ae8ff3366563 INFO: 172.21.0.1:42114 - "POST /node/69ac807046f7ae8ff3366563 HTTP/1.0" 202 Accepted 2026-03-09 19:45:53,871 - lava_callback - INFO - Starting async job submit for node 69ac807046f7ae8ff3366563 2026-03-09 19:45:53,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:53,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac807046f7ae8ff3366563 HTTP/1.1" 200 None 2026-03-09 19:45:53,891 - lava_callback - DEBUG - Processing node 69ac807046f7ae8ff3366563: baseline-arm-baylibre 2026-03-09 19:45:53,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac807046f7ae8ff3366563 2026-03-09 19:45:53,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:53,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac807046f7ae8ff3366563 HTTP/1.1" 200 None 2026-03-09 19:45:53,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:53,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:45:53,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:53,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac807046f7ae8ff3366563 HTTP/1.1" 200 None 2026-03-09 19:45:53,947 - lava_callback - ERROR - Error processing callback for node 69ac807046f7ae8ff3366563: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55122 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:46:30,966 - lava_callback - INFO - Received callback for node 69af22bece46d8ec186acf31 INFO: 172.21.0.1:41626 - "POST /node/69af22bece46d8ec186acf31 HTTP/1.0" 202 Accepted 2026-03-09 19:46:30,972 - lava_callback - INFO - Starting async job submit for node 69af22bece46d8ec186acf31 2026-03-09 19:46:30,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:46:31,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af22bece46d8ec186acf31 HTTP/1.1" 200 None 2026-03-09 19:46:31,006 - lava_callback - DEBUG - Processing node 69af22bece46d8ec186acf31: baseline-arm64-broonie 2026-03-09 19:46:31,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:46:31,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:46:31,491 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af22bece46d8ec186acf31/log.txt.gz 2026-03-09 19:46:31,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:46:31,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:46:31,905 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af22bece46d8ec186acf31/lava-logs.yaml 2026-03-09 19:46:31,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:46:32,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:46:32,043 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af22bece46d8ec186acf31/lava_callback.json.gz 2026-03-09 19:46:32,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:46:32,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af22bece46d8ec186acf31 HTTP/1.1" 200 None 2026-03-09 19:46:32,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:46:32,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:46:32,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:46:32,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af22bece46d8ec186acf31 HTTP/1.1" 200 None 2026-03-09 19:46:32,206 - lava_callback - INFO - Completed processing callback for node 69af22bece46d8ec186acf31 INFO: 172.21.0.1:41642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34546 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:46:46,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:46:46,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41704 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:47:05,372 - lava_callback - INFO - Received callback for node 69ac80a646f7ae8ff3366575 INFO: 172.21.0.1:37854 - "POST /node/69ac80a646f7ae8ff3366575 HTTP/1.0" 202 Accepted 2026-03-09 19:47:05,377 - lava_callback - INFO - Starting async job submit for node 69ac80a646f7ae8ff3366575 2026-03-09 19:47:05,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:05,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac80a646f7ae8ff3366575 HTTP/1.1" 200 None 2026-03-09 19:47:05,395 - lava_callback - DEBUG - Processing node 69ac80a646f7ae8ff3366575: baseline-arm64-clabbe 2026-03-09 19:47:05,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac80a646f7ae8ff3366575 2026-03-09 19:47:05,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:05,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac80a646f7ae8ff3366575 HTTP/1.1" 200 None 2026-03-09 19:47:05,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:05,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:47:05,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:05,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac80a646f7ae8ff3366575 HTTP/1.1" 200 None 2026-03-09 19:47:05,494 - lava_callback - ERROR - Error processing callback for node 69ac80a646f7ae8ff3366575: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59288 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:47:29,625 - lava_callback - INFO - Received callback for node 69af212ece46d8ec186acef7 INFO: 172.21.0.1:59290 - "POST /node/69af212ece46d8ec186acef7 HTTP/1.0" 202 Accepted 2026-03-09 19:47:29,632 - lava_callback - INFO - Starting async job submit for node 69af212ece46d8ec186acef7 2026-03-09 19:47:29,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:29,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af212ece46d8ec186acef7 HTTP/1.1" 200 None 2026-03-09 19:47:29,679 - lava_callback - DEBUG - Processing node 69af212ece46d8ec186acef7: baseline-nfs-arm64 2026-03-09 19:47:29,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:47:30,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:47:30,137 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af212ece46d8ec186acef7/log.txt.gz 2026-03-09 19:47:30,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:47:30,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:47:30,578 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af212ece46d8ec186acef7/lava-logs.yaml 2026-03-09 19:47:30,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:47:30,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:47:30,725 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af212ece46d8ec186acef7/lava_callback.json.gz 2026-03-09 19:47:30,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:30,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af212ece46d8ec186acef7 HTTP/1.1" 200 None 2026-03-09 19:47:30,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:30,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:47:30,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:30,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af212ece46d8ec186acef7 HTTP/1.1" 200 None 2026-03-09 19:47:30,905 - lava_callback - INFO - Completed processing callback for node 69af212ece46d8ec186acef7 INFO: 172.21.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49116 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:47:46,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:46,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:47:54,040 - lava_callback - INFO - Received callback for node 69ac80df46f7ae8ff336658d INFO: 172.21.0.1:55386 - "POST /node/69ac80df46f7ae8ff336658d HTTP/1.0" 202 Accepted 2026-03-09 19:47:54,046 - lava_callback - INFO - Starting async job submit for node 69ac80df46f7ae8ff336658d 2026-03-09 19:47:54,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:54,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac80df46f7ae8ff336658d HTTP/1.1" 200 None 2026-03-09 19:47:54,062 - lava_callback - DEBUG - Processing node 69ac80df46f7ae8ff336658d: baseline-arm-baylibre 2026-03-09 19:47:54,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac80df46f7ae8ff336658d 2026-03-09 19:47:54,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:54,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac80df46f7ae8ff336658d HTTP/1.1" 200 None 2026-03-09 19:47:54,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:54,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:47:54,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:54,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac80df46f7ae8ff336658d HTTP/1.1" 200 None 2026-03-09 19:47:54,124 - lava_callback - ERROR - Error processing callback for node 69ac80df46f7ae8ff336658d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55806 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:48:26,329 - lava_callback - INFO - Received callback for node 69af23fdce46d8ec186acf6f INFO: 172.21.0.1:55808 - "POST /node/69af23fdce46d8ec186acf6f HTTP/1.0" 202 Accepted 2026-03-09 19:48:26,335 - lava_callback - INFO - Starting async job submit for node 69af23fdce46d8ec186acf6f 2026-03-09 19:48:26,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:26,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af23fdce46d8ec186acf6f HTTP/1.1" 200 None 2026-03-09 19:48:26,367 - lava_callback - DEBUG - Processing node 69af23fdce46d8ec186acf6f: baseline-arm64-foundriesio 2026-03-09 19:48:26,429 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:48:26,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:55814 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:48:26,589 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:48:26,591 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af23fdce46d8ec186acf6f/lava-logs.yaml 2026-03-09 19:48:26,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:48:26,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:48:26,721 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af23fdce46d8ec186acf6f/lava_callback.json.gz 2026-03-09 19:48:26,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:26,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af23fdce46d8ec186acf6f HTTP/1.1" 200 None 2026-03-09 19:48:26,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:26,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:48:26,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:26,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af23fdce46d8ec186acf6f HTTP/1.1" 200 None 2026-03-09 19:48:26,841 - lava_callback - INFO - Completed processing callback for node 69af23fdce46d8ec186acf6f INFO: 172.21.0.1:55826 - "GET /favicon.ico HTTP/1.0" 404 Not Found 2026-03-09 19:48:34,045 - lava_callback - INFO - Received callback for node 69ac810346f7ae8ff3366599 INFO: 172.21.0.1:38676 - "POST /node/69ac810346f7ae8ff3366599 HTTP/1.0" 202 Accepted 2026-03-09 19:48:34,051 - lava_callback - INFO - Starting async job submit for node 69ac810346f7ae8ff3366599 2026-03-09 19:48:34,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:34,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac810346f7ae8ff3366599 HTTP/1.1" 200 None 2026-03-09 19:48:34,069 - lava_callback - DEBUG - Processing node 69ac810346f7ae8ff3366599: baseline-arm-baylibre 2026-03-09 19:48:34,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac810346f7ae8ff3366599 2026-03-09 19:48:34,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:34,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac810346f7ae8ff3366599 HTTP/1.1" 200 None 2026-03-09 19:48:34,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:34,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:48:34,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:34,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac810346f7ae8ff3366599 HTTP/1.1" 200 None 2026-03-09 19:48:34,134 - lava_callback - ERROR - Error processing callback for node 69ac810346f7ae8ff3366599: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38686 - "GET /.well-known/security.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:40810 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:48:46,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:46,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49508 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:49:14,091 - lava_callback - INFO - Received callback for node 69ac812846f7ae8ff33665a9 INFO: 172.21.0.1:47602 - "POST /node/69ac812846f7ae8ff33665a9 HTTP/1.0" 202 Accepted 2026-03-09 19:49:14,097 - lava_callback - INFO - Starting async job submit for node 69ac812846f7ae8ff33665a9 2026-03-09 19:49:14,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:14,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac812846f7ae8ff33665a9 HTTP/1.1" 200 None 2026-03-09 19:49:14,117 - lava_callback - DEBUG - Processing node 69ac812846f7ae8ff33665a9: baseline-arm-baylibre 2026-03-09 19:49:14,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac812846f7ae8ff33665a9 2026-03-09 19:49:14,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:14,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac812846f7ae8ff33665a9 HTTP/1.1" 200 None 2026-03-09 19:49:14,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:14,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:49:14,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:14,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac812846f7ae8ff33665a9 HTTP/1.1" 200 None 2026-03-09 19:49:14,169 - lava_callback - ERROR - Error processing callback for node 69ac812846f7ae8ff33665a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35668 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:49:54,102 - lava_callback - INFO - Received callback for node 69ac814d46f7ae8ff33665b6 INFO: 172.21.0.1:33766 - "POST /node/69ac814d46f7ae8ff33665b6 HTTP/1.0" 202 Accepted 2026-03-09 19:49:54,108 - lava_callback - INFO - Starting async job submit for node 69ac814d46f7ae8ff33665b6 2026-03-09 19:49:54,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:54,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac814d46f7ae8ff33665b6 HTTP/1.1" 200 None 2026-03-09 19:49:54,125 - lava_callback - DEBUG - Processing node 69ac814d46f7ae8ff33665b6: baseline-arm-baylibre 2026-03-09 19:49:54,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac814d46f7ae8ff33665b6 2026-03-09 19:49:54,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:54,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac814d46f7ae8ff33665b6 HTTP/1.1" 200 None 2026-03-09 19:49:54,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:54,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:49:54,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:54,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac814d46f7ae8ff33665b6 HTTP/1.1" 200 None 2026-03-09 19:49:54,179 - lava_callback - ERROR - Error processing callback for node 69ac814d46f7ae8ff33665b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:54:36,626 - lava_callback - INFO - Received callback for node 69af251dce46d8ec186acf97 INFO: 172.21.0.1:50542 - "POST /node/69af251dce46d8ec186acf97 HTTP/1.0" 202 Accepted 2026-03-09 19:54:36,632 - lava_callback - INFO - Starting async job submit for node 69af251dce46d8ec186acf97 2026-03-09 19:54:36,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:54:36,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af251dce46d8ec186acf97 HTTP/1.1" 200 None 2026-03-09 19:54:36,663 - lava_callback - DEBUG - Processing node 69af251dce46d8ec186acf97: baseline-x86 2026-03-09 19:54:36,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:54:37,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:54:37,115 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af251dce46d8ec186acf97/log.txt.gz 2026-03-09 19:54:37,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:54:37,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:54:37,472 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af251dce46d8ec186acf97/lava-logs.yaml 2026-03-09 19:54:37,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:54:37,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:54:37,633 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af251dce46d8ec186acf97/lava_callback.json.gz 2026-03-09 19:54:37,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:54:37,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af251dce46d8ec186acf97 HTTP/1.1" 200 None 2026-03-09 19:54:37,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:54:37,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba54feb96b28aa34a79d HTTP/1.1" 200 None 2026-03-09 19:54:37,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:54:37,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af251dce46d8ec186acf97 HTTP/1.1" 200 None 2026-03-09 19:54:37,721 - lava_callback - INFO - Completed processing callback for node 69af251dce46d8ec186acf97 INFO: 172.21.0.1:48864 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:54:46,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:54:46,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43662 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33512 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:00:22,527 - lava_callback - INFO - Received callback for node 69af26bace46d8ec186acfdb INFO: 172.21.0.1:44854 - "POST /node/69af26bace46d8ec186acfdb HTTP/1.0" 202 Accepted 2026-03-09 20:00:22,533 - lava_callback - INFO - Starting async job submit for node 69af26bace46d8ec186acfdb 2026-03-09 20:00:22,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:00:22,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af26bace46d8ec186acfdb HTTP/1.1" 200 None 2026-03-09 20:00:22,551 - lava_callback - DEBUG - Processing node 69af26bace46d8ec186acfdb: baseline-arm64-clabbe 2026-03-09 20:00:22,555 - lava_callback - WARNING - Failed to upload log 2026-03-09 20:00:22,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:00:22,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:00:22,692 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af26bace46d8ec186acfdb/lava-logs.yaml 2026-03-09 20:00:22,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:00:22,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:00:22,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af26bace46d8ec186acfdb/lava_callback.json.gz 2026-03-09 20:00:22,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:00:22,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af26bace46d8ec186acfdb HTTP/1.1" 200 None 2026-03-09 20:00:22,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:00:22,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:00:22,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:00:22,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af26bace46d8ec186acfdb HTTP/1.1" 200 None 2026-03-09 20:00:22,901 - lava_callback - INFO - Completed processing callback for node 69af26bace46d8ec186acfdb INFO: 172.21.0.1:44866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58646 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:00:46,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:00:47,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44584 - "GET /login HTTP/1.0" 404 Not Found INFO: 172.21.0.1:44596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59290 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:04:10,248 - lava_callback - INFO - Received callback for node 69af2346ce46d8ec186acf47 INFO: 172.21.0.1:55954 - "POST /node/69af2346ce46d8ec186acf47 HTTP/1.0" 202 Accepted 2026-03-09 20:04:10,255 - lava_callback - INFO - Starting async job submit for node 69af2346ce46d8ec186acf47 2026-03-09 20:04:10,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:04:10,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2346ce46d8ec186acf47 HTTP/1.1" 200 None 2026-03-09 20:04:10,328 - lava_callback - DEBUG - Processing node 69af2346ce46d8ec186acf47: ltp-smoke 2026-03-09 20:04:10,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:04:10,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:04:10,982 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af2346ce46d8ec186acf47/log.txt.gz 2026-03-09 20:04:11,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:04:11,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:04:11,588 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af2346ce46d8ec186acf47/lava-logs.yaml 2026-03-09 20:04:11,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:04:11,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:04:11,811 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af2346ce46d8ec186acf47/lava_callback.json.gz 2026-03-09 20:04:11,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:04:11,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2346ce46d8ec186acf47 HTTP/1.1" 200 None 2026-03-09 20:04:11,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:04:11,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef42ace46d8ec186ab814 HTTP/1.1" 200 None 2026-03-09 20:04:11,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:04:11,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2346ce46d8ec186acf47 HTTP/1.1" 200 None 2026-03-09 20:04:12,106 - lava_callback - INFO - Completed processing callback for node 69af2346ce46d8ec186acf47 INFO: 172.21.0.1:55958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55974 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:04:17,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:04:17,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41008 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:15:11,316 - lava_callback - INFO - Received callback for node 69af2a34ce46d8ec186ad096 INFO: 172.21.0.1:39576 - "POST /node/69af2a34ce46d8ec186ad096 HTTP/1.0" 202 Accepted 2026-03-09 20:15:11,321 - lava_callback - INFO - Starting async job submit for node 69af2a34ce46d8ec186ad096 2026-03-09 20:15:11,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:15:11,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2a34ce46d8ec186ad096 HTTP/1.1" 200 None 2026-03-09 20:15:11,358 - lava_callback - DEBUG - Processing node 69af2a34ce46d8ec186ad096: baseline-arm64-broonie 2026-03-09 20:15:11,364 - lava_callback - WARNING - Failed to upload log 2026-03-09 20:15:11,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:15:11,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:15:11,498 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af2a34ce46d8ec186ad096/lava-logs.yaml 2026-03-09 20:15:11,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:15:11,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:15:11,663 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af2a34ce46d8ec186ad096/lava_callback.json.gz 2026-03-09 20:15:11,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:15:11,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2a34ce46d8ec186ad096 HTTP/1.1" 200 None 2026-03-09 20:15:11,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:15:11,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:15:11,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:15:11,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2a34ce46d8ec186ad096 HTTP/1.1" 200 None 2026-03-09 20:15:11,728 - lava_callback - INFO - Completed processing callback for node 69af2a34ce46d8ec186ad096 INFO: 172.21.0.1:39584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39598 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:15:17,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:15:17,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37134 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:17:49,685 - lava_callback - INFO - Received callback for node 69af2a95ce46d8ec186ad0a8 INFO: 172.21.0.1:37148 - "POST /node/69af2a95ce46d8ec186ad0a8 HTTP/1.0" 202 Accepted 2026-03-09 20:17:49,691 - lava_callback - INFO - Starting async job submit for node 69af2a95ce46d8ec186ad0a8 2026-03-09 20:17:49,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:17:49,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2a95ce46d8ec186ad0a8 HTTP/1.1" 200 None 2026-03-09 20:17:49,738 - lava_callback - DEBUG - Processing node 69af2a95ce46d8ec186ad0a8: baseline-arm64 2026-03-09 20:17:50,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:17:50,293 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:17:50,295 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2a95ce46d8ec186ad0a8/log.txt.gz 2026-03-09 20:17:50,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:17:50,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:17:50,793 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2a95ce46d8ec186ad0a8/lava-logs.yaml 2026-03-09 20:17:50,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:17:50,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:17:50,933 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2a95ce46d8ec186ad0a8/lava_callback.json.gz DEBUG: Setup failed for 69af2a95ce46d8ec186ad0a8. Transitting job node result: incomplete -> fail 2026-03-09 20:17:50,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:17:51,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2a95ce46d8ec186ad0a8 HTTP/1.1" 200 None 2026-03-09 20:17:51,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:17:51,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:17:51,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:17:51,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2a95ce46d8ec186ad0a8 HTTP/1.1" 200 None 2026-03-09 20:17:51,196 - lava_callback - INFO - Completed processing callback for node 69af2a95ce46d8ec186ad0a8 INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54916 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:18:17,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:18:17,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54836 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:19:14,230 - lava_callback - INFO - Received callback for node 69af2ae1ce46d8ec186ad0af INFO: 172.21.0.1:59274 - "POST /node/69af2ae1ce46d8ec186ad0af HTTP/1.0" 202 Accepted 2026-03-09 20:19:14,237 - lava_callback - INFO - Starting async job submit for node 69af2ae1ce46d8ec186ad0af 2026-03-09 20:19:14,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:19:14,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2ae1ce46d8ec186ad0af HTTP/1.1" 200 None 2026-03-09 20:19:14,286 - lava_callback - DEBUG - Processing node 69af2ae1ce46d8ec186ad0af: baseline-arm64 2026-03-09 20:19:15,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:59288 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 20:19:15,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:19:15,267 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2ae1ce46d8ec186ad0af/log.txt.gz 2026-03-09 20:19:15,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:19:16,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:19:16,018 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2ae1ce46d8ec186ad0af/lava-logs.yaml 2026-03-09 20:19:16,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:19:16,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:19:16,200 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2ae1ce46d8ec186ad0af/lava_callback.json.gz DEBUG: Setup failed for 69af2ae1ce46d8ec186ad0af. Transitting job node result: incomplete -> fail 2026-03-09 20:19:16,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:19:16,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2ae1ce46d8ec186ad0af HTTP/1.1" 200 None 2026-03-09 20:19:16,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:19:16,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:19:16,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:19:16,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2ae1ce46d8ec186ad0af HTTP/1.1" 200 None 2026-03-09 20:19:16,429 - lava_callback - INFO - Completed processing callback for node 69af2ae1ce46d8ec186ad0af INFO: 172.21.0.1:59294 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:19:17,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:19:17,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34112 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:20:36,205 - lava_callback - INFO - Received callback for node 69af2b32ce46d8ec186ad0bd INFO: 172.21.0.1:55984 - "POST /node/69af2b32ce46d8ec186ad0bd HTTP/1.0" 202 Accepted 2026-03-09 20:20:36,212 - lava_callback - INFO - Starting async job submit for node 69af2b32ce46d8ec186ad0bd 2026-03-09 20:20:36,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:20:36,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2b32ce46d8ec186ad0bd HTTP/1.1" 200 None 2026-03-09 20:20:36,257 - lava_callback - DEBUG - Processing node 69af2b32ce46d8ec186ad0bd: baseline-arm64 INFO: 172.21.0.1:56000 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:20:36,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:20:36,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:20:36,791 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b32ce46d8ec186ad0bd/log.txt.gz 2026-03-09 20:20:36,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:20:37,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:20:37,198 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b32ce46d8ec186ad0bd/lava-logs.yaml 2026-03-09 20:20:37,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:20:37,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:20:37,365 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b32ce46d8ec186ad0bd/lava_callback.json.gz DEBUG: Setup failed for 69af2b32ce46d8ec186ad0bd. Transitting job node result: incomplete -> fail 2026-03-09 20:20:37,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:20:37,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2b32ce46d8ec186ad0bd HTTP/1.1" 200 None 2026-03-09 20:20:37,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:20:37,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:20:37,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:20:37,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2b32ce46d8ec186ad0bd HTTP/1.1" 200 None 2026-03-09 20:20:37,580 - lava_callback - INFO - Completed processing callback for node 69af2b32ce46d8ec186ad0bd INFO: 172.21.0.1:57378 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:20:47,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:20:47,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50192 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:21:51,414 - lava_callback - INFO - Received callback for node 69af2b7fce46d8ec186ad0d0 INFO: 172.21.0.1:34600 - "POST /node/69af2b7fce46d8ec186ad0d0 HTTP/1.0" 202 Accepted 2026-03-09 20:21:51,420 - lava_callback - INFO - Starting async job submit for node 69af2b7fce46d8ec186ad0d0 2026-03-09 20:21:51,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:21:51,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2b7fce46d8ec186ad0d0 HTTP/1.1" 200 None 2026-03-09 20:21:51,466 - lava_callback - DEBUG - Processing node 69af2b7fce46d8ec186ad0d0: baseline-arm64 2026-03-09 20:21:51,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:21:51,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:21:51,935 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b7fce46d8ec186ad0d0/log.txt.gz 2026-03-09 20:21:52,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:21:52,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:21:52,354 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b7fce46d8ec186ad0d0/lava-logs.yaml 2026-03-09 20:21:52,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:21:52,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:21:52,517 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b7fce46d8ec186ad0d0/lava_callback.json.gz DEBUG: Setup failed for 69af2b7fce46d8ec186ad0d0. Transitting job node result: incomplete -> fail 2026-03-09 20:21:52,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:21:52,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2b7fce46d8ec186ad0d0 HTTP/1.1" 200 None 2026-03-09 20:21:52,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:21:52,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:21:52,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:21:52,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2b7fce46d8ec186ad0d0 HTTP/1.1" 200 None 2026-03-09 20:21:52,722 - lava_callback - INFO - Completed processing callback for node 69af2b7fce46d8ec186ad0d0 INFO: 172.21.0.1:34602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43870 - "GET /node/699447d75ae459b9df3daaa4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:43874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43884 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:22:17,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:22:17,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56690 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:23:48,641 - lava_callback - INFO - Received callback for node 69af2851ce46d8ec186ad031 INFO: 172.21.0.1:56694 - "POST /node/69af2851ce46d8ec186ad031 HTTP/1.0" 202 Accepted 2026-03-09 20:23:48,648 - lava_callback - INFO - Starting async job submit for node 69af2851ce46d8ec186ad031 2026-03-09 20:23:48,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:23:48,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2851ce46d8ec186ad031 HTTP/1.1" 200 None 2026-03-09 20:23:48,719 - lava_callback - DEBUG - Processing node 69af2851ce46d8ec186ad031: baseline-arm 2026-03-09 20:23:49,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:23:49,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:23:49,638 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af2851ce46d8ec186ad031/log.txt.gz 2026-03-09 20:23:49,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:23:50,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:23:50,265 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af2851ce46d8ec186ad031/lava-logs.yaml 2026-03-09 20:23:50,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:23:50,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:23:50,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af2851ce46d8ec186ad031/lava_callback.json.gz DEBUG: Setup failed for 69af2851ce46d8ec186ad031. Transitting job node result: incomplete -> fail 2026-03-09 20:23:50,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:23:50,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2851ce46d8ec186ad031 HTTP/1.1" 200 None 2026-03-09 20:23:50,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:23:50,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 20:23:50,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:23:50,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2851ce46d8ec186ad031 HTTP/1.1" 200 None 2026-03-09 20:23:50,728 - lava_callback - INFO - Completed processing callback for node 69af2851ce46d8ec186ad031 INFO: 172.21.0.1:50260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51966 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:24:17,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:17,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 20:24:23,693 - lava_callback - INFO - Received callback for node 69af2c15ce46d8ec186ad0f6 INFO: 172.21.0.1:43304 - "POST /node/69af2c15ce46d8ec186ad0f6 HTTP/1.0" 202 Accepted 2026-03-09 20:24:23,700 - lava_callback - INFO - Starting async job submit for node 69af2c15ce46d8ec186ad0f6 2026-03-09 20:24:23,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:23,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2c15ce46d8ec186ad0f6 HTTP/1.1" 200 None 2026-03-09 20:24:23,754 - lava_callback - DEBUG - Processing node 69af2c15ce46d8ec186ad0f6: baseline-arm64 2026-03-09 20:24:24,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:24:24,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:24:24,426 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2c15ce46d8ec186ad0f6/log.txt.gz 2026-03-09 20:24:24,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:24:24,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:24:24,934 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2c15ce46d8ec186ad0f6/lava-logs.yaml 2026-03-09 20:24:24,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:24:25,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:24:25,093 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2c15ce46d8ec186ad0f6/lava_callback.json.gz DEBUG: Setup failed for 69af2c15ce46d8ec186ad0f6. Transitting job node result: incomplete -> fail 2026-03-09 20:24:25,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:25,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2c15ce46d8ec186ad0f6 HTTP/1.1" 200 None 2026-03-09 20:24:25,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:25,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:24:25,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:25,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2c15ce46d8ec186ad0f6 HTTP/1.1" 200 None 2026-03-09 20:24:25,287 - lava_callback - INFO - Completed processing callback for node 69af2c15ce46d8ec186ad0f6 INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37716 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:24:47,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:47,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52730 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:26:23,933 - lava_callback - INFO - Received callback for node 69af2c8ace46d8ec186ad119 INFO: 172.21.0.1:46038 - "POST /node/69af2c8ace46d8ec186ad119 HTTP/1.0" 202 Accepted 2026-03-09 20:26:23,940 - lava_callback - INFO - Starting async job submit for node 69af2c8ace46d8ec186ad119 2026-03-09 20:26:23,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:26:23,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2c8ace46d8ec186ad119 HTTP/1.1" 200 None 2026-03-09 20:26:23,988 - lava_callback - DEBUG - Processing node 69af2c8ace46d8ec186ad119: baseline-nfs-arm64 2026-03-09 20:26:24,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:26:24,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:26:24,589 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2c8ace46d8ec186ad119/log.txt.gz 2026-03-09 20:26:24,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:26:25,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:26:25,090 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2c8ace46d8ec186ad119/lava-logs.yaml 2026-03-09 20:26:25,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:26:25,238 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:26:25,239 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2c8ace46d8ec186ad119/lava_callback.json.gz DEBUG: Setup failed for 69af2c8ace46d8ec186ad119. Transitting job node result: incomplete -> fail 2026-03-09 20:26:25,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:26:25,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2c8ace46d8ec186ad119 HTTP/1.1" 200 None 2026-03-09 20:26:25,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:26:25,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:26:25,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:26:25,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2c8ace46d8ec186ad119 HTTP/1.1" 200 None 2026-03-09 20:26:25,447 - lava_callback - INFO - Completed processing callback for node 69af2c8ace46d8ec186ad119 INFO: 172.21.0.1:46050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49206 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:26:47,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:26:47,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50990 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:28:17,034 - lava_callback - INFO - Received callback for node 69af2d0ece46d8ec186ad132 INFO: 172.21.0.1:50998 - "POST /node/69af2d0ece46d8ec186ad132 HTTP/1.0" 202 Accepted 2026-03-09 20:28:17,040 - lava_callback - INFO - Starting async job submit for node 69af2d0ece46d8ec186ad132 2026-03-09 20:28:17,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:28:17,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2d0ece46d8ec186ad132 HTTP/1.1" 200 None 2026-03-09 20:28:17,091 - lava_callback - DEBUG - Processing node 69af2d0ece46d8ec186ad132: baseline-arm64-foundriesio 2026-03-09 20:28:17,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:28:17,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:28:17,477 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af2d0ece46d8ec186ad132/log.txt.gz 2026-03-09 20:28:17,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:28:17,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:28:17,854 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af2d0ece46d8ec186ad132/lava-logs.yaml 2026-03-09 20:28:17,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:28:17,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:28:17,988 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af2d0ece46d8ec186ad132/lava_callback.json.gz DEBUG: Setup failed for 69af2d0ece46d8ec186ad132. Transitting job node result: incomplete -> fail 2026-03-09 20:28:18,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:28:18,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2d0ece46d8ec186ad132 HTTP/1.1" 200 None 2026-03-09 20:28:18,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:28:18,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:28:18,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:28:18,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2d0ece46d8ec186ad132 HTTP/1.1" 200 None 2026-03-09 20:28:18,188 - lava_callback - INFO - Completed processing callback for node 69af2d0ece46d8ec186ad132 INFO: 172.21.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37194 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:28:47,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:28:47,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50428 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:31:08,815 - lava_callback - INFO - Received callback for node 69af2d99ce46d8ec186ad150 INFO: 172.21.0.1:50436 - "POST /node/69af2d99ce46d8ec186ad150 HTTP/1.0" 202 Accepted 2026-03-09 20:31:08,822 - lava_callback - INFO - Starting async job submit for node 69af2d99ce46d8ec186ad150 2026-03-09 20:31:08,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:31:08,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2d99ce46d8ec186ad150 HTTP/1.1" 200 None 2026-03-09 20:31:08,868 - lava_callback - DEBUG - Processing node 69af2d99ce46d8ec186ad150: baseline-nfs-arm64 2026-03-09 20:31:09,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:31:09,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:31:09,349 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2d99ce46d8ec186ad150/log.txt.gz 2026-03-09 20:31:09,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:31:09,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:31:09,787 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2d99ce46d8ec186ad150/lava-logs.yaml 2026-03-09 20:31:09,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:31:09,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:31:09,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2d99ce46d8ec186ad150/lava_callback.json.gz DEBUG: Setup failed for 69af2d99ce46d8ec186ad150. Transitting job node result: incomplete -> fail 2026-03-09 20:31:10,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:31:10,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2d99ce46d8ec186ad150 HTTP/1.1" 200 None 2026-03-09 20:31:10,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:31:10,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:31:10,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:31:10,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2d99ce46d8ec186ad150 HTTP/1.1" 200 None 2026-03-09 20:31:10,169 - lava_callback - INFO - Completed processing callback for node 69af2d99ce46d8ec186ad150 INFO: 172.21.0.1:51884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51888 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:31:17,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:31:17,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58488 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:32:23,426 - lava_callback - INFO - Received callback for node 69af2de3ce46d8ec186ad155 INFO: 172.21.0.1:59030 - "POST /node/69af2de3ce46d8ec186ad155 HTTP/1.0" 202 Accepted 2026-03-09 20:32:23,433 - lava_callback - INFO - Starting async job submit for node 69af2de3ce46d8ec186ad155 2026-03-09 20:32:23,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:32:23,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2de3ce46d8ec186ad155 HTTP/1.1" 200 None 2026-03-09 20:32:23,493 - lava_callback - DEBUG - Processing node 69af2de3ce46d8ec186ad155: baseline-nfs-arm64 2026-03-09 20:32:24,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:32:24,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:32:24,505 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2de3ce46d8ec186ad155/log.txt.gz 2026-03-09 20:32:24,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:32:25,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:32:25,211 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2de3ce46d8ec186ad155/lava-logs.yaml 2026-03-09 20:32:25,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:32:25,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:32:25,432 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2de3ce46d8ec186ad155/lava_callback.json.gz DEBUG: Setup failed for 69af2de3ce46d8ec186ad155. Transitting job node result: incomplete -> fail 2026-03-09 20:32:25,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:32:25,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2de3ce46d8ec186ad155 HTTP/1.1" 200 None 2026-03-09 20:32:25,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:32:25,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:32:25,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:32:25,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2de3ce46d8ec186ad155 HTTP/1.1" 200 None 2026-03-09 20:32:25,626 - lava_callback - INFO - Completed processing callback for node 69af2de3ce46d8ec186ad155 INFO: 172.21.0.1:59040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52968 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:32:47,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:32:47,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39016 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:34:10,907 - lava_callback - INFO - Received callback for node 69af2e5ace46d8ec186ad176 INFO: 172.21.0.1:56228 - "POST /node/69af2e5ace46d8ec186ad176 HTTP/1.0" 202 Accepted 2026-03-09 20:34:10,914 - lava_callback - INFO - Starting async job submit for node 69af2e5ace46d8ec186ad176 2026-03-09 20:34:10,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:34:10,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2e5ace46d8ec186ad176 HTTP/1.1" 200 None 2026-03-09 20:34:10,975 - lava_callback - DEBUG - Processing node 69af2e5ace46d8ec186ad176: baseline-nfs-arm64 2026-03-09 20:34:11,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:34:11,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:34:11,701 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2e5ace46d8ec186ad176/log.txt.gz 2026-03-09 20:34:12,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:34:12,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:34:12,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2e5ace46d8ec186ad176/lava-logs.yaml 2026-03-09 20:34:12,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:34:12,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:34:12,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2e5ace46d8ec186ad176/lava_callback.json.gz DEBUG: Setup failed for 69af2e5ace46d8ec186ad176. Transitting job node result: incomplete -> fail 2026-03-09 20:34:12,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:34:12,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2e5ace46d8ec186ad176 HTTP/1.1" 200 None 2026-03-09 20:34:12,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:34:12,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:34:12,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:34:12,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2e5ace46d8ec186ad176 HTTP/1.1" 200 None 2026-03-09 20:34:12,676 - lava_callback - INFO - Completed processing callback for node 69af2e5ace46d8ec186ad176 INFO: 172.21.0.1:56234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56248 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:34:17,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:34:17,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45702 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:36:10,742 - lava_callback - INFO - Received callback for node 69af2ecece46d8ec186ad193 INFO: 172.21.0.1:49606 - "POST /node/69af2ecece46d8ec186ad193 HTTP/1.0" 202 Accepted 2026-03-09 20:36:10,748 - lava_callback - INFO - Starting async job submit for node 69af2ecece46d8ec186ad193 2026-03-09 20:36:10,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:36:10,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2ecece46d8ec186ad193 HTTP/1.1" 200 None 2026-03-09 20:36:10,798 - lava_callback - DEBUG - Processing node 69af2ecece46d8ec186ad193: baseline-nfs-arm64 2026-03-09 20:36:11,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:36:11,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:36:11,304 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2ecece46d8ec186ad193/log.txt.gz 2026-03-09 20:36:11,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:36:11,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:36:11,727 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2ecece46d8ec186ad193/lava-logs.yaml 2026-03-09 20:36:11,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:36:11,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:36:11,871 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2ecece46d8ec186ad193/lava_callback.json.gz DEBUG: Setup failed for 69af2ecece46d8ec186ad193. Transitting job node result: incomplete -> fail 2026-03-09 20:36:11,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:36:11,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2ecece46d8ec186ad193 HTTP/1.1" 200 None 2026-03-09 20:36:11,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:36:11,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:36:11,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:36:11,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2ecece46d8ec186ad193 HTTP/1.1" 200 None 2026-03-09 20:36:12,068 - lava_callback - INFO - Completed processing callback for node 69af2ecece46d8ec186ad193 INFO: 172.21.0.1:49608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49620 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:36:17,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:36:17,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58650 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:37:08,751 - lava_callback - INFO - Received callback for node 69af2f1bce46d8ec186ad19a INFO: 172.21.0.1:58654 - "POST /node/69af2f1bce46d8ec186ad19a HTTP/1.0" 202 Accepted 2026-03-09 20:37:08,758 - lava_callback - INFO - Starting async job submit for node 69af2f1bce46d8ec186ad19a 2026-03-09 20:37:08,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:37:08,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2f1bce46d8ec186ad19a HTTP/1.1" 200 None 2026-03-09 20:37:08,808 - lava_callback - DEBUG - Processing node 69af2f1bce46d8ec186ad19a: baseline-nfs-arm64 2026-03-09 20:37:09,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:37:09,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:37:09,388 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2f1bce46d8ec186ad19a/log.txt.gz 2026-03-09 20:37:09,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:37:09,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:37:09,850 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2f1bce46d8ec186ad19a/lava-logs.yaml 2026-03-09 20:37:09,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:37:09,996 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:37:09,998 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2f1bce46d8ec186ad19a/lava_callback.json.gz DEBUG: Setup failed for 69af2f1bce46d8ec186ad19a. Transitting job node result: incomplete -> fail 2026-03-09 20:37:10,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:37:10,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2f1bce46d8ec186ad19a HTTP/1.1" 200 None 2026-03-09 20:37:10,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:37:10,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:37:10,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:37:10,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2f1bce46d8ec186ad19a HTTP/1.1" 200 None 2026-03-09 20:37:10,227 - lava_callback - INFO - Completed processing callback for node 69af2f1bce46d8ec186ad19a INFO: 172.21.0.1:54114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54126 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:37:17,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:37:17,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47492 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:40:00,663 - lava_callback - INFO - Received callback for node 69af2f60ce46d8ec186ad1ab INFO: 172.21.0.1:35340 - "POST /node/69af2f60ce46d8ec186ad1ab HTTP/1.0" 202 Accepted 2026-03-09 20:40:00,669 - lava_callback - INFO - Starting async job submit for node 69af2f60ce46d8ec186ad1ab 2026-03-09 20:40:00,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:00,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2f60ce46d8ec186ad1ab HTTP/1.1" 200 None 2026-03-09 20:40:00,717 - lava_callback - DEBUG - Processing node 69af2f60ce46d8ec186ad1ab: baseline-arm64 2026-03-09 20:40:01,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:01,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:01,320 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2f60ce46d8ec186ad1ab/log.txt.gz 2026-03-09 20:40:01,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:01,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:01,772 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2f60ce46d8ec186ad1ab/lava-logs.yaml 2026-03-09 20:40:01,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:01,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:01,942 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2f60ce46d8ec186ad1ab/lava_callback.json.gz DEBUG: Setup failed for 69af2f60ce46d8ec186ad1ab. Transitting job node result: incomplete -> fail 2026-03-09 20:40:02,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:02,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2f60ce46d8ec186ad1ab HTTP/1.1" 200 None 2026-03-09 20:40:02,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:02,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:40:02,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:02,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2f60ce46d8ec186ad1ab HTTP/1.1" 200 None 2026-03-09 20:40:02,152 - lava_callback - INFO - Completed processing callback for node 69af2f60ce46d8ec186ad1ab INFO: 172.21.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48466 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:40:17,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:17,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45922 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:40:30,782 - lava_callback - INFO - Received callback for node 69af2fdace46d8ec186ad1c6 INFO: 172.21.0.1:59978 - "POST /node/69af2fdace46d8ec186ad1c6 HTTP/1.0" 202 Accepted 2026-03-09 20:40:30,788 - lava_callback - INFO - Starting async job submit for node 69af2fdace46d8ec186ad1c6 2026-03-09 20:40:30,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:30,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2fdace46d8ec186ad1c6 HTTP/1.1" 200 None 2026-03-09 20:40:30,845 - lava_callback - DEBUG - Processing node 69af2fdace46d8ec186ad1c6: baseline-arm64 2026-03-09 20:40:31,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:31,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:31,360 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2fdace46d8ec186ad1c6/log.txt.gz 2026-03-09 20:40:31,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:31,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:31,770 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2fdace46d8ec186ad1c6/lava-logs.yaml 2026-03-09 20:40:31,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:31,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:31,933 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2fdace46d8ec186ad1c6/lava_callback.json.gz DEBUG: Setup failed for 69af2fdace46d8ec186ad1c6. Transitting job node result: incomplete -> fail 2026-03-09 20:40:32,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:32,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2fdace46d8ec186ad1c6 HTTP/1.1" 200 None 2026-03-09 20:40:32,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:32,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:40:32,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:32,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2fdace46d8ec186ad1c6 HTTP/1.1" 200 None 2026-03-09 20:40:32,146 - lava_callback - INFO - Completed processing callback for node 69af2fdace46d8ec186ad1c6 INFO: 172.21.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:40:47,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:47,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39284 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:43:37,198 - lava_callback - INFO - Received callback for node 69ac8de346f7ae8ff336685d INFO: 172.21.0.1:39292 - "POST /node/69ac8de346f7ae8ff336685d HTTP/1.0" 202 Accepted 2026-03-09 20:43:37,204 - lava_callback - INFO - Starting async job submit for node 69ac8de346f7ae8ff336685d 2026-03-09 20:43:37,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:37,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8de346f7ae8ff336685d HTTP/1.1" 200 None 2026-03-09 20:43:37,220 - lava_callback - DEBUG - Processing node 69ac8de346f7ae8ff336685d: baseline-arm-baylibre 2026-03-09 20:43:37,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8de346f7ae8ff336685d 2026-03-09 20:43:37,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:37,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8de346f7ae8ff336685d HTTP/1.1" 200 None 2026-03-09 20:43:37,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:37,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:43:37,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:37,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8de346f7ae8ff336685d HTTP/1.1" 200 None 2026-03-09 20:43:37,293 - lava_callback - ERROR - Error processing callback for node 69ac8de346f7ae8ff336685d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 20:43:45,989 - lava_callback - INFO - Received callback for node 69af302bce46d8ec186ad1d3 INFO: 172.21.0.1:54538 - "POST /node/69af302bce46d8ec186ad1d3 HTTP/1.0" 202 Accepted 2026-03-09 20:43:45,996 - lava_callback - INFO - Starting async job submit for node 69af302bce46d8ec186ad1d3 2026-03-09 20:43:46,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:46,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af302bce46d8ec186ad1d3 HTTP/1.1" 200 None 2026-03-09 20:43:46,048 - lava_callback - DEBUG - Processing node 69af302bce46d8ec186ad1d3: baseline-nfs-arm64 2026-03-09 20:43:46,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:54546 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:43:46,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:43:46,598 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af302bce46d8ec186ad1d3/log.txt.gz 2026-03-09 20:43:46,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:43:47,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:43:47,059 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af302bce46d8ec186ad1d3/lava-logs.yaml 2026-03-09 20:43:47,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:43:47,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:43:47,213 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af302bce46d8ec186ad1d3/lava_callback.json.gz DEBUG: Setup failed for 69af302bce46d8ec186ad1d3. Transitting job node result: incomplete -> fail 2026-03-09 20:43:47,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:47,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af302bce46d8ec186ad1d3 HTTP/1.1" 200 None 2026-03-09 20:43:47,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:47,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:43:47,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:47,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af302bce46d8ec186ad1d3 HTTP/1.1" 200 None 2026-03-09 20:43:47,410 - lava_callback - INFO - Completed processing callback for node 69af302bce46d8ec186ad1d3 INFO: 172.21.0.1:49942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59540 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:44:17,075 - lava_callback - INFO - Received callback for node 69ac8e1c46f7ae8ff3366880 INFO: 172.21.0.1:59548 - "POST /node/69ac8e1c46f7ae8ff3366880 HTTP/1.0" 202 Accepted 2026-03-09 20:44:17,081 - lava_callback - INFO - Starting async job submit for node 69ac8e1c46f7ae8ff3366880 2026-03-09 20:44:17,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:44:17,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8e1c46f7ae8ff3366880 HTTP/1.1" 200 None 2026-03-09 20:44:17,097 - lava_callback - DEBUG - Processing node 69ac8e1c46f7ae8ff3366880: baseline-arm-baylibre 2026-03-09 20:44:17,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8e1c46f7ae8ff3366880 2026-03-09 20:44:17,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:44:17,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8e1c46f7ae8ff3366880 HTTP/1.1" 200 None 2026-03-09 20:44:17,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:44:17,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:44:17,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:44:17,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8e1c46f7ae8ff3366880 HTTP/1.1" 200 None 2026-03-09 20:44:17,161 - lava_callback - ERROR - Error processing callback for node 69ac8e1c46f7ae8ff3366880: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 20:44:17,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:44:17,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38598 - "GET /node/69a75a595a0c0e10c481c6ff%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56782 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:45:57,249 - lava_callback - INFO - Received callback for node 69ac8e7c46f7ae8ff336689d INFO: 172.21.0.1:56794 - "POST /node/69ac8e7c46f7ae8ff336689d HTTP/1.0" 202 Accepted 2026-03-09 20:45:57,255 - lava_callback - INFO - Starting async job submit for node 69ac8e7c46f7ae8ff336689d 2026-03-09 20:45:57,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:45:57,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8e7c46f7ae8ff336689d HTTP/1.1" 200 None 2026-03-09 20:45:57,270 - lava_callback - DEBUG - Processing node 69ac8e7c46f7ae8ff336689d: baseline-arm-baylibre 2026-03-09 20:45:57,271 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8e7c46f7ae8ff336689d 2026-03-09 20:45:57,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:45:57,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8e7c46f7ae8ff336689d HTTP/1.1" 200 None 2026-03-09 20:45:57,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:45:57,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:45:57,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:45:57,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8e7c46f7ae8ff336689d HTTP/1.1" 200 None 2026-03-09 20:45:57,327 - lava_callback - ERROR - Error processing callback for node 69ac8e7c46f7ae8ff336689d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51654 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:46:10,421 - lava_callback - INFO - Received callback for node 69af313cce46d8ec186ad1fd INFO: 172.21.0.1:34022 - "POST /node/69af313cce46d8ec186ad1fd HTTP/1.0" 202 Accepted 2026-03-09 20:46:10,427 - lava_callback - INFO - Starting async job submit for node 69af313cce46d8ec186ad1fd 2026-03-09 20:46:10,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:10,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af313cce46d8ec186ad1fd HTTP/1.1" 200 None 2026-03-09 20:46:10,459 - lava_callback - DEBUG - Processing node 69af313cce46d8ec186ad1fd: baseline-x86 2026-03-09 20:46:10,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:46:10,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:46:10,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af313cce46d8ec186ad1fd/log.txt.gz 2026-03-09 20:46:11,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:46:11,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:46:11,317 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af313cce46d8ec186ad1fd/lava-logs.yaml 2026-03-09 20:46:11,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:46:11,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:46:11,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af313cce46d8ec186ad1fd/lava_callback.json.gz 2026-03-09 20:46:11,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:11,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af313cce46d8ec186ad1fd HTTP/1.1" 200 None 2026-03-09 20:46:11,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:11,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 20:46:11,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:11,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af313cce46d8ec186ad1fd HTTP/1.1" 200 None 2026-03-09 20:46:11,552 - lava_callback - INFO - Completed processing callback for node 69af313cce46d8ec186ad1fd INFO: 172.21.0.1:34024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34026 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:46:17,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:17,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50102 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:46:37,187 - lava_callback - INFO - Received callback for node 69ac8ea146f7ae8ff33668b0 INFO: 172.21.0.1:50110 - "POST /node/69ac8ea146f7ae8ff33668b0 HTTP/1.0" 202 Accepted 2026-03-09 20:46:37,192 - lava_callback - INFO - Starting async job submit for node 69ac8ea146f7ae8ff33668b0 2026-03-09 20:46:37,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:37,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ea146f7ae8ff33668b0 HTTP/1.1" 200 None 2026-03-09 20:46:37,209 - lava_callback - DEBUG - Processing node 69ac8ea146f7ae8ff33668b0: baseline-arm-baylibre 2026-03-09 20:46:37,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8ea146f7ae8ff33668b0 2026-03-09 20:46:37,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:37,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ea146f7ae8ff33668b0 HTTP/1.1" 200 None 2026-03-09 20:46:37,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:37,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:46:37,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:37,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8ea146f7ae8ff33668b0 HTTP/1.1" 200 None 2026-03-09 20:46:37,265 - lava_callback - ERROR - Error processing callback for node 69ac8ea146f7ae8ff33668b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59010 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:47:17,295 - lava_callback - INFO - Received callback for node 69ac8ec446f7ae8ff33668c8 INFO: 172.21.0.1:59012 - "POST /node/69ac8ec446f7ae8ff33668c8 HTTP/1.0" 202 Accepted 2026-03-09 20:47:17,300 - lava_callback - INFO - Starting async job submit for node 69ac8ec446f7ae8ff33668c8 2026-03-09 20:47:17,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:17,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ec446f7ae8ff33668c8 HTTP/1.1" 200 None 2026-03-09 20:47:17,316 - lava_callback - DEBUG - Processing node 69ac8ec446f7ae8ff33668c8: baseline-arm-baylibre 2026-03-09 20:47:17,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8ec446f7ae8ff33668c8 2026-03-09 20:47:17,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:17,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ec446f7ae8ff33668c8 HTTP/1.1" 200 None 2026-03-09 20:47:17,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:17,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:47:17,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:17,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8ec446f7ae8ff33668c8 HTTP/1.1" 200 None 2026-03-09 20:47:17,368 - lava_callback - ERROR - Error processing callback for node 69ac8ec446f7ae8ff33668c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52720 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:47:57,275 - lava_callback - INFO - Received callback for node 69ac8ee846f7ae8ff33668d8 INFO: 172.21.0.1:52726 - "POST /node/69ac8ee846f7ae8ff33668d8 HTTP/1.0" 202 Accepted 2026-03-09 20:47:57,280 - lava_callback - INFO - Starting async job submit for node 69ac8ee846f7ae8ff33668d8 2026-03-09 20:47:57,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:57,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ee846f7ae8ff33668d8 HTTP/1.1" 200 None 2026-03-09 20:47:57,306 - lava_callback - DEBUG - Processing node 69ac8ee846f7ae8ff33668d8: baseline-arm-baylibre 2026-03-09 20:47:57,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8ee846f7ae8ff33668d8 2026-03-09 20:47:57,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:57,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ee846f7ae8ff33668d8 HTTP/1.1" 200 None 2026-03-09 20:47:57,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:57,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:47:57,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:57,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8ee846f7ae8ff33668d8 HTTP/1.1" 200 None 2026-03-09 20:47:57,373 - lava_callback - ERROR - Error processing callback for node 69ac8ee846f7ae8ff33668d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57924 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:48:17,284 - lava_callback - INFO - Received callback for node 69ac8f0c46f7ae8ff33668e7 INFO: 172.21.0.1:57926 - "POST /node/69ac8f0c46f7ae8ff33668e7 HTTP/1.0" 202 Accepted 2026-03-09 20:48:17,289 - lava_callback - INFO - Starting async job submit for node 69ac8f0c46f7ae8ff33668e7 2026-03-09 20:48:17,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:48:17,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f0c46f7ae8ff33668e7 HTTP/1.1" 200 None 2026-03-09 20:48:17,305 - lava_callback - DEBUG - Processing node 69ac8f0c46f7ae8ff33668e7: baseline-arm-baylibre 2026-03-09 20:48:17,305 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8f0c46f7ae8ff33668e7 2026-03-09 20:48:17,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:48:17,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f0c46f7ae8ff33668e7 HTTP/1.1" 200 None 2026-03-09 20:48:17,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:48:17,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:48:17,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:48:17,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8f0c46f7ae8ff33668e7 HTTP/1.1" 200 None 2026-03-09 20:48:17,363 - lava_callback - ERROR - Error processing callback for node 69ac8f0c46f7ae8ff33668e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45672 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:49:37,399 - lava_callback - INFO - Received callback for node 69ac8f4d46f7ae8ff3366906 INFO: 172.21.0.1:45682 - "POST /node/69ac8f4d46f7ae8ff3366906 HTTP/1.0" 202 Accepted 2026-03-09 20:49:37,405 - lava_callback - INFO - Starting async job submit for node 69ac8f4d46f7ae8ff3366906 2026-03-09 20:49:37,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:37,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f4d46f7ae8ff3366906 HTTP/1.1" 200 None 2026-03-09 20:49:37,430 - lava_callback - DEBUG - Processing node 69ac8f4d46f7ae8ff3366906: baseline-arm-baylibre 2026-03-09 20:49:37,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8f4d46f7ae8ff3366906 2026-03-09 20:49:37,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:37,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f4d46f7ae8ff3366906 HTTP/1.1" 200 None 2026-03-09 20:49:37,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:37,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:49:37,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:37,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8f4d46f7ae8ff3366906 HTTP/1.1" 200 None 2026-03-09 20:49:37,492 - lava_callback - ERROR - Error processing callback for node 69ac8f4d46f7ae8ff3366906: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51766 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:49:57,401 - lava_callback - INFO - Received callback for node 69ac8f7146f7ae8ff3366917 INFO: 172.21.0.1:51768 - "POST /node/69ac8f7146f7ae8ff3366917 HTTP/1.0" 202 Accepted 2026-03-09 20:49:57,407 - lava_callback - INFO - Starting async job submit for node 69ac8f7146f7ae8ff3366917 2026-03-09 20:49:57,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:57,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f7146f7ae8ff3366917 HTTP/1.1" 200 None 2026-03-09 20:49:57,424 - lava_callback - DEBUG - Processing node 69ac8f7146f7ae8ff3366917: baseline-arm-baylibre 2026-03-09 20:49:57,424 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8f7146f7ae8ff3366917 2026-03-09 20:49:57,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:57,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f7146f7ae8ff3366917 HTTP/1.1" 200 None 2026-03-09 20:49:57,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:57,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:49:57,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:57,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8f7146f7ae8ff3366917 HTTP/1.1" 200 None 2026-03-09 20:49:57,480 - lava_callback - ERROR - Error processing callback for node 69ac8f7146f7ae8ff3366917: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:50:37,412 - lava_callback - INFO - Received callback for node 69ac8f9646f7ae8ff336692e INFO: 172.21.0.1:47926 - "POST /node/69ac8f9646f7ae8ff336692e HTTP/1.0" 202 Accepted 2026-03-09 20:50:37,418 - lava_callback - INFO - Starting async job submit for node 69ac8f9646f7ae8ff336692e 2026-03-09 20:50:37,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:50:37,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f9646f7ae8ff336692e HTTP/1.1" 200 None 2026-03-09 20:50:37,464 - lava_callback - DEBUG - Processing node 69ac8f9646f7ae8ff336692e: baseline-arm-baylibre 2026-03-09 20:50:37,464 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8f9646f7ae8ff336692e 2026-03-09 20:50:37,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:50:37,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f9646f7ae8ff336692e HTTP/1.1" 200 None 2026-03-09 20:50:37,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:50:37,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:50:37,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:50:37,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8f9646f7ae8ff336692e HTTP/1.1" 200 None 2026-03-09 20:50:37,516 - lava_callback - ERROR - Error processing callback for node 69ac8f9646f7ae8ff336692e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34804 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:51:17,479 - lava_callback - INFO - Received callback for node 69ac8fb946f7ae8ff3366940 INFO: 172.21.0.1:34816 - "POST /node/69ac8fb946f7ae8ff3366940 HTTP/1.0" 202 Accepted 2026-03-09 20:51:17,485 - lava_callback - INFO - Starting async job submit for node 69ac8fb946f7ae8ff3366940 2026-03-09 20:51:17,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:51:17,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8fb946f7ae8ff3366940 HTTP/1.1" 200 None 2026-03-09 20:51:17,501 - lava_callback - DEBUG - Processing node 69ac8fb946f7ae8ff3366940: baseline-arm-baylibre 2026-03-09 20:51:17,501 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8fb946f7ae8ff3366940 2026-03-09 20:51:17,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:51:17,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8fb946f7ae8ff3366940 HTTP/1.1" 200 None 2026-03-09 20:51:17,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:51:17,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:51:17,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:51:17,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8fb946f7ae8ff3366940 HTTP/1.1" 200 None 2026-03-09 20:51:17,560 - lava_callback - ERROR - Error processing callback for node 69ac8fb946f7ae8ff3366940: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46748 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:52:57,558 - lava_callback - INFO - Received callback for node 69ac902146f7ae8ff3366970 INFO: 172.21.0.1:46760 - "POST /node/69ac902146f7ae8ff3366970 HTTP/1.0" 202 Accepted 2026-03-09 20:52:57,563 - lava_callback - INFO - Starting async job submit for node 69ac902146f7ae8ff3366970 2026-03-09 20:52:57,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:52:57,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac902146f7ae8ff3366970 HTTP/1.1" 200 None 2026-03-09 20:52:57,579 - lava_callback - DEBUG - Processing node 69ac902146f7ae8ff3366970: baseline-arm-baylibre 2026-03-09 20:52:57,579 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac902146f7ae8ff3366970 2026-03-09 20:52:57,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:52:57,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac902146f7ae8ff3366970 HTTP/1.1" 200 None 2026-03-09 20:52:57,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:52:57,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:52:57,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:52:57,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac902146f7ae8ff3366970 HTTP/1.1" 200 None 2026-03-09 20:52:57,643 - lava_callback - ERROR - Error processing callback for node 69ac902146f7ae8ff3366970: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48560 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:53:37,553 - lava_callback - INFO - Received callback for node 69ac904246f7ae8ff3366981 INFO: 172.21.0.1:48562 - "POST /node/69ac904246f7ae8ff3366981 HTTP/1.0" 202 Accepted 2026-03-09 20:53:37,559 - lava_callback - INFO - Starting async job submit for node 69ac904246f7ae8ff3366981 2026-03-09 20:53:37,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:53:37,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac904246f7ae8ff3366981 HTTP/1.1" 200 None 2026-03-09 20:53:37,574 - lava_callback - DEBUG - Processing node 69ac904246f7ae8ff3366981: baseline-arm-baylibre 2026-03-09 20:53:37,574 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac904246f7ae8ff3366981 2026-03-09 20:53:37,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:53:37,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac904246f7ae8ff3366981 HTTP/1.1" 200 None 2026-03-09 20:53:37,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:53:37,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:53:37,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:53:37,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac904246f7ae8ff3366981 HTTP/1.1" 200 None 2026-03-09 20:53:37,657 - lava_callback - ERROR - Error processing callback for node 69ac904246f7ae8ff3366981: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39286 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:54:17,626 - lava_callback - INFO - Received callback for node 69ac906646f7ae8ff3366996 INFO: 172.21.0.1:39302 - "POST /node/69ac906646f7ae8ff3366996 HTTP/1.0" 202 Accepted 2026-03-09 20:54:17,632 - lava_callback - INFO - Starting async job submit for node 69ac906646f7ae8ff3366996 2026-03-09 20:54:17,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:17,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac906646f7ae8ff3366996 HTTP/1.1" 200 None 2026-03-09 20:54:17,650 - lava_callback - DEBUG - Processing node 69ac906646f7ae8ff3366996: baseline-arm-baylibre 2026-03-09 20:54:17,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac906646f7ae8ff3366996 2026-03-09 20:54:17,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:17,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac906646f7ae8ff3366996 HTTP/1.1" 200 None 2026-03-09 20:54:17,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:17,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:54:17,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:17,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac906646f7ae8ff3366996 HTTP/1.1" 200 None 2026-03-09 20:54:17,773 - lava_callback - ERROR - Error processing callback for node 69ac906646f7ae8ff3366996: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44934 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:54:37,664 - lava_callback - INFO - Received callback for node 69ac908a46f7ae8ff33669ab INFO: 172.21.0.1:44938 - "POST /node/69ac908a46f7ae8ff33669ab HTTP/1.0" 202 Accepted 2026-03-09 20:54:37,670 - lava_callback - INFO - Starting async job submit for node 69ac908a46f7ae8ff33669ab 2026-03-09 20:54:37,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:37,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac908a46f7ae8ff33669ab HTTP/1.1" 200 None 2026-03-09 20:54:37,686 - lava_callback - DEBUG - Processing node 69ac908a46f7ae8ff33669ab: baseline-arm-baylibre 2026-03-09 20:54:37,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac908a46f7ae8ff33669ab 2026-03-09 20:54:37,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:37,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac908a46f7ae8ff33669ab HTTP/1.1" 200 None 2026-03-09 20:54:37,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:37,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:54:37,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:37,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac908a46f7ae8ff33669ab HTTP/1.1" 200 None 2026-03-09 20:54:37,749 - lava_callback - ERROR - Error processing callback for node 69ac908a46f7ae8ff33669ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42170 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:55:17,694 - lava_callback - INFO - Received callback for node 69ac90ad46f7ae8ff33669b9 INFO: 172.21.0.1:42180 - "POST /node/69ac90ad46f7ae8ff33669b9 HTTP/1.0" 202 Accepted 2026-03-09 20:55:17,700 - lava_callback - INFO - Starting async job submit for node 69ac90ad46f7ae8ff33669b9 2026-03-09 20:55:17,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:17,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac90ad46f7ae8ff33669b9 HTTP/1.1" 200 None 2026-03-09 20:55:17,723 - lava_callback - DEBUG - Processing node 69ac90ad46f7ae8ff33669b9: baseline-arm-baylibre 2026-03-09 20:55:17,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac90ad46f7ae8ff33669b9 2026-03-09 20:55:17,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:17,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac90ad46f7ae8ff33669b9 HTTP/1.1" 200 None 2026-03-09 20:55:17,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:17,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:55:17,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:17,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac90ad46f7ae8ff33669b9 HTTP/1.1" 200 None 2026-03-09 20:55:17,784 - lava_callback - ERROR - Error processing callback for node 69ac90ad46f7ae8ff33669b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:55:57,720 - lava_callback - INFO - Received callback for node 69ac90d246f7ae8ff33669d7 INFO: 172.21.0.1:48904 - "POST /node/69ac90d246f7ae8ff33669d7 HTTP/1.0" 202 Accepted 2026-03-09 20:55:57,726 - lava_callback - INFO - Starting async job submit for node 69ac90d246f7ae8ff33669d7 2026-03-09 20:55:57,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:57,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac90d246f7ae8ff33669d7 HTTP/1.1" 200 None 2026-03-09 20:55:57,751 - lava_callback - DEBUG - Processing node 69ac90d246f7ae8ff33669d7: baseline-arm-baylibre 2026-03-09 20:55:57,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac90d246f7ae8ff33669d7 2026-03-09 20:55:57,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:57,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac90d246f7ae8ff33669d7 HTTP/1.1" 200 None 2026-03-09 20:55:57,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:57,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:55:57,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:57,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac90d246f7ae8ff33669d7 HTTP/1.1" 200 None 2026-03-09 20:55:57,813 - lava_callback - ERROR - Error processing callback for node 69ac90d246f7ae8ff33669d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57944 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:57:50,406 - lava_callback - INFO - Received callback for node 69af2fb9ce46d8ec186ad1bf INFO: 172.21.0.1:45632 - "POST /node/69af2fb9ce46d8ec186ad1bf HTTP/1.0" 202 Accepted 2026-03-09 20:57:50,453 - lava_callback - INFO - Starting async job submit for node 69af2fb9ce46d8ec186ad1bf 2026-03-09 20:57:50,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:57:50,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2fb9ce46d8ec186ad1bf HTTP/1.1" 200 None 2026-03-09 20:57:50,492 - lava_callback - DEBUG - Processing node 69af2fb9ce46d8ec186ad1bf: baseline-arm-pengutronix 2026-03-09 20:57:50,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:57:50,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:57:50,747 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af2fb9ce46d8ec186ad1bf/log.txt.gz 2026-03-09 20:57:50,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:57:50,974 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:57:50,976 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af2fb9ce46d8ec186ad1bf/lava-logs.yaml 2026-03-09 20:57:50,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:57:51,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:57:51,121 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af2fb9ce46d8ec186ad1bf/lava_callback.json.gz DEBUG: Setup failed for 69af2fb9ce46d8ec186ad1bf. Transitting job node result: incomplete -> fail 2026-03-09 20:57:51,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:57:51,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2fb9ce46d8ec186ad1bf HTTP/1.1" 200 None 2026-03-09 20:57:51,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:57:51,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 20:57:51,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:57:51,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2fb9ce46d8ec186ad1bf HTTP/1.1" 200 None 2026-03-09 20:57:51,285 - lava_callback - INFO - Completed processing callback for node 69af2fb9ce46d8ec186ad1bf INFO: 172.21.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55298 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:58:17,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:17,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45786 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:58:28,813 - lava_callback - INFO - Received callback for node 69ac915f46f7ae8ff3366a29 INFO: 172.21.0.1:45792 - "POST /node/69ac915f46f7ae8ff3366a29 HTTP/1.0" 202 Accepted 2026-03-09 20:58:28,819 - lava_callback - INFO - Starting async job submit for node 69ac915f46f7ae8ff3366a29 2026-03-09 20:58:28,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:28,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac915f46f7ae8ff3366a29 HTTP/1.1" 200 None 2026-03-09 20:58:28,834 - lava_callback - DEBUG - Processing node 69ac915f46f7ae8ff3366a29: baseline-arm-clabbe 2026-03-09 20:58:28,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac915f46f7ae8ff3366a29 2026-03-09 20:58:28,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:28,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac915f46f7ae8ff3366a29 HTTP/1.1" 200 None 2026-03-09 20:58:28,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:28,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:58:28,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:28,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac915f46f7ae8ff3366a29 HTTP/1.1" 200 None 2026-03-09 20:58:28,912 - lava_callback - ERROR - Error processing callback for node 69ac915f46f7ae8ff3366a29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33666 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:58:48,592 - lava_callback - INFO - Received callback for node 69ac918446f7ae8ff3366a46 INFO: 172.21.0.1:33672 - "POST /node/69ac918446f7ae8ff3366a46 HTTP/1.0" 202 Accepted 2026-03-09 20:58:48,598 - lava_callback - INFO - Starting async job submit for node 69ac918446f7ae8ff3366a46 2026-03-09 20:58:48,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:48,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac918446f7ae8ff3366a46 HTTP/1.1" 200 None 2026-03-09 20:58:48,613 - lava_callback - DEBUG - Processing node 69ac918446f7ae8ff3366a46: baseline-arm-clabbe 2026-03-09 20:58:48,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac918446f7ae8ff3366a46 2026-03-09 20:58:48,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:48,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac918446f7ae8ff3366a46 HTTP/1.1" 200 None 2026-03-09 20:58:48,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:48,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:58:48,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:48,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac918446f7ae8ff3366a46 HTTP/1.1" 200 None 2026-03-09 20:58:48,670 - lava_callback - ERROR - Error processing callback for node 69ac918446f7ae8ff3366a46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45504 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:59:28,651 - lava_callback - INFO - Received callback for node 69ac91a846f7ae8ff3366a5e INFO: 172.21.0.1:45514 - "POST /node/69ac91a846f7ae8ff3366a5e HTTP/1.0" 202 Accepted 2026-03-09 20:59:28,657 - lava_callback - INFO - Starting async job submit for node 69ac91a846f7ae8ff3366a5e 2026-03-09 20:59:28,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:28,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac91a846f7ae8ff3366a5e HTTP/1.1" 200 None 2026-03-09 20:59:28,672 - lava_callback - DEBUG - Processing node 69ac91a846f7ae8ff3366a5e: baseline-arm-clabbe 2026-03-09 20:59:28,673 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac91a846f7ae8ff3366a5e 2026-03-09 20:59:28,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:28,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac91a846f7ae8ff3366a5e HTTP/1.1" 200 None 2026-03-09 20:59:28,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:28,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:59:28,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:28,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac91a846f7ae8ff3366a5e HTTP/1.1" 200 None 2026-03-09 20:59:28,740 - lava_callback - ERROR - Error processing callback for node 69ac91a846f7ae8ff3366a5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49366 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:59:58,698 - lava_callback - INFO - Received callback for node 69ac91c746f7ae8ff3366a72 INFO: 172.21.0.1:49370 - "POST /node/69ac91c746f7ae8ff3366a72 HTTP/1.0" 202 Accepted 2026-03-09 20:59:58,704 - lava_callback - INFO - Starting async job submit for node 69ac91c746f7ae8ff3366a72 2026-03-09 20:59:58,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:58,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac91c746f7ae8ff3366a72 HTTP/1.1" 200 None 2026-03-09 20:59:58,720 - lava_callback - DEBUG - Processing node 69ac91c746f7ae8ff3366a72: baseline-arm-baylibre 2026-03-09 20:59:58,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac91c746f7ae8ff3366a72 2026-03-09 20:59:58,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:58,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac91c746f7ae8ff3366a72 HTTP/1.1" 200 None 2026-03-09 20:59:58,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:58,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 20:59:58,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:58,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac91c746f7ae8ff3366a72 HTTP/1.1" 200 None 2026-03-09 20:59:58,786 - lava_callback - ERROR - Error processing callback for node 69ac91c746f7ae8ff3366a72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51416 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:00:11,349 - lava_callback - INFO - Received callback for node 69af31ecce46d8ec186ad220 INFO: 172.21.0.1:43222 - "POST /node/69af31ecce46d8ec186ad220 HTTP/1.0" 202 Accepted 2026-03-09 21:00:11,355 - lava_callback - INFO - Starting async job submit for node 69af31ecce46d8ec186ad220 2026-03-09 21:00:11,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:00:11,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af31ecce46d8ec186ad220 HTTP/1.1" 200 None 2026-03-09 21:00:11,404 - lava_callback - DEBUG - Processing node 69af31ecce46d8ec186ad220: baseline-arm64-broonie 2026-03-09 21:00:11,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:00:11,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:00:11,789 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af31ecce46d8ec186ad220/log.txt.gz 2026-03-09 21:00:11,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:00:12,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:00:12,171 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af31ecce46d8ec186ad220/lava-logs.yaml 2026-03-09 21:00:12,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:00:12,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:00:12,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af31ecce46d8ec186ad220/lava_callback.json.gz DEBUG: Setup failed for 69af31ecce46d8ec186ad220. Transitting job node result: incomplete -> fail 2026-03-09 21:00:12,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:00:12,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af31ecce46d8ec186ad220 HTTP/1.1" 200 None 2026-03-09 21:00:12,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:00:12,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:00:12,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:00:12,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af31ecce46d8ec186ad220 HTTP/1.1" 200 None 2026-03-09 21:00:12,577 - lava_callback - INFO - Completed processing callback for node 69af31ecce46d8ec186ad220 INFO: 172.21.0.1:43224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43232 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:00:17,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:00:17,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55180 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:05:08,821 - lava_callback - INFO - Received callback for node 69ac930146f7ae8ff3366b2c INFO: 172.21.0.1:55190 - "POST /node/69ac930146f7ae8ff3366b2c HTTP/1.0" 202 Accepted 2026-03-09 21:05:08,826 - lava_callback - INFO - Starting async job submit for node 69ac930146f7ae8ff3366b2c 2026-03-09 21:05:08,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:05:08,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac930146f7ae8ff3366b2c HTTP/1.1" 200 None 2026-03-09 21:05:08,842 - lava_callback - DEBUG - Processing node 69ac930146f7ae8ff3366b2c: baseline-arm-clabbe 2026-03-09 21:05:08,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac930146f7ae8ff3366b2c 2026-03-09 21:05:08,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:05:08,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac930146f7ae8ff3366b2c HTTP/1.1" 200 None 2026-03-09 21:05:08,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:05:08,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:05:08,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:05:08,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac930146f7ae8ff3366b2c HTTP/1.1" 200 None 2026-03-09 21:05:08,898 - lava_callback - ERROR - Error processing callback for node 69ac930146f7ae8ff3366b2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60792 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:07:08,983 - lava_callback - INFO - Received callback for node 69ac936b46f7ae8ff3366b6d INFO: 172.21.0.1:60802 - "POST /node/69ac936b46f7ae8ff3366b6d HTTP/1.0" 202 Accepted 2026-03-09 21:07:08,988 - lava_callback - INFO - Starting async job submit for node 69ac936b46f7ae8ff3366b6d 2026-03-09 21:07:08,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:09,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac936b46f7ae8ff3366b6d HTTP/1.1" 200 None 2026-03-09 21:07:09,008 - lava_callback - DEBUG - Processing node 69ac936b46f7ae8ff3366b6d: baseline-arm-clabbe 2026-03-09 21:07:09,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac936b46f7ae8ff3366b6d 2026-03-09 21:07:09,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:09,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac936b46f7ae8ff3366b6d HTTP/1.1" 200 None 2026-03-09 21:07:09,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:09,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:07:09,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:09,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac936b46f7ae8ff3366b6d HTTP/1.1" 200 None 2026-03-09 21:07:09,067 - lava_callback - ERROR - Error processing callback for node 69ac936b46f7ae8ff3366b6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42158 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:07:48,991 - lava_callback - INFO - Received callback for node 69ac938f46f7ae8ff3366b8c INFO: 172.21.0.1:42172 - "POST /node/69ac938f46f7ae8ff3366b8c HTTP/1.0" 202 Accepted 2026-03-09 21:07:48,996 - lava_callback - INFO - Starting async job submit for node 69ac938f46f7ae8ff3366b8c 2026-03-09 21:07:48,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:49,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac938f46f7ae8ff3366b8c HTTP/1.1" 200 None 2026-03-09 21:07:49,012 - lava_callback - DEBUG - Processing node 69ac938f46f7ae8ff3366b8c: baseline-arm-clabbe 2026-03-09 21:07:49,012 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac938f46f7ae8ff3366b8c 2026-03-09 21:07:49,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:49,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac938f46f7ae8ff3366b8c HTTP/1.1" 200 None 2026-03-09 21:07:49,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:49,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:07:49,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:49,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac938f46f7ae8ff3366b8c HTTP/1.1" 200 None 2026-03-09 21:07:49,065 - lava_callback - ERROR - Error processing callback for node 69ac938f46f7ae8ff3366b8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60518 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:08:09,040 - lava_callback - INFO - Received callback for node 69ac93b346f7ae8ff3366b97 INFO: 172.21.0.1:60526 - "POST /node/69ac93b346f7ae8ff3366b97 HTTP/1.0" 202 Accepted 2026-03-09 21:08:09,045 - lava_callback - INFO - Starting async job submit for node 69ac93b346f7ae8ff3366b97 2026-03-09 21:08:09,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:08:09,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac93b346f7ae8ff3366b97 HTTP/1.1" 200 None 2026-03-09 21:08:09,062 - lava_callback - DEBUG - Processing node 69ac93b346f7ae8ff3366b97: baseline-arm-clabbe 2026-03-09 21:08:09,062 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac93b346f7ae8ff3366b97 2026-03-09 21:08:09,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:08:09,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac93b346f7ae8ff3366b97 HTTP/1.1" 200 None 2026-03-09 21:08:09,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:08:09,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:08:09,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:08:09,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac93b346f7ae8ff3366b97 HTTP/1.1" 200 None 2026-03-09 21:08:09,140 - lava_callback - ERROR - Error processing callback for node 69ac93b346f7ae8ff3366b97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42108 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:09:29,092 - lava_callback - INFO - Received callback for node 69ac93fa46f7ae8ff3366bc0 INFO: 172.21.0.1:42116 - "POST /node/69ac93fa46f7ae8ff3366bc0 HTTP/1.0" 202 Accepted 2026-03-09 21:09:29,097 - lava_callback - INFO - Starting async job submit for node 69ac93fa46f7ae8ff3366bc0 2026-03-09 21:09:29,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:09:29,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac93fa46f7ae8ff3366bc0 HTTP/1.1" 200 None 2026-03-09 21:09:29,123 - lava_callback - DEBUG - Processing node 69ac93fa46f7ae8ff3366bc0: baseline-arm-clabbe 2026-03-09 21:09:29,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac93fa46f7ae8ff3366bc0 2026-03-09 21:09:29,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:09:29,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac93fa46f7ae8ff3366bc0 HTTP/1.1" 200 None 2026-03-09 21:09:29,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:09:29,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:09:29,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:09:29,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac93fa46f7ae8ff3366bc0 HTTP/1.1" 200 None 2026-03-09 21:09:29,180 - lava_callback - ERROR - Error processing callback for node 69ac93fa46f7ae8ff3366bc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36914 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:10:09,030 - lava_callback - INFO - Received callback for node 69ac941e46f7ae8ff3366bdd INFO: 172.21.0.1:36920 - "POST /node/69ac941e46f7ae8ff3366bdd HTTP/1.0" 202 Accepted 2026-03-09 21:10:09,036 - lava_callback - INFO - Starting async job submit for node 69ac941e46f7ae8ff3366bdd 2026-03-09 21:10:09,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:10:09,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac941e46f7ae8ff3366bdd HTTP/1.1" 200 None 2026-03-09 21:10:09,061 - lava_callback - DEBUG - Processing node 69ac941e46f7ae8ff3366bdd: baseline-arm-clabbe 2026-03-09 21:10:09,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac941e46f7ae8ff3366bdd 2026-03-09 21:10:09,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:10:09,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac941e46f7ae8ff3366bdd HTTP/1.1" 200 None 2026-03-09 21:10:09,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:10:09,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:10:09,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:10:09,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac941e46f7ae8ff3366bdd HTTP/1.1" 200 None 2026-03-09 21:10:09,131 - lava_callback - ERROR - Error processing callback for node 69ac941e46f7ae8ff3366bdd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:14:18,844 - lava_callback - INFO - Received callback for node 69ac952046f7ae8ff3366c40 INFO: 172.21.0.1:34502 - "POST /node/69ac952046f7ae8ff3366c40 HTTP/1.0" 202 Accepted 2026-03-09 21:14:18,849 - lava_callback - INFO - Starting async job submit for node 69ac952046f7ae8ff3366c40 2026-03-09 21:14:18,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:14:18,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac952046f7ae8ff3366c40 HTTP/1.1" 200 None 2026-03-09 21:14:18,865 - lava_callback - DEBUG - Processing node 69ac952046f7ae8ff3366c40: baseline-arm64-baylibre 2026-03-09 21:14:18,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac952046f7ae8ff3366c40 2026-03-09 21:14:18,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:14:18,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac952046f7ae8ff3366c40 HTTP/1.1" 200 None 2026-03-09 21:14:18,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:14:18,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:14:18,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:14:18,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac952046f7ae8ff3366c40 HTTP/1.1" 200 None 2026-03-09 21:14:18,920 - lava_callback - ERROR - Error processing callback for node 69ac952046f7ae8ff3366c40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32800 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:15:12,145 - lava_callback - INFO - Received callback for node 69af3678ce46d8ec186ad2d5 INFO: 172.21.0.1:41956 - "POST /node/69af3678ce46d8ec186ad2d5 HTTP/1.0" 202 Accepted 2026-03-09 21:15:12,151 - lava_callback - INFO - Starting async job submit for node 69af3678ce46d8ec186ad2d5 2026-03-09 21:15:12,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:12,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3678ce46d8ec186ad2d5 HTTP/1.1" 200 None 2026-03-09 21:15:12,210 - lava_callback - DEBUG - Processing node 69af3678ce46d8ec186ad2d5: baseline-arm64-broonie 2026-03-09 21:15:12,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:15:12,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:15:12,719 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3678ce46d8ec186ad2d5/log.txt.gz 2026-03-09 21:15:12,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:15:13,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:15:13,158 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3678ce46d8ec186ad2d5/lava-logs.yaml 2026-03-09 21:15:13,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:15:13,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:15:13,350 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3678ce46d8ec186ad2d5/lava_callback.json.gz DEBUG: Setup failed for 69af3678ce46d8ec186ad2d5. Transitting job node result: incomplete -> fail 2026-03-09 21:15:13,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:13,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3678ce46d8ec186ad2d5 HTTP/1.1" 200 None 2026-03-09 21:15:13,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:13,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:15:13,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:13,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3678ce46d8ec186ad2d5 HTTP/1.1" 200 None 2026-03-09 21:15:13,573 - lava_callback - INFO - Completed processing callback for node 69af3678ce46d8ec186ad2d5 INFO: 172.21.0.1:41964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41974 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:15:17,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:17,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56504 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:15:38,814 - lava_callback - INFO - Received callback for node 69ac956a46f7ae8ff3366c5b INFO: 172.21.0.1:56518 - "POST /node/69ac956a46f7ae8ff3366c5b HTTP/1.0" 202 Accepted 2026-03-09 21:15:38,820 - lava_callback - INFO - Starting async job submit for node 69ac956a46f7ae8ff3366c5b 2026-03-09 21:15:38,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:38,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac956a46f7ae8ff3366c5b HTTP/1.1" 200 None 2026-03-09 21:15:38,836 - lava_callback - DEBUG - Processing node 69ac956a46f7ae8ff3366c5b: baseline-arm64-baylibre 2026-03-09 21:15:38,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac956a46f7ae8ff3366c5b 2026-03-09 21:15:38,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:38,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac956a46f7ae8ff3366c5b HTTP/1.1" 200 None 2026-03-09 21:15:38,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:38,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:15:38,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:38,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac956a46f7ae8ff3366c5b HTTP/1.1" 200 None 2026-03-09 21:15:38,898 - lava_callback - ERROR - Error processing callback for node 69ac956a46f7ae8ff3366c5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:16:58,671 - lava_callback - INFO - Received callback for node 69af30c6ce46d8ec186ad1ee INFO: 172.21.0.1:56098 - "POST /node/69af30c6ce46d8ec186ad1ee HTTP/1.0" 202 Accepted 2026-03-09 21:16:58,677 - lava_callback - INFO - Starting async job submit for node 69af30c6ce46d8ec186ad1ee 2026-03-09 21:16:58,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:16:58,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af30c6ce46d8ec186ad1ee HTTP/1.1" 200 None 2026-03-09 21:16:58,728 - lava_callback - DEBUG - Processing node 69af30c6ce46d8ec186ad1ee: baseline-arm64-broonie 2026-03-09 21:16:59,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:16:59,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:16:59,367 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af30c6ce46d8ec186ad1ee/log.txt.gz 2026-03-09 21:16:59,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:16:59,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:16:59,846 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af30c6ce46d8ec186ad1ee/lava-logs.yaml 2026-03-09 21:16:59,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:38810 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:17:00,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:17:00,036 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af30c6ce46d8ec186ad1ee/lava_callback.json.gz DEBUG: Setup failed for 69af30c6ce46d8ec186ad1ee. Transitting job node result: incomplete -> fail 2026-03-09 21:17:00,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:00,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af30c6ce46d8ec186ad1ee HTTP/1.1" 200 None 2026-03-09 21:17:00,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:00,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-09 21:17:00,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:00,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af30c6ce46d8ec186ad1ee HTTP/1.1" 200 None 2026-03-09 21:17:00,234 - lava_callback - INFO - Completed processing callback for node 69af30c6ce46d8ec186ad1ee INFO: 172.21.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33652 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:17:17,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:17,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46620 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:17:38,487 - lava_callback - INFO - Received callback for node 69ae8934c660ead8a389aee2 INFO: 172.21.0.1:46622 - "POST /node/69ae8934c660ead8a389aee2 HTTP/1.0" 202 Accepted 2026-03-09 21:17:38,494 - lava_callback - INFO - Starting async job submit for node 69ae8934c660ead8a389aee2 2026-03-09 21:17:38,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:38,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8934c660ead8a389aee2 HTTP/1.1" 200 None 2026-03-09 21:17:38,533 - lava_callback - DEBUG - Processing node 69ae8934c660ead8a389aee2: baseline-arm-broonie 2026-03-09 21:17:38,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:17:38,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:17:38,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae8934c660ead8a389aee2/log.txt.gz 2026-03-09 21:17:39,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:17:39,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:17:39,229 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae8934c660ead8a389aee2/lava-logs.yaml 2026-03-09 21:17:39,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:17:39,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:17:39,402 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae8934c660ead8a389aee2/lava_callback.json.gz 2026-03-09 21:17:39,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:39,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8934c660ead8a389aee2 HTTP/1.1" 200 None 2026-03-09 21:17:39,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:39,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:17:39,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:39,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae8934c660ead8a389aee2 HTTP/1.1" 200 None 2026-03-09 21:17:39,486 - lava_callback - INFO - Completed processing callback for node 69ae8934c660ead8a389aee2 INFO: 172.21.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40712 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:40714 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:17:47,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:47,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36430 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:36444 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36460 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:36466 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:36468 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:17:58,908 - lava_callback - INFO - Received callback for node 69ac95fb46f7ae8ff3366c99 INFO: 172.21.0.1:36476 - "POST /node/69ac95fb46f7ae8ff3366c99 HTTP/1.0" 202 Accepted 2026-03-09 21:17:58,914 - lava_callback - INFO - Starting async job submit for node 69ac95fb46f7ae8ff3366c99 2026-03-09 21:17:58,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:58,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac95fb46f7ae8ff3366c99 HTTP/1.1" 200 None 2026-03-09 21:17:58,946 - lava_callback - DEBUG - Processing node 69ac95fb46f7ae8ff3366c99: baseline-arm64-baylibre 2026-03-09 21:17:58,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac95fb46f7ae8ff3366c99 2026-03-09 21:17:58,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:58,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac95fb46f7ae8ff3366c99 HTTP/1.1" 200 None 2026-03-09 21:17:58,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:58,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:17:58,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:58,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac95fb46f7ae8ff3366c99 HTTP/1.1" 200 None 2026-03-09 21:17:59,000 - lava_callback - ERROR - Error processing callback for node 69ac95fb46f7ae8ff3366c99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49616 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:19:18,979 - lava_callback - INFO - Received callback for node 69ac964446f7ae8ff3366cbc INFO: 172.21.0.1:49626 - "POST /node/69ac964446f7ae8ff3366cbc HTTP/1.0" 202 Accepted 2026-03-09 21:19:18,985 - lava_callback - INFO - Starting async job submit for node 69ac964446f7ae8ff3366cbc 2026-03-09 21:19:18,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:19,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac964446f7ae8ff3366cbc HTTP/1.1" 200 None 2026-03-09 21:19:19,001 - lava_callback - DEBUG - Processing node 69ac964446f7ae8ff3366cbc: baseline-arm64-baylibre 2026-03-09 21:19:19,002 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac964446f7ae8ff3366cbc 2026-03-09 21:19:19,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:19,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac964446f7ae8ff3366cbc HTTP/1.1" 200 None 2026-03-09 21:19:19,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:19,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-09 21:19:19,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:19,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac964446f7ae8ff3366cbc HTTP/1.1" 200 None 2026-03-09 21:19:19,058 - lava_callback - ERROR - Error processing callback for node 69ac964446f7ae8ff3366cbc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42358 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:19:38,985 - lava_callback - INFO - Received callback for node 69ae8963c660ead8a389af89 INFO: 172.21.0.1:42362 - "POST /node/69ae8963c660ead8a389af89 HTTP/1.0" 202 Accepted 2026-03-09 21:19:38,995 - lava_callback - INFO - Starting async job submit for node 69ae8963c660ead8a389af89 2026-03-09 21:19:39,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:39,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8963c660ead8a389af89 HTTP/1.1" 200 None 2026-03-09 21:19:39,046 - lava_callback - DEBUG - Processing node 69ae8963c660ead8a389af89: ltp-cap_bounds 2026-03-09 21:19:39,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:19:39,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:19:39,632 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69ae8963c660ead8a389af89/log.txt.gz 2026-03-09 21:19:39,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:19:40,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:19:40,103 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69ae8963c660ead8a389af89/lava-logs.yaml 2026-03-09 21:19:40,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:19:40,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:19:40,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69ae8963c660ead8a389af89/lava_callback.json.gz 2026-03-09 21:19:40,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:40,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8963c660ead8a389af89 HTTP/1.1" 200 None 2026-03-09 21:19:40,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:40,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:19:40,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:40,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae8963c660ead8a389af89 HTTP/1.1" 200 None 2026-03-09 21:19:40,365 - lava_callback - INFO - Completed processing callback for node 69ae8963c660ead8a389af89 INFO: 172.21.0.1:43170 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:19:47,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:47,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57824 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:20:19,005 - lava_callback - INFO - Received callback for node 69ac969046f7ae8ff3366cd2 INFO: 172.21.0.1:57832 - "POST /node/69ac969046f7ae8ff3366cd2 HTTP/1.0" 202 Accepted 2026-03-09 21:20:19,010 - lava_callback - INFO - Starting async job submit for node 69ac969046f7ae8ff3366cd2 2026-03-09 21:20:19,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:20:19,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac969046f7ae8ff3366cd2 HTTP/1.1" 200 None 2026-03-09 21:20:19,027 - lava_callback - DEBUG - Processing node 69ac969046f7ae8ff3366cd2: baseline-arm64-baylibre 2026-03-09 21:20:19,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac969046f7ae8ff3366cd2 2026-03-09 21:20:19,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:20:19,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac969046f7ae8ff3366cd2 HTTP/1.1" 200 None 2026-03-09 21:20:19,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:20:19,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:20:19,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:20:19,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac969046f7ae8ff3366cd2 HTTP/1.1" 200 None 2026-03-09 21:20:19,082 - lava_callback - ERROR - Error processing callback for node 69ac969046f7ae8ff3366cd2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50130 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:21:39,141 - lava_callback - INFO - Received callback for node 69ac96d946f7ae8ff3366cee INFO: 172.21.0.1:50138 - "POST /node/69ac96d946f7ae8ff3366cee HTTP/1.0" 202 Accepted 2026-03-09 21:21:39,147 - lava_callback - INFO - Starting async job submit for node 69ac96d946f7ae8ff3366cee 2026-03-09 21:21:39,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:21:39,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac96d946f7ae8ff3366cee HTTP/1.1" 200 None 2026-03-09 21:21:39,164 - lava_callback - DEBUG - Processing node 69ac96d946f7ae8ff3366cee: baseline-arm64-baylibre 2026-03-09 21:21:39,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac96d946f7ae8ff3366cee 2026-03-09 21:21:39,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:21:39,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac96d946f7ae8ff3366cee HTTP/1.1" 200 None 2026-03-09 21:21:39,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:21:39,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-09 21:21:39,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:21:39,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac96d946f7ae8ff3366cee HTTP/1.1" 200 None 2026-03-09 21:21:39,226 - lava_callback - ERROR - Error processing callback for node 69ac96d946f7ae8ff3366cee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54250 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:21:58,884 - lava_callback - INFO - Received callback for node 69ae8965c660ead8a389af8c INFO: 172.21.0.1:54260 - "POST /node/69ae8965c660ead8a389af8c HTTP/1.0" 202 Accepted 2026-03-09 21:21:58,890 - lava_callback - INFO - Starting async job submit for node 69ae8965c660ead8a389af8c 2026-03-09 21:21:58,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:21:58,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8965c660ead8a389af8c HTTP/1.1" 200 None 2026-03-09 21:21:58,930 - lava_callback - DEBUG - Processing node 69ae8965c660ead8a389af8c: ltp-containers 2026-03-09 21:21:59,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:21:59,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:21:59,507 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69ae8965c660ead8a389af8c/log.txt.gz 2026-03-09 21:21:59,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46244 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:21:59,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:21:59,943 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69ae8965c660ead8a389af8c/lava-logs.yaml 2026-03-09 21:21:59,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:22:00,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:22:00,117 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69ae8965c660ead8a389af8c/lava_callback.json.gz 2026-03-09 21:22:00,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:22:00,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8965c660ead8a389af8c HTTP/1.1" 200 None 2026-03-09 21:22:00,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:22:00,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:22:00,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:22:00,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae8965c660ead8a389af8c HTTP/1.1" 200 None 2026-03-09 21:22:00,206 - lava_callback - INFO - Completed processing callback for node 69ae8965c660ead8a389af8c INFO: 172.21.0.1:46254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46260 - "GET /node/69939574cebe99dd8e0743c0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57424 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:22:17,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:22:17,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41762 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:23:19,078 - lava_callback - INFO - Received callback for node 69af362bce46d8ec186ad2ce INFO: 172.21.0.1:41772 - "POST /node/69af362bce46d8ec186ad2ce HTTP/1.0" 202 Accepted 2026-03-09 21:23:19,085 - lava_callback - INFO - Starting async job submit for node 69af362bce46d8ec186ad2ce 2026-03-09 21:23:19,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:19,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af362bce46d8ec186ad2ce HTTP/1.1" 200 None 2026-03-09 21:23:19,136 - lava_callback - DEBUG - Processing node 69af362bce46d8ec186ad2ce: baseline-arm64-broonie 2026-03-09 21:23:19,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:19,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:19,702 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af362bce46d8ec186ad2ce/log.txt.gz 2026-03-09 21:23:19,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:20,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:20,155 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af362bce46d8ec186ad2ce/lava-logs.yaml 2026-03-09 21:23:20,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:20,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:20,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af362bce46d8ec186ad2ce/lava_callback.json.gz DEBUG: Setup failed for 69af362bce46d8ec186ad2ce. Transitting job node result: incomplete -> fail 2026-03-09 21:23:20,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:20,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af362bce46d8ec186ad2ce HTTP/1.1" 200 None 2026-03-09 21:23:20,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:20,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:23:20,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:20,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af362bce46d8ec186ad2ce HTTP/1.1" 200 None 2026-03-09 21:23:20,564 - lava_callback - INFO - Completed processing callback for node 69af362bce46d8ec186ad2ce INFO: 172.21.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49852 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:23:39,129 - lava_callback - INFO - Received callback for node 69ae8966c660ead8a389af8f INFO: 172.21.0.1:49860 - "POST /node/69ae8966c660ead8a389af8f HTTP/1.0" 202 Accepted 2026-03-09 21:23:39,135 - lava_callback - INFO - Starting async job submit for node 69ae8966c660ead8a389af8f 2026-03-09 21:23:39,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:39,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8966c660ead8a389af8f HTTP/1.1" 200 None 2026-03-09 21:23:39,174 - lava_callback - DEBUG - Processing node 69ae8966c660ead8a389af8f: ltp-controllers 2026-03-09 21:23:39,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:39,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:39,786 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69ae8966c660ead8a389af8f/log.txt.gz 2026-03-09 21:23:40,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:40,081 - lava_callback - INFO - Received callback for node 69af36c7ce46d8ec186ad2e3 INFO: 172.21.0.1:33440 - "POST /node/69af36c7ce46d8ec186ad2e3 HTTP/1.0" 202 Accepted 2026-03-09 21:23:40,087 - lava_callback - INFO - Starting async job submit for node 69af36c7ce46d8ec186ad2e3 2026-03-09 21:23:40,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af36c7ce46d8ec186ad2e3 HTTP/1.1" 200 None 2026-03-09 21:23:40,106 - lava_callback - DEBUG - Processing node 69af36c7ce46d8ec186ad2e3: baseline-arm64-broonie 2026-03-09 21:23:40,111 - lava_callback - WARNING - Failed to upload log 2026-03-09 21:23:40,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:40,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:40,298 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69ae8966c660ead8a389af8f/lava-logs.yaml 2026-03-09 21:23:40,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:40,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:40,430 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af36c7ce46d8ec186ad2e3/lava-logs.yaml 2026-03-09 21:23:40,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:40,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:40,479 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69ae8966c660ead8a389af8f/lava_callback.json.gz 2026-03-09 21:23:40,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8966c660ead8a389af8f HTTP/1.1" 200 None 2026-03-09 21:23:40,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:23:40,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae8966c660ead8a389af8f HTTP/1.1" 200 None 2026-03-09 21:23:40,588 - lava_callback - INFO - Completed processing callback for node 69ae8966c660ead8a389af8f 2026-03-09 21:23:40,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:40,600 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af36c7ce46d8ec186ad2e3/lava_callback.json.gz 2026-03-09 21:23:40,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af36c7ce46d8ec186ad2e3 HTTP/1.1" 200 None 2026-03-09 21:23:40,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:23:40,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af36c7ce46d8ec186ad2e3 HTTP/1.1" 200 None 2026-03-09 21:23:40,720 - lava_callback - INFO - Completed processing callback for node 69af36c7ce46d8ec186ad2e3 INFO: 172.21.0.1:33444 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:23:47,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:47,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51890 - "GET /node/6991f8e101ff1e8e0859de42%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:25:19,457 - lava_callback - INFO - Received callback for node 69ac97b646f7ae8ff3366d38 INFO: 172.21.0.1:52142 - "POST /node/69ac97b646f7ae8ff3366d38 HTTP/1.0" 202 Accepted 2026-03-09 21:25:19,463 - lava_callback - INFO - Starting async job submit for node 69ac97b646f7ae8ff3366d38 2026-03-09 21:25:19,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:25:19,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac97b646f7ae8ff3366d38 HTTP/1.1" 200 None 2026-03-09 21:25:19,479 - lava_callback - DEBUG - Processing node 69ac97b646f7ae8ff3366d38: baseline-arm64-baylibre 2026-03-09 21:25:19,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac97b646f7ae8ff3366d38 2026-03-09 21:25:19,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:25:19,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac97b646f7ae8ff3366d38 HTTP/1.1" 200 None 2026-03-09 21:25:19,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:25:19,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:25:19,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:25:19,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac97b646f7ae8ff3366d38 HTTP/1.1" 200 None 2026-03-09 21:25:19,534 - lava_callback - ERROR - Error processing callback for node 69ac97b646f7ae8ff3366d38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44248 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:26:02,456 - lava_callback - INFO - Received callback for node 69af3702ce46d8ec186ad2e8 INFO: 172.21.0.1:44258 - "POST /node/69af3702ce46d8ec186ad2e8 HTTP/1.0" 202 Accepted 2026-03-09 21:26:02,462 - lava_callback - INFO - Starting async job submit for node 69af3702ce46d8ec186ad2e8 2026-03-09 21:26:02,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:02,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3702ce46d8ec186ad2e8 HTTP/1.1" 200 None 2026-03-09 21:26:02,507 - lava_callback - DEBUG - Processing node 69af3702ce46d8ec186ad2e8: baseline-arm64 2026-03-09 21:26:02,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:02,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:02,962 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3702ce46d8ec186ad2e8/log.txt.gz 2026-03-09 21:26:03,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:03,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:03,342 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3702ce46d8ec186ad2e8/lava-logs.yaml 2026-03-09 21:26:03,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:03,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:03,482 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3702ce46d8ec186ad2e8/lava_callback.json.gz 2026-03-09 21:26:03,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:03,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3702ce46d8ec186ad2e8 HTTP/1.1" 200 None 2026-03-09 21:26:03,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:03,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:26:03,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:03,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3702ce46d8ec186ad2e8 HTTP/1.1" 200 None 2026-03-09 21:26:03,644 - lava_callback - INFO - Completed processing callback for node 69af3702ce46d8ec186ad2e8 INFO: 172.21.0.1:44272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55400 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:26:17,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:17,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 21:26:18,956 - lava_callback - INFO - Received callback for node 69ae8969c660ead8a389af92 INFO: 172.21.0.1:55406 - "POST /node/69ae8969c660ead8a389af92 HTTP/1.0" 202 Accepted 2026-03-09 21:26:18,963 - lava_callback - INFO - Starting async job submit for node 69ae8969c660ead8a389af92 2026-03-09 21:26:18,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:19,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8969c660ead8a389af92 HTTP/1.1" 200 None 2026-03-09 21:26:19,002 - lava_callback - DEBUG - Processing node 69ae8969c660ead8a389af92: ltp-cve 2026-03-09 21:26:19,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:19,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:19,538 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69ae8969c660ead8a389af92/log.txt.gz 2026-03-09 21:26:19,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:20,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:20,005 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69ae8969c660ead8a389af92/lava-logs.yaml 2026-03-09 21:26:20,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:20,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:20,189 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69ae8969c660ead8a389af92/lava_callback.json.gz 2026-03-09 21:26:20,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:20,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8969c660ead8a389af92 HTTP/1.1" 200 None 2026-03-09 21:26:20,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:20,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:26:20,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:20,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae8969c660ead8a389af92 HTTP/1.1" 200 None 2026-03-09 21:26:20,274 - lava_callback - INFO - Completed processing callback for node 69ae8969c660ead8a389af92 INFO: 172.21.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50464 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:26:47,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:47,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46548 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:27:08,348 - lava_callback - INFO - Received callback for node 69af383dce46d8ec186ad30e INFO: 172.21.0.1:46558 - "POST /node/69af383dce46d8ec186ad30e HTTP/1.0" 202 Accepted 2026-03-09 21:27:08,354 - lava_callback - INFO - Starting async job submit for node 69af383dce46d8ec186ad30e 2026-03-09 21:27:08,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:08,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af383dce46d8ec186ad30e HTTP/1.1" 200 None 2026-03-09 21:27:08,404 - lava_callback - DEBUG - Processing node 69af383dce46d8ec186ad30e: baseline-nfs-arm64 2026-03-09 21:27:08,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:08,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:08,923 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af383dce46d8ec186ad30e/log.txt.gz 2026-03-09 21:27:09,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:09,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:09,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af383dce46d8ec186ad30e/lava-logs.yaml 2026-03-09 21:27:09,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:09,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:09,486 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af383dce46d8ec186ad30e/lava_callback.json.gz 2026-03-09 21:27:09,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:09,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af383dce46d8ec186ad30e HTTP/1.1" 200 None 2026-03-09 21:27:09,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:09,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:27:09,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:09,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af383dce46d8ec186ad30e HTTP/1.1" 200 None 2026-03-09 21:27:09,651 - lava_callback - INFO - Completed processing callback for node 69af383dce46d8ec186ad30e INFO: 172.21.0.1:37548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37562 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:27:17,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:17,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58638 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:27:39,654 - lava_callback - INFO - Received callback for node 69af21d6ce46d8ec186acf0e INFO: 172.21.0.1:58654 - "POST /node/69af21d6ce46d8ec186acf0e HTTP/1.0" 202 Accepted 2026-03-09 21:27:39,660 - lava_callback - INFO - Starting async job submit for node 69af21d6ce46d8ec186acf0e 2026-03-09 21:27:39,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:39,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af21d6ce46d8ec186acf0e HTTP/1.1" 200 None 2026-03-09 21:27:39,712 - lava_callback - DEBUG - Processing node 69af21d6ce46d8ec186acf0e: baseline-arm64-broonie 2026-03-09 21:27:40,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:40,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:40,346 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af21d6ce46d8ec186acf0e/log.txt.gz 2026-03-09 21:27:40,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:40,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:40,785 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af21d6ce46d8ec186acf0e/lava-logs.yaml 2026-03-09 21:27:40,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:40,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:40,965 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af21d6ce46d8ec186acf0e/lava_callback.json.gz DEBUG: Setup failed for 69af21d6ce46d8ec186acf0e. Transitting job node result: incomplete -> fail 2026-03-09 21:27:41,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:41,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af21d6ce46d8ec186acf0e HTTP/1.1" 200 None 2026-03-09 21:27:41,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:41,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:27:41,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:41,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af21d6ce46d8ec186acf0e HTTP/1.1" 200 None 2026-03-09 21:27:41,165 - lava_callback - INFO - Completed processing callback for node 69af21d6ce46d8ec186acf0e INFO: 172.21.0.1:45526 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:27:47,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:47,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55832 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:28:59,850 - lava_callback - INFO - Received callback for node 69ac989046f7ae8ff3366d92 INFO: 172.21.0.1:48528 - "POST /node/69ac989046f7ae8ff3366d92 HTTP/1.0" 202 Accepted 2026-03-09 21:28:59,856 - lava_callback - INFO - Starting async job submit for node 69ac989046f7ae8ff3366d92 2026-03-09 21:28:59,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:28:59,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac989046f7ae8ff3366d92 HTTP/1.1" 200 None 2026-03-09 21:28:59,874 - lava_callback - DEBUG - Processing node 69ac989046f7ae8ff3366d92: baseline-arm64-baylibre 2026-03-09 21:28:59,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac989046f7ae8ff3366d92 2026-03-09 21:28:59,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:28:59,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac989046f7ae8ff3366d92 HTTP/1.1" 200 None 2026-03-09 21:28:59,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:28:59,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:28:59,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:28:59,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac989046f7ae8ff3366d92 HTTP/1.1" 200 None 2026-03-09 21:28:59,963 - lava_callback - ERROR - Error processing callback for node 69ac989046f7ae8ff3366d92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55154 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:30:19,626 - lava_callback - INFO - Received callback for node 69ac98d946f7ae8ff3366dae INFO: 172.21.0.1:55166 - "POST /node/69ac98d946f7ae8ff3366dae HTTP/1.0" 202 Accepted 2026-03-09 21:30:19,632 - lava_callback - INFO - Starting async job submit for node 69ac98d946f7ae8ff3366dae 2026-03-09 21:30:19,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:19,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac98d946f7ae8ff3366dae HTTP/1.1" 200 None 2026-03-09 21:30:19,657 - lava_callback - DEBUG - Processing node 69ac98d946f7ae8ff3366dae: baseline-arm64-baylibre 2026-03-09 21:30:19,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac98d946f7ae8ff3366dae 2026-03-09 21:30:19,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:19,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac98d946f7ae8ff3366dae HTTP/1.1" 200 None 2026-03-09 21:30:19,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:19,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-09 21:30:19,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:19,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac98d946f7ae8ff3366dae HTTP/1.1" 200 None 2026-03-09 21:30:19,723 - lava_callback - ERROR - Error processing callback for node 69ac98d946f7ae8ff3366dae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:30:39,642 - lava_callback - INFO - Received callback for node 69ae896cc660ead8a389af99 INFO: 172.21.0.1:52034 - "POST /node/69ae896cc660ead8a389af99 HTTP/1.0" 202 Accepted 2026-03-09 21:30:39,649 - lava_callback - INFO - Starting async job submit for node 69ae896cc660ead8a389af99 2026-03-09 21:30:39,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:39,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae896cc660ead8a389af99 HTTP/1.1" 200 None 2026-03-09 21:30:39,690 - lava_callback - DEBUG - Processing node 69ae896cc660ead8a389af99: ltp-fs_bind 2026-03-09 21:30:40,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:30:40,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:30:40,217 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69ae896cc660ead8a389af99/log.txt.gz 2026-03-09 21:30:40,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:30:40,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:30:40,677 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69ae896cc660ead8a389af99/lava-logs.yaml 2026-03-09 21:30:40,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:30:40,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:30:40,859 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69ae896cc660ead8a389af99/lava_callback.json.gz 2026-03-09 21:30:40,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:40,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae896cc660ead8a389af99 HTTP/1.1" 200 None 2026-03-09 21:30:40,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:40,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:30:40,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:40,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae896cc660ead8a389af99 HTTP/1.1" 200 None 2026-03-09 21:30:40,952 - lava_callback - INFO - Completed processing callback for node 69ae896cc660ead8a389af99 INFO: 172.21.0.1:39198 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:30:47,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:47,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47766 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:31:30,102 - lava_callback - INFO - Received callback for node 69ac992846f7ae8ff3366dc7 INFO: 172.21.0.1:37974 - "POST /node/69ac992846f7ae8ff3366dc7 HTTP/1.0" 202 Accepted 2026-03-09 21:31:30,107 - lava_callback - INFO - Starting async job submit for node 69ac992846f7ae8ff3366dc7 2026-03-09 21:31:30,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:31:30,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac992846f7ae8ff3366dc7 HTTP/1.1" 200 None 2026-03-09 21:31:30,123 - lava_callback - DEBUG - Processing node 69ac992846f7ae8ff3366dc7: baseline-arm64-clabbe 2026-03-09 21:31:30,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac992846f7ae8ff3366dc7 2026-03-09 21:31:30,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:31:30,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac992846f7ae8ff3366dc7 HTTP/1.1" 200 None 2026-03-09 21:31:30,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:31:30,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-09 21:31:30,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:31:30,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac992846f7ae8ff3366dc7 HTTP/1.1" 200 None 2026-03-09 21:31:30,197 - lava_callback - ERROR - Error processing callback for node 69ac992846f7ae8ff3366dc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33496 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:33:50,162 - lava_callback - INFO - Received callback for node 69ac99bb46f7ae8ff3366e0f INFO: 172.21.0.1:39860 - "POST /node/69ac99bb46f7ae8ff3366e0f HTTP/1.0" 202 Accepted 2026-03-09 21:33:50,167 - lava_callback - INFO - Starting async job submit for node 69ac99bb46f7ae8ff3366e0f 2026-03-09 21:33:50,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:33:50,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac99bb46f7ae8ff3366e0f HTTP/1.1" 200 None 2026-03-09 21:33:50,188 - lava_callback - DEBUG - Processing node 69ac99bb46f7ae8ff3366e0f: baseline-arm64-clabbe 2026-03-09 21:33:50,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac99bb46f7ae8ff3366e0f 2026-03-09 21:33:50,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:33:50,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac99bb46f7ae8ff3366e0f HTTP/1.1" 200 None 2026-03-09 21:33:50,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:33:50,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-09 21:33:50,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:33:50,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac99bb46f7ae8ff3366e0f HTTP/1.1" 200 None 2026-03-09 21:33:50,250 - lava_callback - ERROR - Error processing callback for node 69ac99bb46f7ae8ff3366e0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58834 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:34:59,848 - lava_callback - INFO - Received callback for node 69ac99f346f7ae8ff3366e2c INFO: 172.21.0.1:48742 - "POST /node/69ac99f346f7ae8ff3366e2c HTTP/1.0" 202 Accepted 2026-03-09 21:34:59,854 - lava_callback - INFO - Starting async job submit for node 69ac99f346f7ae8ff3366e2c 2026-03-09 21:34:59,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:34:59,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac99f346f7ae8ff3366e2c HTTP/1.1" 200 None 2026-03-09 21:34:59,871 - lava_callback - DEBUG - Processing node 69ac99f346f7ae8ff3366e2c: baseline-arm-baylibre 2026-03-09 21:34:59,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac99f346f7ae8ff3366e2c 2026-03-09 21:34:59,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:34:59,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac99f346f7ae8ff3366e2c HTTP/1.1" 200 None 2026-03-09 21:34:59,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:34:59,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:34:59,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:34:59,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac99f346f7ae8ff3366e2c HTTP/1.1" 200 None 2026-03-09 21:34:59,945 - lava_callback - ERROR - Error processing callback for node 69ac99f346f7ae8ff3366e2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57108 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:35:19,861 - lava_callback - INFO - Received callback for node 69ac9a1646f7ae8ff3366e3f INFO: 172.21.0.1:55948 - "POST /node/69ac9a1646f7ae8ff3366e3f HTTP/1.0" 202 Accepted 2026-03-09 21:35:19,867 - lava_callback - INFO - Starting async job submit for node 69ac9a1646f7ae8ff3366e3f 2026-03-09 21:35:19,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:19,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a1646f7ae8ff3366e3f HTTP/1.1" 200 None 2026-03-09 21:35:19,883 - lava_callback - DEBUG - Processing node 69ac9a1646f7ae8ff3366e3f: baseline-arm-baylibre 2026-03-09 21:35:19,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9a1646f7ae8ff3366e3f 2026-03-09 21:35:19,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:19,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a1646f7ae8ff3366e3f HTTP/1.1" 200 None 2026-03-09 21:35:19,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:19,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:35:19,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:19,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9a1646f7ae8ff3366e3f HTTP/1.1" 200 None 2026-03-09 21:35:19,937 - lava_callback - ERROR - Error processing callback for node 69ac9a1646f7ae8ff3366e3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39914 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:35:59,896 - lava_callback - INFO - Received callback for node 69ac9a3946f7ae8ff3366e4f INFO: 172.21.0.1:40918 - "POST /node/69ac9a3946f7ae8ff3366e4f HTTP/1.0" 202 Accepted 2026-03-09 21:35:59,902 - lava_callback - INFO - Starting async job submit for node 69ac9a3946f7ae8ff3366e4f 2026-03-09 21:35:59,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:59,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a3946f7ae8ff3366e4f HTTP/1.1" 200 None 2026-03-09 21:35:59,919 - lava_callback - DEBUG - Processing node 69ac9a3946f7ae8ff3366e4f: baseline-arm-baylibre 2026-03-09 21:35:59,919 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9a3946f7ae8ff3366e4f 2026-03-09 21:35:59,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:59,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a3946f7ae8ff3366e4f HTTP/1.1" 200 None 2026-03-09 21:35:59,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:59,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:35:59,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:59,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9a3946f7ae8ff3366e4f HTTP/1.1" 200 None 2026-03-09 21:35:59,971 - lava_callback - ERROR - Error processing callback for node 69ac9a3946f7ae8ff3366e4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35458 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:36:39,955 - lava_callback - INFO - Received callback for node 69ac9a5e46f7ae8ff3366e60 INFO: 172.21.0.1:44474 - "POST /node/69ac9a5e46f7ae8ff3366e60 HTTP/1.0" 202 Accepted 2026-03-09 21:36:39,960 - lava_callback - INFO - Starting async job submit for node 69ac9a5e46f7ae8ff3366e60 2026-03-09 21:36:39,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:36:39,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a5e46f7ae8ff3366e60 HTTP/1.1" 200 None 2026-03-09 21:36:39,977 - lava_callback - DEBUG - Processing node 69ac9a5e46f7ae8ff3366e60: baseline-arm-baylibre 2026-03-09 21:36:39,977 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9a5e46f7ae8ff3366e60 2026-03-09 21:36:39,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:36:40,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a5e46f7ae8ff3366e60 HTTP/1.1" 200 None 2026-03-09 21:36:40,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:36:40,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:36:40,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:36:40,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9a5e46f7ae8ff3366e60 HTTP/1.1" 200 None 2026-03-09 21:36:40,054 - lava_callback - ERROR - Error processing callback for node 69ac9a5e46f7ae8ff3366e60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52460 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:37:19,976 - lava_callback - INFO - Received callback for node 69ac9a8546f7ae8ff3366e74 INFO: 172.21.0.1:52836 - "POST /node/69ac9a8546f7ae8ff3366e74 HTTP/1.0" 202 Accepted 2026-03-09 21:37:19,981 - lava_callback - INFO - Starting async job submit for node 69ac9a8546f7ae8ff3366e74 2026-03-09 21:37:19,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:37:19,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a8546f7ae8ff3366e74 HTTP/1.1" 200 None 2026-03-09 21:37:19,998 - lava_callback - DEBUG - Processing node 69ac9a8546f7ae8ff3366e74: baseline-arm-baylibre 2026-03-09 21:37:19,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9a8546f7ae8ff3366e74 2026-03-09 21:37:19,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:37:20,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a8546f7ae8ff3366e74 HTTP/1.1" 200 None 2026-03-09 21:37:20,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:37:20,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:37:20,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:37:20,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9a8546f7ae8ff3366e74 HTTP/1.1" 200 None 2026-03-09 21:37:20,062 - lava_callback - ERROR - Error processing callback for node 69ac9a8546f7ae8ff3366e74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41472 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:38:00,019 - lava_callback - INFO - Received callback for node 69ac9aaf46f7ae8ff3366e8d INFO: 172.21.0.1:42934 - "POST /node/69ac9aaf46f7ae8ff3366e8d HTTP/1.0" 202 Accepted 2026-03-09 21:38:00,025 - lava_callback - INFO - Starting async job submit for node 69ac9aaf46f7ae8ff3366e8d 2026-03-09 21:38:00,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:38:00,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9aaf46f7ae8ff3366e8d HTTP/1.1" 200 None 2026-03-09 21:38:00,042 - lava_callback - DEBUG - Processing node 69ac9aaf46f7ae8ff3366e8d: baseline-arm-baylibre 2026-03-09 21:38:00,042 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9aaf46f7ae8ff3366e8d 2026-03-09 21:38:00,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:38:00,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9aaf46f7ae8ff3366e8d HTTP/1.1" 200 None 2026-03-09 21:38:00,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:38:00,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:38:00,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:38:00,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9aaf46f7ae8ff3366e8d HTTP/1.1" 200 None 2026-03-09 21:38:00,105 - lava_callback - ERROR - Error processing callback for node 69ac9aaf46f7ae8ff3366e8d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39218 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:39:00,055 - lava_callback - INFO - Received callback for node 69ac9adf46f7ae8ff3366eaa INFO: 172.21.0.1:38578 - "POST /node/69ac9adf46f7ae8ff3366eaa HTTP/1.0" 202 Accepted 2026-03-09 21:39:00,061 - lava_callback - INFO - Starting async job submit for node 69ac9adf46f7ae8ff3366eaa 2026-03-09 21:39:00,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:39:00,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9adf46f7ae8ff3366eaa HTTP/1.1" 200 None 2026-03-09 21:39:00,085 - lava_callback - DEBUG - Processing node 69ac9adf46f7ae8ff3366eaa: baseline-arm64-baylibre 2026-03-09 21:39:00,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9adf46f7ae8ff3366eaa 2026-03-09 21:39:00,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:39:00,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9adf46f7ae8ff3366eaa HTTP/1.1" 200 None 2026-03-09 21:39:00,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:39:00,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6edb32da7ccec10683eaa HTTP/1.1" 200 None 2026-03-09 21:39:00,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:39:00,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9adf46f7ae8ff3366eaa HTTP/1.1" 200 None 2026-03-09 21:39:00,144 - lava_callback - ERROR - Error processing callback for node 69ac9adf46f7ae8ff3366eaa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45550 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:41:20,244 - lava_callback - INFO - Received callback for node 69ac9b7646f7ae8ff3366ee3 INFO: 172.21.0.1:49166 - "POST /node/69ac9b7646f7ae8ff3366ee3 HTTP/1.0" 202 Accepted 2026-03-09 21:41:20,250 - lava_callback - INFO - Starting async job submit for node 69ac9b7646f7ae8ff3366ee3 2026-03-09 21:41:20,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:41:20,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9b7646f7ae8ff3366ee3 HTTP/1.1" 200 None 2026-03-09 21:41:20,265 - lava_callback - DEBUG - Processing node 69ac9b7646f7ae8ff3366ee3: baseline-arm64-baylibre 2026-03-09 21:41:20,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9b7646f7ae8ff3366ee3 2026-03-09 21:41:20,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:41:20,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9b7646f7ae8ff3366ee3 HTTP/1.1" 200 None 2026-03-09 21:41:20,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:41:20,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:41:20,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:41:20,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9b7646f7ae8ff3366ee3 HTTP/1.1" 200 None 2026-03-09 21:41:20,322 - lava_callback - ERROR - Error processing callback for node 69ac9b7646f7ae8ff3366ee3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35244 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:45:40,507 - lava_callback - INFO - Received callback for node 69ac9c7946f7ae8ff3366f3d INFO: 172.21.0.1:54016 - "POST /node/69ac9c7946f7ae8ff3366f3d HTTP/1.0" 202 Accepted 2026-03-09 21:45:40,513 - lava_callback - INFO - Starting async job submit for node 69ac9c7946f7ae8ff3366f3d 2026-03-09 21:45:40,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:45:40,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9c7946f7ae8ff3366f3d HTTP/1.1" 200 None 2026-03-09 21:45:40,529 - lava_callback - DEBUG - Processing node 69ac9c7946f7ae8ff3366f3d: baseline-arm64-baylibre 2026-03-09 21:45:40,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9c7946f7ae8ff3366f3d 2026-03-09 21:45:40,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:45:40,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9c7946f7ae8ff3366f3d HTTP/1.1" 200 None 2026-03-09 21:45:40,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:45:40,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:45:40,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:45:40,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9c7946f7ae8ff3366f3d HTTP/1.1" 200 None 2026-03-09 21:45:40,611 - lava_callback - ERROR - Error processing callback for node 69ac9c7946f7ae8ff3366f3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39696 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:48:00,703 - lava_callback - INFO - Received callback for node 69ac9d0d46f7ae8ff3366f6a INFO: 172.21.0.1:39712 - "POST /node/69ac9d0d46f7ae8ff3366f6a HTTP/1.0" 202 Accepted 2026-03-09 21:48:00,709 - lava_callback - INFO - Starting async job submit for node 69ac9d0d46f7ae8ff3366f6a 2026-03-09 21:48:00,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:48:00,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9d0d46f7ae8ff3366f6a HTTP/1.1" 200 None 2026-03-09 21:48:00,725 - lava_callback - DEBUG - Processing node 69ac9d0d46f7ae8ff3366f6a: baseline-arm64-baylibre 2026-03-09 21:48:00,726 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9d0d46f7ae8ff3366f6a 2026-03-09 21:48:00,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:48:00,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9d0d46f7ae8ff3366f6a HTTP/1.1" 200 None 2026-03-09 21:48:00,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:48:00,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6edb32da7ccec10683eaa HTTP/1.1" 200 None 2026-03-09 21:48:00,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:48:00,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9d0d46f7ae8ff3366f6a HTTP/1.1" 200 None 2026-03-09 21:48:00,795 - lava_callback - ERROR - Error processing callback for node 69ac9d0d46f7ae8ff3366f6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49472 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:49:33,803 - lava_callback - INFO - Received callback for node 69af2814ce46d8ec186ad02c INFO: 172.21.0.1:42572 - "POST /node/69af2814ce46d8ec186ad02c HTTP/1.0" 202 Accepted 2026-03-09 21:49:33,809 - lava_callback - INFO - Starting async job submit for node 69af2814ce46d8ec186ad02c 2026-03-09 21:49:33,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:49:33,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2814ce46d8ec186ad02c HTTP/1.1" 200 None 2026-03-09 21:49:33,860 - lava_callback - DEBUG - Processing node 69af2814ce46d8ec186ad02c: baseline-arm64-broonie 2026-03-09 21:49:34,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:49:34,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:49:34,244 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af2814ce46d8ec186ad02c/log.txt.gz 2026-03-09 21:49:34,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:49:34,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:49:34,609 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af2814ce46d8ec186ad02c/lava-logs.yaml 2026-03-09 21:49:34,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:49:34,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:49:34,750 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af2814ce46d8ec186ad02c/lava_callback.json.gz DEBUG: Setup failed for 69af2814ce46d8ec186ad02c. Transitting job node result: incomplete -> fail 2026-03-09 21:49:34,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:49:34,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2814ce46d8ec186ad02c HTTP/1.1" 200 None 2026-03-09 21:49:34,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:49:34,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:49:34,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:49:34,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2814ce46d8ec186ad02c HTTP/1.1" 200 None 2026-03-09 21:49:34,979 - lava_callback - INFO - Completed processing callback for node 69af2814ce46d8ec186ad02c INFO: 172.21.0.1:42578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55074 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:49:47,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:49:47,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38824 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:50:50,941 - lava_callback - INFO - Received callback for node 69ac9daa46f7ae8ff3366faf INFO: 172.21.0.1:58246 - "POST /node/69ac9daa46f7ae8ff3366faf HTTP/1.0" 202 Accepted 2026-03-09 21:50:50,947 - lava_callback - INFO - Starting async job submit for node 69ac9daa46f7ae8ff3366faf 2026-03-09 21:50:50,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:50:50,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9daa46f7ae8ff3366faf HTTP/1.1" 200 None 2026-03-09 21:50:50,963 - lava_callback - DEBUG - Processing node 69ac9daa46f7ae8ff3366faf: baseline-arm64-clabbe 2026-03-09 21:50:50,964 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9daa46f7ae8ff3366faf 2026-03-09 21:50:50,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:50:50,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9daa46f7ae8ff3366faf HTTP/1.1" 200 None 2026-03-09 21:50:50,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:50:50,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:50:50,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:50:51,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9daa46f7ae8ff3366faf HTTP/1.1" 200 None 2026-03-09 21:50:51,023 - lava_callback - ERROR - Error processing callback for node 69ac9daa46f7ae8ff3366faf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:50:59,904 - lava_callback - INFO - Received callback for node 69af3fadce46d8ec186ad486 INFO: 172.21.0.1:33766 - "POST /node/69af3fadce46d8ec186ad486 HTTP/1.0" 202 Accepted 2026-03-09 21:50:59,910 - lava_callback - INFO - Starting async job submit for node 69af3fadce46d8ec186ad486 2026-03-09 21:50:59,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:50:59,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3fadce46d8ec186ad486 HTTP/1.1" 200 None 2026-03-09 21:50:59,966 - lava_callback - DEBUG - Processing node 69af3fadce46d8ec186ad486: baseline-arm64-broonie INFO: 172.21.0.1:33768 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:51:00,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:51:00,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:51:00,585 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3fadce46d8ec186ad486/log.txt.gz 2026-03-09 21:51:00,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:51:01,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:51:01,076 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3fadce46d8ec186ad486/lava-logs.yaml 2026-03-09 21:51:01,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:51:01,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:51:01,250 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3fadce46d8ec186ad486/lava_callback.json.gz DEBUG: Setup failed for 69af3fadce46d8ec186ad486. Transitting job node result: incomplete -> fail 2026-03-09 21:51:01,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:51:01,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3fadce46d8ec186ad486 HTTP/1.1" 200 None 2026-03-09 21:51:01,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:51:01,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:51:01,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:51:01,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3fadce46d8ec186ad486 HTTP/1.1" 200 None 2026-03-09 21:51:01,461 - lava_callback - INFO - Completed processing callback for node 69af3fadce46d8ec186ad486 INFO: 172.21.0.1:33774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52964 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:51:17,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:51:17,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37382 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:53:21,125 - lava_callback - INFO - Received callback for node 69ac9e4246f7ae8ff3366fdc INFO: 172.21.0.1:55568 - "POST /node/69ac9e4246f7ae8ff3366fdc HTTP/1.0" 202 Accepted 2026-03-09 21:53:21,131 - lava_callback - INFO - Starting async job submit for node 69ac9e4246f7ae8ff3366fdc 2026-03-09 21:53:21,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:53:21,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9e4246f7ae8ff3366fdc HTTP/1.1" 200 None 2026-03-09 21:53:21,147 - lava_callback - DEBUG - Processing node 69ac9e4246f7ae8ff3366fdc: baseline-arm64-baylibre 2026-03-09 21:53:21,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9e4246f7ae8ff3366fdc 2026-03-09 21:53:21,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:53:21,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9e4246f7ae8ff3366fdc HTTP/1.1" 200 None 2026-03-09 21:53:21,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:53:21,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:53:21,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:53:21,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9e4246f7ae8ff3366fdc HTTP/1.1" 200 None 2026-03-09 21:53:21,212 - lava_callback - ERROR - Error processing callback for node 69ac9e4246f7ae8ff3366fdc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:54:00,066 - lava_callback - INFO - Received callback for node 69af4048ce46d8ec186ad496 INFO: 172.21.0.1:38990 - "POST /node/69af4048ce46d8ec186ad496 HTTP/1.0" 202 Accepted 2026-03-09 21:54:00,073 - lava_callback - INFO - Starting async job submit for node 69af4048ce46d8ec186ad496 2026-03-09 21:54:00,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:00,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af4048ce46d8ec186ad496 HTTP/1.1" 200 None 2026-03-09 21:54:00,125 - lava_callback - DEBUG - Processing node 69af4048ce46d8ec186ad496: baseline-arm64-broonie INFO: 172.21.0.1:39004 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:54:00,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:00,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:00,692 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af4048ce46d8ec186ad496/log.txt.gz 2026-03-09 21:54:00,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:01,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:01,166 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af4048ce46d8ec186ad496/lava-logs.yaml 2026-03-09 21:54:01,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:01,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:01,340 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af4048ce46d8ec186ad496/lava_callback.json.gz DEBUG: Setup failed for 69af4048ce46d8ec186ad496. Transitting job node result: incomplete -> fail 2026-03-09 21:54:01,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:01,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af4048ce46d8ec186ad496 HTTP/1.1" 200 None 2026-03-09 21:54:01,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:01,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:54:01,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:01,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af4048ce46d8ec186ad496 HTTP/1.1" 200 None 2026-03-09 21:54:01,548 - lava_callback - INFO - Completed processing callback for node 69af4048ce46d8ec186ad496 INFO: 172.21.0.1:39010 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:54:13,730 - lava_callback - INFO - Received callback for node 69af3cf2ce46d8ec186ad412 INFO: 172.21.0.1:47542 - "POST /node/69af3cf2ce46d8ec186ad412 HTTP/1.0" 202 Accepted 2026-03-09 21:54:13,782 - lava_callback - INFO - Starting async job submit for node 69af3cf2ce46d8ec186ad412 2026-03-09 21:54:13,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:13,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3cf2ce46d8ec186ad412 HTTP/1.1" 200 None 2026-03-09 21:54:13,823 - lava_callback - DEBUG - Processing node 69af3cf2ce46d8ec186ad412: baseline-arm-pengutronix 2026-03-09 21:54:13,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:14,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:14,066 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af3cf2ce46d8ec186ad412/log.txt.gz 2026-03-09 21:54:14,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:14,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:14,284 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af3cf2ce46d8ec186ad412/lava-logs.yaml 2026-03-09 21:54:14,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:14,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:14,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af3cf2ce46d8ec186ad412/lava_callback.json.gz DEBUG: Setup failed for 69af3cf2ce46d8ec186ad412. Transitting job node result: incomplete -> fail 2026-03-09 21:54:14,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:14,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3cf2ce46d8ec186ad412 HTTP/1.1" 200 None 2026-03-09 21:54:14,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:14,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 21:54:14,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:14,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3cf2ce46d8ec186ad412 HTTP/1.1" 200 None 2026-03-09 21:54:14,597 - lava_callback - INFO - Completed processing callback for node 69af3cf2ce46d8ec186ad412 INFO: 172.21.0.1:47556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47560 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:54:17,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:17,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43798 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:54:40,981 - lava_callback - INFO - Received callback for node 69ac9e9046f7ae8ff3366ffe INFO: 172.21.0.1:55660 - "POST /node/69ac9e9046f7ae8ff3366ffe HTTP/1.0" 202 Accepted 2026-03-09 21:54:40,987 - lava_callback - INFO - Starting async job submit for node 69ac9e9046f7ae8ff3366ffe 2026-03-09 21:54:40,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:41,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9e9046f7ae8ff3366ffe HTTP/1.1" 200 None 2026-03-09 21:54:41,003 - lava_callback - DEBUG - Processing node 69ac9e9046f7ae8ff3366ffe: baseline-arm64-baylibre 2026-03-09 21:54:41,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9e9046f7ae8ff3366ffe 2026-03-09 21:54:41,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:41,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9e9046f7ae8ff3366ffe HTTP/1.1" 200 None 2026-03-09 21:54:41,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:41,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6edb32da7ccec10683eaa HTTP/1.1" 200 None 2026-03-09 21:54:41,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:41,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9e9046f7ae8ff3366ffe HTTP/1.1" 200 None 2026-03-09 21:54:41,071 - lava_callback - ERROR - Error processing callback for node 69ac9e9046f7ae8ff3366ffe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35286 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:56:01,068 - lava_callback - INFO - Received callback for node 69ac9ee246f7ae8ff3367022 INFO: 172.21.0.1:35290 - "POST /node/69ac9ee246f7ae8ff3367022 HTTP/1.0" 202 Accepted 2026-03-09 21:56:01,074 - lava_callback - INFO - Starting async job submit for node 69ac9ee246f7ae8ff3367022 2026-03-09 21:56:01,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:56:01,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9ee246f7ae8ff3367022 HTTP/1.1" 200 None 2026-03-09 21:56:01,092 - lava_callback - DEBUG - Processing node 69ac9ee246f7ae8ff3367022: baseline-arm64-baylibre 2026-03-09 21:56:01,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9ee246f7ae8ff3367022 2026-03-09 21:56:01,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:56:01,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9ee246f7ae8ff3367022 HTTP/1.1" 200 None 2026-03-09 21:56:01,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:56:01,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:56:01,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:56:01,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9ee246f7ae8ff3367022 HTTP/1.1" 200 None 2026-03-09 21:56:01,157 - lava_callback - ERROR - Error processing callback for node 69ac9ee246f7ae8ff3367022: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45162 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:57:31,297 - lava_callback - INFO - Received callback for node 69ac9f3a46f7ae8ff3367044 INFO: 172.21.0.1:56290 - "POST /node/69ac9f3a46f7ae8ff3367044 HTTP/1.0" 202 Accepted 2026-03-09 21:57:31,302 - lava_callback - INFO - Starting async job submit for node 69ac9f3a46f7ae8ff3367044 2026-03-09 21:57:31,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:57:31,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9f3a46f7ae8ff3367044 HTTP/1.1" 200 None 2026-03-09 21:57:31,318 - lava_callback - DEBUG - Processing node 69ac9f3a46f7ae8ff3367044: baseline-arm64-clabbe 2026-03-09 21:57:31,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9f3a46f7ae8ff3367044 2026-03-09 21:57:31,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:57:31,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9f3a46f7ae8ff3367044 HTTP/1.1" 200 None 2026-03-09 21:57:31,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:57:31,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:57:31,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:57:31,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9f3a46f7ae8ff3367044 HTTP/1.1" 200 None 2026-03-09 21:57:31,378 - lava_callback - ERROR - Error processing callback for node 69ac9f3a46f7ae8ff3367044: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40666 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:58:51,305 - lava_callback - INFO - Received callback for node 69ac9f8746f7ae8ff3367067 INFO: 172.21.0.1:59462 - "POST /node/69ac9f8746f7ae8ff3367067 HTTP/1.0" 202 Accepted 2026-03-09 21:58:51,311 - lava_callback - INFO - Starting async job submit for node 69ac9f8746f7ae8ff3367067 2026-03-09 21:58:51,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:58:51,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9f8746f7ae8ff3367067 HTTP/1.1" 200 None 2026-03-09 21:58:51,345 - lava_callback - DEBUG - Processing node 69ac9f8746f7ae8ff3367067: baseline-arm64-clabbe 2026-03-09 21:58:51,345 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9f8746f7ae8ff3367067 2026-03-09 21:58:51,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:58:51,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9f8746f7ae8ff3367067 HTTP/1.1" 200 None 2026-03-09 21:58:51,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:58:51,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:58:51,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:58:51,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9f8746f7ae8ff3367067 HTTP/1.1" 200 None 2026-03-09 21:58:51,403 - lava_callback - ERROR - Error processing callback for node 69ac9f8746f7ae8ff3367067: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36256 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:59:41,254 - lava_callback - INFO - Received callback for node 69ac9fca46f7ae8ff336707e INFO: 172.21.0.1:33464 - "POST /node/69ac9fca46f7ae8ff336707e HTTP/1.0" 202 Accepted 2026-03-09 21:59:41,259 - lava_callback - INFO - Starting async job submit for node 69ac9fca46f7ae8ff336707e 2026-03-09 21:59:41,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:41,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9fca46f7ae8ff336707e HTTP/1.1" 200 None 2026-03-09 21:59:41,276 - lava_callback - DEBUG - Processing node 69ac9fca46f7ae8ff336707e: baseline-arm64-baylibre 2026-03-09 21:59:41,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9fca46f7ae8ff336707e 2026-03-09 21:59:41,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:41,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9fca46f7ae8ff336707e HTTP/1.1" 200 None 2026-03-09 21:59:41,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:41,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6edb32da7ccec10683eaa HTTP/1.1" 200 None 2026-03-09 21:59:41,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:41,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9fca46f7ae8ff336707e HTTP/1.1" 200 None 2026-03-09 21:59:41,360 - lava_callback - ERROR - Error processing callback for node 69ac9fca46f7ae8ff336707e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33480 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:59:47,063 - lava_callback - INFO - Received callback for node 69af3751ce46d8ec186ad2f8 INFO: 172.21.0.1:33494 - "POST /node/69af3751ce46d8ec186ad2f8 HTTP/1.0" 202 Accepted 2026-03-09 21:59:47,070 - lava_callback - INFO - Starting async job submit for node 69af3751ce46d8ec186ad2f8 2026-03-09 21:59:47,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:47,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3751ce46d8ec186ad2f8 HTTP/1.1" 200 None 2026-03-09 21:59:47,110 - lava_callback - DEBUG - Processing node 69af3751ce46d8ec186ad2f8: baseline-arm64 2026-03-09 21:59:47,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:59:47,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:59:47,795 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3751ce46d8ec186ad2f8/log.txt.gz 2026-03-09 21:59:48,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:59:48,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:59:48,275 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3751ce46d8ec186ad2f8/lava-logs.yaml 2026-03-09 21:59:48,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:59:48,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:59:48,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3751ce46d8ec186ad2f8/lava_callback.json.gz 2026-03-09 21:59:48,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:48,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3751ce46d8ec186ad2f8 HTTP/1.1" 200 None 2026-03-09 21:59:48,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:48,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:59:48,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:48,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3751ce46d8ec186ad2f8 HTTP/1.1" 200 None 2026-03-09 21:59:48,594 - lava_callback - INFO - Completed processing callback for node 69af3751ce46d8ec186ad2f8 INFO: 172.21.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42606 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:00:17,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:00:17,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57922 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:00:55,407 - lava_callback - INFO - Received callback for node 69af42a5ce46d8ec186ad50d INFO: 172.21.0.1:54636 - "POST /node/69af42a5ce46d8ec186ad50d HTTP/1.0" 202 Accepted 2026-03-09 22:00:55,413 - lava_callback - INFO - Starting async job submit for node 69af42a5ce46d8ec186ad50d 2026-03-09 22:00:55,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:00:55,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af42a5ce46d8ec186ad50d HTTP/1.1" 200 None 2026-03-09 22:00:55,468 - lava_callback - DEBUG - Processing node 69af42a5ce46d8ec186ad50d: baseline-arm64-broonie 2026-03-09 22:00:55,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:00:55,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:00:55,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af42a5ce46d8ec186ad50d/log.txt.gz 2026-03-09 22:00:55,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:00:56,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:00:56,217 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af42a5ce46d8ec186ad50d/lava-logs.yaml 2026-03-09 22:00:56,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:00:56,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:00:56,378 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af42a5ce46d8ec186ad50d/lava_callback.json.gz DEBUG: Setup failed for 69af42a5ce46d8ec186ad50d. Transitting job node result: incomplete -> fail 2026-03-09 22:00:56,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:00:56,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af42a5ce46d8ec186ad50d HTTP/1.1" 200 None 2026-03-09 22:00:56,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:00:56,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 22:00:56,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:00:56,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af42a5ce46d8ec186ad50d HTTP/1.1" 200 None INFO: 172.21.0.1:54638 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:00:56,593 - lava_callback - INFO - Completed processing callback for node 69af42a5ce46d8ec186ad50d INFO: 172.21.0.1:45560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53002 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:01:17,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:01:17,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44140 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:01:56,889 - lava_callback - INFO - Received callback for node 69af37a5ce46d8ec186ad2ff INFO: 172.21.0.1:44154 - "POST /node/69af37a5ce46d8ec186ad2ff HTTP/1.0" 202 Accepted 2026-03-09 22:01:56,896 - lava_callback - INFO - Starting async job submit for node 69af37a5ce46d8ec186ad2ff 2026-03-09 22:01:56,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:01:56,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af37a5ce46d8ec186ad2ff HTTP/1.1" 200 None 2026-03-09 22:01:56,962 - lava_callback - DEBUG - Processing node 69af37a5ce46d8ec186ad2ff: baseline-nfs-arm64 2026-03-09 22:01:57,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:01:57,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:01:57,629 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af37a5ce46d8ec186ad2ff/log.txt.gz 2026-03-09 22:01:57,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:01:58,133 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:01:58,135 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af37a5ce46d8ec186ad2ff/lava-logs.yaml 2026-03-09 22:01:58,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:01:58,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:01:58,337 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af37a5ce46d8ec186ad2ff/lava_callback.json.gz 2026-03-09 22:01:58,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:01:58,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af37a5ce46d8ec186ad2ff HTTP/1.1" 200 None 2026-03-09 22:01:58,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:01:58,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 22:01:58,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:01:58,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af37a5ce46d8ec186ad2ff HTTP/1.1" 200 None 2026-03-09 22:01:58,492 - lava_callback - INFO - Completed processing callback for node 69af37a5ce46d8ec186ad2ff INFO: 172.21.0.1:40492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56828 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:02:17,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:02:17,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:06:01,641 - lava_callback - INFO - Received callback for node 69aca14646f7ae8ff3367104 INFO: 172.21.0.1:40924 - "POST /node/69aca14646f7ae8ff3367104 HTTP/1.0" 202 Accepted 2026-03-09 22:06:01,646 - lava_callback - INFO - Starting async job submit for node 69aca14646f7ae8ff3367104 2026-03-09 22:06:01,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:01,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca14646f7ae8ff3367104 HTTP/1.1" 200 None 2026-03-09 22:06:01,692 - lava_callback - DEBUG - Processing node 69aca14646f7ae8ff3367104: baseline-arm-baylibre 2026-03-09 22:06:01,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca14646f7ae8ff3367104 2026-03-09 22:06:01,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:01,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca14646f7ae8ff3367104 HTTP/1.1" 200 None 2026-03-09 22:06:01,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:01,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:06:01,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:01,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca14646f7ae8ff3367104 HTTP/1.1" 200 None 2026-03-09 22:06:01,791 - lava_callback - ERROR - Error processing callback for node 69aca14646f7ae8ff3367104: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34300 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:06:41,616 - lava_callback - INFO - Received callback for node 69aca16a46f7ae8ff3367111 INFO: 172.21.0.1:57732 - "POST /node/69aca16a46f7ae8ff3367111 HTTP/1.0" 202 Accepted 2026-03-09 22:06:41,621 - lava_callback - INFO - Starting async job submit for node 69aca16a46f7ae8ff3367111 2026-03-09 22:06:41,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:41,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca16a46f7ae8ff3367111 HTTP/1.1" 200 None 2026-03-09 22:06:41,638 - lava_callback - DEBUG - Processing node 69aca16a46f7ae8ff3367111: baseline-arm-baylibre 2026-03-09 22:06:41,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca16a46f7ae8ff3367111 2026-03-09 22:06:41,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:41,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca16a46f7ae8ff3367111 HTTP/1.1" 200 None 2026-03-09 22:06:41,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:41,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:06:41,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:41,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca16a46f7ae8ff3367111 HTTP/1.1" 200 None 2026-03-09 22:06:41,696 - lava_callback - ERROR - Error processing callback for node 69aca16a46f7ae8ff3367111: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48340 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:07:21,720 - lava_callback - INFO - Received callback for node 69aca18e46f7ae8ff3367121 INFO: 172.21.0.1:41334 - "POST /node/69aca18e46f7ae8ff3367121 HTTP/1.0" 202 Accepted 2026-03-09 22:07:21,725 - lava_callback - INFO - Starting async job submit for node 69aca18e46f7ae8ff3367121 2026-03-09 22:07:21,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:07:21,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca18e46f7ae8ff3367121 HTTP/1.1" 200 None 2026-03-09 22:07:21,742 - lava_callback - DEBUG - Processing node 69aca18e46f7ae8ff3367121: baseline-arm-baylibre 2026-03-09 22:07:21,742 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca18e46f7ae8ff3367121 2026-03-09 22:07:21,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:07:21,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca18e46f7ae8ff3367121 HTTP/1.1" 200 None 2026-03-09 22:07:21,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:07:21,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:07:21,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:07:21,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca18e46f7ae8ff3367121 HTTP/1.1" 200 None 2026-03-09 22:07:21,792 - lava_callback - ERROR - Error processing callback for node 69aca18e46f7ae8ff3367121: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34436 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:08:01,682 - lava_callback - INFO - Received callback for node 69aca1b146f7ae8ff3367134 INFO: 172.21.0.1:34446 - "POST /node/69aca1b146f7ae8ff3367134 HTTP/1.0" 202 Accepted 2026-03-09 22:08:01,688 - lava_callback - INFO - Starting async job submit for node 69aca1b146f7ae8ff3367134 2026-03-09 22:08:01,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:01,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1b146f7ae8ff3367134 HTTP/1.1" 200 None 2026-03-09 22:08:01,704 - lava_callback - DEBUG - Processing node 69aca1b146f7ae8ff3367134: baseline-arm-baylibre 2026-03-09 22:08:01,704 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca1b146f7ae8ff3367134 2026-03-09 22:08:01,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:01,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1b146f7ae8ff3367134 HTTP/1.1" 200 None 2026-03-09 22:08:01,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:01,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:08:01,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:01,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca1b146f7ae8ff3367134 HTTP/1.1" 200 None 2026-03-09 22:08:01,761 - lava_callback - ERROR - Error processing callback for node 69aca1b146f7ae8ff3367134: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36784 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:08:41,764 - lava_callback - INFO - Received callback for node 69aca1d646f7ae8ff3367143 INFO: 172.21.0.1:57438 - "POST /node/69aca1d646f7ae8ff3367143 HTTP/1.0" 202 Accepted 2026-03-09 22:08:41,771 - lava_callback - INFO - Starting async job submit for node 69aca1d646f7ae8ff3367143 2026-03-09 22:08:41,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:41,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1d646f7ae8ff3367143 HTTP/1.1" 200 None 2026-03-09 22:08:41,840 - lava_callback - DEBUG - Processing node 69aca1d646f7ae8ff3367143: baseline-arm-baylibre 2026-03-09 22:08:41,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca1d646f7ae8ff3367143 2026-03-09 22:08:41,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:41,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1d646f7ae8ff3367143 HTTP/1.1" 200 None 2026-03-09 22:08:41,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:41,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:08:41,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:41,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca1d646f7ae8ff3367143 HTTP/1.1" 200 None 2026-03-09 22:08:41,893 - lava_callback - ERROR - Error processing callback for node 69aca1d646f7ae8ff3367143: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48630 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:09:01,752 - lava_callback - INFO - Received callback for node 69aca1fa46f7ae8ff3367158 INFO: 172.21.0.1:48642 - "POST /node/69aca1fa46f7ae8ff3367158 HTTP/1.0" 202 Accepted 2026-03-09 22:09:01,758 - lava_callback - INFO - Starting async job submit for node 69aca1fa46f7ae8ff3367158 2026-03-09 22:09:01,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:09:01,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1fa46f7ae8ff3367158 HTTP/1.1" 200 None 2026-03-09 22:09:01,782 - lava_callback - DEBUG - Processing node 69aca1fa46f7ae8ff3367158: baseline-arm-baylibre 2026-03-09 22:09:01,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca1fa46f7ae8ff3367158 2026-03-09 22:09:01,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:09:01,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1fa46f7ae8ff3367158 HTTP/1.1" 200 None 2026-03-09 22:09:01,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:09:01,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:09:01,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:09:01,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca1fa46f7ae8ff3367158 HTTP/1.1" 200 None 2026-03-09 22:09:01,839 - lava_callback - ERROR - Error processing callback for node 69aca1fa46f7ae8ff3367158: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:21:31,967 - lava_callback - INFO - Received callback for node 69af454cce46d8ec186ad567 INFO: 172.21.0.1:52720 - "POST /node/69af454cce46d8ec186ad567 HTTP/1.0" 202 Accepted 2026-03-09 22:21:31,974 - lava_callback - INFO - Starting async job submit for node 69af454cce46d8ec186ad567 2026-03-09 22:21:32,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:21:32,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af454cce46d8ec186ad567 HTTP/1.1" 200 None 2026-03-09 22:21:32,063 - lava_callback - DEBUG - Processing node 69af454cce46d8ec186ad567: baseline-nfs-arm64 2026-03-09 22:21:32,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:21:32,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:21:32,523 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af454cce46d8ec186ad567/log.txt.gz 2026-03-09 22:21:32,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:21:32,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:21:32,910 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af454cce46d8ec186ad567/lava-logs.yaml 2026-03-09 22:21:32,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:21:33,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:21:33,073 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af454cce46d8ec186ad567/lava_callback.json.gz 2026-03-09 22:21:33,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:21:33,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af454cce46d8ec186ad567 HTTP/1.1" 200 None 2026-03-09 22:21:33,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:21:33,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-09 22:21:33,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:21:33,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af454cce46d8ec186ad567 HTTP/1.1" 200 None 2026-03-09 22:21:33,234 - lava_callback - INFO - Completed processing callback for node 69af454cce46d8ec186ad567 INFO: 172.21.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52238 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:21:47,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:21:47,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34426 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:38:23,439 - lava_callback - INFO - Received callback for node 69aca8da46f7ae8ff33672a9 INFO: 172.21.0.1:39580 - "POST /node/69aca8da46f7ae8ff33672a9 HTTP/1.0" 202 Accepted 2026-03-09 22:38:23,445 - lava_callback - INFO - Starting async job submit for node 69aca8da46f7ae8ff33672a9 2026-03-09 22:38:23,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:38:23,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca8da46f7ae8ff33672a9 HTTP/1.1" 200 None 2026-03-09 22:38:23,479 - lava_callback - DEBUG - Processing node 69aca8da46f7ae8ff33672a9: baseline-arm64-baylibre 2026-03-09 22:38:23,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca8da46f7ae8ff33672a9 2026-03-09 22:38:23,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:38:23,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca8da46f7ae8ff33672a9 HTTP/1.1" 200 None 2026-03-09 22:38:23,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:38:23,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:38:23,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:38:23,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca8da46f7ae8ff33672a9 HTTP/1.1" 200 None 2026-03-09 22:38:23,578 - lava_callback - ERROR - Error processing callback for node 69aca8da46f7ae8ff33672a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51966 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:39:43,574 - lava_callback - INFO - Received callback for node 69aca92a46f7ae8ff33672c2 INFO: 172.21.0.1:58722 - "POST /node/69aca92a46f7ae8ff33672c2 HTTP/1.0" 202 Accepted 2026-03-09 22:39:43,579 - lava_callback - INFO - Starting async job submit for node 69aca92a46f7ae8ff33672c2 2026-03-09 22:39:43,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:39:43,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca92a46f7ae8ff33672c2 HTTP/1.1" 200 None 2026-03-09 22:39:43,601 - lava_callback - DEBUG - Processing node 69aca92a46f7ae8ff33672c2: baseline-arm64-baylibre 2026-03-09 22:39:43,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca92a46f7ae8ff33672c2 2026-03-09 22:39:43,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:39:43,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca92a46f7ae8ff33672c2 HTTP/1.1" 200 None 2026-03-09 22:39:43,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:39:43,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:39:43,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:39:43,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca92a46f7ae8ff33672c2 HTTP/1.1" 200 None 2026-03-09 22:39:43,658 - lava_callback - ERROR - Error processing callback for node 69aca92a46f7ae8ff33672c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41732 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:42:23,692 - lava_callback - INFO - Received callback for node 69aca9ca46f7ae8ff3367318 INFO: 172.21.0.1:36422 - "POST /node/69aca9ca46f7ae8ff3367318 HTTP/1.0" 202 Accepted 2026-03-09 22:42:23,697 - lava_callback - INFO - Starting async job submit for node 69aca9ca46f7ae8ff3367318 2026-03-09 22:42:23,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:42:23,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca9ca46f7ae8ff3367318 HTTP/1.1" 200 None 2026-03-09 22:42:23,717 - lava_callback - DEBUG - Processing node 69aca9ca46f7ae8ff3367318: baseline-arm64-baylibre 2026-03-09 22:42:23,717 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca9ca46f7ae8ff3367318 2026-03-09 22:42:23,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:42:23,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca9ca46f7ae8ff3367318 HTTP/1.1" 200 None 2026-03-09 22:42:23,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:42:23,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:42:23,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:42:23,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca9ca46f7ae8ff3367318 HTTP/1.1" 200 None 2026-03-09 22:42:23,768 - lava_callback - ERROR - Error processing callback for node 69aca9ca46f7ae8ff3367318: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40396 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:45:03,834 - lava_callback - INFO - Received callback for node 69acaa6246f7ae8ff336738c INFO: 172.21.0.1:40404 - "POST /node/69acaa6246f7ae8ff336738c HTTP/1.0" 202 Accepted 2026-03-09 22:45:03,839 - lava_callback - INFO - Starting async job submit for node 69acaa6246f7ae8ff336738c 2026-03-09 22:45:03,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:45:03,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acaa6246f7ae8ff336738c HTTP/1.1" 200 None 2026-03-09 22:45:03,865 - lava_callback - DEBUG - Processing node 69acaa6246f7ae8ff336738c: baseline-arm64-baylibre 2026-03-09 22:45:03,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acaa6246f7ae8ff336738c 2026-03-09 22:45:03,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:45:03,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acaa6246f7ae8ff336738c HTTP/1.1" 200 None 2026-03-09 22:45:03,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:45:03,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:45:03,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:45:03,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acaa6246f7ae8ff336738c HTTP/1.1" 200 None 2026-03-09 22:45:03,956 - lava_callback - ERROR - Error processing callback for node 69acaa6246f7ae8ff336738c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:53:33,898 - lava_callback - INFO - Received callback for node 69acac6b46f7ae8ff3367492 INFO: 172.21.0.1:43690 - "POST /node/69acac6b46f7ae8ff3367492 HTTP/1.0" 202 Accepted 2026-03-09 22:53:33,903 - lava_callback - INFO - Starting async job submit for node 69acac6b46f7ae8ff3367492 2026-03-09 22:53:33,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:53:33,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acac6b46f7ae8ff3367492 HTTP/1.1" 200 None 2026-03-09 22:53:33,925 - lava_callback - DEBUG - Processing node 69acac6b46f7ae8ff3367492: baseline-arm-clabbe 2026-03-09 22:53:33,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acac6b46f7ae8ff3367492 2026-03-09 22:53:33,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:53:33,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acac6b46f7ae8ff3367492 HTTP/1.1" 200 None 2026-03-09 22:53:33,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:53:33,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 22:53:33,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:53:33,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acac6b46f7ae8ff3367492 HTTP/1.1" 200 None 2026-03-09 22:53:33,985 - lava_callback - ERROR - Error processing callback for node 69acac6b46f7ae8ff3367492: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53870 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:54:53,832 - lava_callback - INFO - Received callback for node 69acacb546f7ae8ff33674ae INFO: 172.21.0.1:43370 - "POST /node/69acacb546f7ae8ff33674ae HTTP/1.0" 202 Accepted 2026-03-09 22:54:53,837 - lava_callback - INFO - Starting async job submit for node 69acacb546f7ae8ff33674ae 2026-03-09 22:54:53,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:54:53,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acacb546f7ae8ff33674ae HTTP/1.1" 200 None 2026-03-09 22:54:53,977 - lava_callback - DEBUG - Processing node 69acacb546f7ae8ff33674ae: baseline-arm-clabbe 2026-03-09 22:54:53,977 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acacb546f7ae8ff33674ae 2026-03-09 22:54:53,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:54:53,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acacb546f7ae8ff33674ae HTTP/1.1" 200 None 2026-03-09 22:54:53,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:54:54,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 22:54:54,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:54:54,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acacb546f7ae8ff33674ae HTTP/1.1" 200 None 2026-03-09 22:54:54,035 - lava_callback - ERROR - Error processing callback for node 69acacb546f7ae8ff33674ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46408 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:55:33,863 - lava_callback - INFO - Received callback for node 69acacdd46f7ae8ff33674c5 INFO: 172.21.0.1:34402 - "POST /node/69acacdd46f7ae8ff33674c5 HTTP/1.0" 202 Accepted 2026-03-09 22:55:33,868 - lava_callback - INFO - Starting async job submit for node 69acacdd46f7ae8ff33674c5 2026-03-09 22:55:33,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:55:33,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acacdd46f7ae8ff33674c5 HTTP/1.1" 200 None 2026-03-09 22:55:33,919 - lava_callback - DEBUG - Processing node 69acacdd46f7ae8ff33674c5: baseline-arm-clabbe 2026-03-09 22:55:33,919 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acacdd46f7ae8ff33674c5 2026-03-09 22:55:33,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:55:33,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acacdd46f7ae8ff33674c5 HTTP/1.1" 200 None 2026-03-09 22:55:33,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:55:33,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 22:55:33,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:55:33,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acacdd46f7ae8ff33674c5 HTTP/1.1" 200 None 2026-03-09 22:55:33,970 - lava_callback - ERROR - Error processing callback for node 69acacdd46f7ae8ff33674c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34352 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:56:24,571 - lava_callback - INFO - Received callback for node 69acad0b46f7ae8ff33674e4 INFO: 172.21.0.1:33164 - "POST /node/69acad0b46f7ae8ff33674e4 HTTP/1.0" 202 Accepted 2026-03-09 22:56:24,576 - lava_callback - INFO - Starting async job submit for node 69acad0b46f7ae8ff33674e4 2026-03-09 22:56:24,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:56:24,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad0b46f7ae8ff33674e4 HTTP/1.1" 200 None 2026-03-09 22:56:24,596 - lava_callback - DEBUG - Processing node 69acad0b46f7ae8ff33674e4: baseline-arm64-baylibre 2026-03-09 22:56:24,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acad0b46f7ae8ff33674e4 2026-03-09 22:56:24,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:56:24,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad0b46f7ae8ff33674e4 HTTP/1.1" 200 None 2026-03-09 22:56:24,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:56:24,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:56:24,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:56:24,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acad0b46f7ae8ff33674e4 HTTP/1.1" 200 None 2026-03-09 22:56:24,652 - lava_callback - ERROR - Error processing callback for node 69acad0b46f7ae8ff33674e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40334 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:57:33,950 - lava_callback - INFO - Received callback for node 69acad5346f7ae8ff3367517 INFO: 172.21.0.1:42856 - "POST /node/69acad5346f7ae8ff3367517 HTTP/1.0" 202 Accepted 2026-03-09 22:57:33,955 - lava_callback - INFO - Starting async job submit for node 69acad5346f7ae8ff3367517 2026-03-09 22:57:33,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:57:33,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad5346f7ae8ff3367517 HTTP/1.1" 200 None 2026-03-09 22:57:33,971 - lava_callback - DEBUG - Processing node 69acad5346f7ae8ff3367517: baseline-arm-clabbe 2026-03-09 22:57:33,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acad5346f7ae8ff3367517 2026-03-09 22:57:33,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:57:33,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad5346f7ae8ff3367517 HTTP/1.1" 200 None 2026-03-09 22:57:33,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:57:34,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 22:57:34,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:57:34,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acad5346f7ae8ff3367517 HTTP/1.1" 200 None 2026-03-09 22:57:34,023 - lava_callback - ERROR - Error processing callback for node 69acad5346f7ae8ff3367517: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56426 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:58:24,490 - lava_callback - INFO - Received callback for node 69acad8146f7ae8ff336752c INFO: 172.21.0.1:56406 - "POST /node/69acad8146f7ae8ff336752c HTTP/1.0" 202 Accepted 2026-03-09 22:58:24,496 - lava_callback - INFO - Starting async job submit for node 69acad8146f7ae8ff336752c 2026-03-09 22:58:24,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:58:24,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad8146f7ae8ff336752c HTTP/1.1" 200 None 2026-03-09 22:58:24,511 - lava_callback - DEBUG - Processing node 69acad8146f7ae8ff336752c: baseline-arm64-baylibre 2026-03-09 22:58:24,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acad8146f7ae8ff336752c 2026-03-09 22:58:24,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:58:24,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad8146f7ae8ff336752c HTTP/1.1" 200 None 2026-03-09 22:58:24,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:58:24,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:58:24,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:58:24,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acad8146f7ae8ff336752c HTTP/1.1" 200 None 2026-03-09 22:58:24,582 - lava_callback - ERROR - Error processing callback for node 69acad8146f7ae8ff336752c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56304 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:59:34,067 - lava_callback - INFO - Received callback for node 69acadc646f7ae8ff3367550 INFO: 172.21.0.1:59832 - "POST /node/69acadc646f7ae8ff3367550 HTTP/1.0" 202 Accepted 2026-03-09 22:59:34,073 - lava_callback - INFO - Starting async job submit for node 69acadc646f7ae8ff3367550 2026-03-09 22:59:34,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:59:34,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acadc646f7ae8ff3367550 HTTP/1.1" 200 None 2026-03-09 22:59:34,089 - lava_callback - DEBUG - Processing node 69acadc646f7ae8ff3367550: baseline-arm-clabbe 2026-03-09 22:59:34,089 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acadc646f7ae8ff3367550 2026-03-09 22:59:34,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:59:34,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acadc646f7ae8ff3367550 HTTP/1.1" 200 None 2026-03-09 22:59:34,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:59:34,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 22:59:34,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:59:34,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acadc646f7ae8ff3367550 HTTP/1.1" 200 None 2026-03-09 22:59:34,148 - lava_callback - ERROR - Error processing callback for node 69acadc646f7ae8ff3367550: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44574 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:00:14,272 - lava_callback - INFO - Received callback for node 69acadeb46f7ae8ff3367569 INFO: 172.21.0.1:41492 - "POST /node/69acadeb46f7ae8ff3367569 HTTP/1.0" 202 Accepted 2026-03-09 23:00:14,277 - lava_callback - INFO - Starting async job submit for node 69acadeb46f7ae8ff3367569 2026-03-09 23:00:14,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:00:14,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acadeb46f7ae8ff3367569 HTTP/1.1" 200 None 2026-03-09 23:00:14,293 - lava_callback - DEBUG - Processing node 69acadeb46f7ae8ff3367569: baseline-arm-clabbe 2026-03-09 23:00:14,293 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acadeb46f7ae8ff3367569 2026-03-09 23:00:14,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:00:14,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acadeb46f7ae8ff3367569 HTTP/1.1" 200 None 2026-03-09 23:00:14,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:00:14,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 23:00:14,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:00:14,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acadeb46f7ae8ff3367569 HTTP/1.1" 200 None 2026-03-09 23:00:14,357 - lava_callback - ERROR - Error processing callback for node 69acadeb46f7ae8ff3367569: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45590 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:06:34,318 - lava_callback - INFO - Received callback for node 69acaf7546f7ae8ff336760d INFO: 172.21.0.1:54020 - "POST /node/69acaf7546f7ae8ff336760d HTTP/1.0" 202 Accepted 2026-03-09 23:06:34,323 - lava_callback - INFO - Starting async job submit for node 69acaf7546f7ae8ff336760d 2026-03-09 23:06:34,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:06:34,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acaf7546f7ae8ff336760d HTTP/1.1" 200 None 2026-03-09 23:06:34,351 - lava_callback - DEBUG - Processing node 69acaf7546f7ae8ff336760d: baseline-arm-clabbe 2026-03-09 23:06:34,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acaf7546f7ae8ff336760d 2026-03-09 23:06:34,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:06:34,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acaf7546f7ae8ff336760d HTTP/1.1" 200 None 2026-03-09 23:06:34,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:06:34,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 23:06:34,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:06:34,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acaf7546f7ae8ff336760d HTTP/1.1" 200 None 2026-03-09 23:06:34,432 - lava_callback - ERROR - Error processing callback for node 69acaf7546f7ae8ff336760d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40722 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:14:05,722 - lava_callback - INFO - Received callback for node 69acb12c46f7ae8ff33676ab INFO: 172.21.0.1:40726 - "POST /node/69acb12c46f7ae8ff33676ab HTTP/1.0" 202 Accepted 2026-03-09 23:14:05,728 - lava_callback - INFO - Starting async job submit for node 69acb12c46f7ae8ff33676ab 2026-03-09 23:14:05,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:14:05,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb12c46f7ae8ff33676ab HTTP/1.1" 200 None 2026-03-09 23:14:05,748 - lava_callback - DEBUG - Processing node 69acb12c46f7ae8ff33676ab: baseline-arm64-baylibre 2026-03-09 23:14:05,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb12c46f7ae8ff33676ab 2026-03-09 23:14:05,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:14:05,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb12c46f7ae8ff33676ab HTTP/1.1" 200 None 2026-03-09 23:14:05,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:14:05,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 23:14:05,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:14:05,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb12c46f7ae8ff33676ab HTTP/1.1" 200 None 2026-03-09 23:14:05,804 - lava_callback - ERROR - Error processing callback for node 69acb12c46f7ae8ff33676ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45452 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:15:34,633 - lava_callback - INFO - Received callback for node 69acb18046f7ae8ff33676ca INFO: 172.21.0.1:51168 - "POST /node/69acb18046f7ae8ff33676ca HTTP/1.0" 202 Accepted 2026-03-09 23:15:34,639 - lava_callback - INFO - Starting async job submit for node 69acb18046f7ae8ff33676ca 2026-03-09 23:15:34,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:15:34,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb18046f7ae8ff33676ca HTTP/1.1" 200 None 2026-03-09 23:15:34,655 - lava_callback - DEBUG - Processing node 69acb18046f7ae8ff33676ca: baseline-arm64-clabbe 2026-03-09 23:15:34,655 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb18046f7ae8ff33676ca 2026-03-09 23:15:34,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:15:34,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb18046f7ae8ff33676ca HTTP/1.1" 200 None 2026-03-09 23:15:34,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:15:34,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 23:15:34,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:15:34,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb18046f7ae8ff33676ca HTTP/1.1" 200 None 2026-03-09 23:15:34,714 - lava_callback - ERROR - Error processing callback for node 69acb18046f7ae8ff33676ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43878 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:16:54,660 - lava_callback - INFO - Received callback for node 69acb1d146f7ae8ff33676ee INFO: 172.21.0.1:34778 - "POST /node/69acb1d146f7ae8ff33676ee HTTP/1.0" 202 Accepted 2026-03-09 23:16:54,666 - lava_callback - INFO - Starting async job submit for node 69acb1d146f7ae8ff33676ee 2026-03-09 23:16:54,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:16:54,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb1d146f7ae8ff33676ee HTTP/1.1" 200 None 2026-03-09 23:16:54,683 - lava_callback - DEBUG - Processing node 69acb1d146f7ae8ff33676ee: baseline-arm64-clabbe 2026-03-09 23:16:54,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb1d146f7ae8ff33676ee 2026-03-09 23:16:54,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:16:54,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb1d146f7ae8ff33676ee HTTP/1.1" 200 None 2026-03-09 23:16:54,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:16:54,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 23:16:54,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:16:54,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb1d146f7ae8ff33676ee HTTP/1.1" 200 None 2026-03-09 23:16:54,741 - lava_callback - ERROR - Error processing callback for node 69acb1d146f7ae8ff33676ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43350 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:17:54,709 - lava_callback - INFO - Received callback for node 69acb21246f7ae8ff336770a INFO: 172.21.0.1:40012 - "POST /node/69acb21246f7ae8ff336770a HTTP/1.0" 202 Accepted 2026-03-09 23:17:54,714 - lava_callback - INFO - Starting async job submit for node 69acb21246f7ae8ff336770a 2026-03-09 23:17:54,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:17:54,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb21246f7ae8ff336770a HTTP/1.1" 200 None 2026-03-09 23:17:54,738 - lava_callback - DEBUG - Processing node 69acb21246f7ae8ff336770a: baseline-arm-clabbe 2026-03-09 23:17:54,738 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb21246f7ae8ff336770a 2026-03-09 23:17:54,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:17:54,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb21246f7ae8ff336770a HTTP/1.1" 200 None 2026-03-09 23:17:54,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:17:54,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 23:17:54,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:17:54,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb21246f7ae8ff336770a HTTP/1.1" 200 None 2026-03-09 23:17:54,815 - lava_callback - ERROR - Error processing callback for node 69acb21246f7ae8ff336770a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52430 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:18:34,681 - lava_callback - INFO - Received callback for node 69acb23646f7ae8ff336771c INFO: 172.21.0.1:57624 - "POST /node/69acb23646f7ae8ff336771c HTTP/1.0" 202 Accepted 2026-03-09 23:18:34,686 - lava_callback - INFO - Starting async job submit for node 69acb23646f7ae8ff336771c 2026-03-09 23:18:34,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:18:34,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb23646f7ae8ff336771c HTTP/1.1" 200 None 2026-03-09 23:18:34,702 - lava_callback - DEBUG - Processing node 69acb23646f7ae8ff336771c: baseline-arm-clabbe 2026-03-09 23:18:34,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb23646f7ae8ff336771c 2026-03-09 23:18:34,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:18:34,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb23646f7ae8ff336771c HTTP/1.1" 200 None 2026-03-09 23:18:34,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:18:34,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 23:18:34,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:18:34,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb23646f7ae8ff336771c HTTP/1.1" 200 None 2026-03-09 23:18:34,797 - lava_callback - ERROR - Error processing callback for node 69acb23646f7ae8ff336771c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39344 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:23:06,017 - lava_callback - INFO - Received callback for node 69acb35446f7ae8ff3367793 INFO: 172.21.0.1:39352 - "POST /node/69acb35446f7ae8ff3367793 HTTP/1.0" 202 Accepted 2026-03-09 23:23:06,023 - lava_callback - INFO - Starting async job submit for node 69acb35446f7ae8ff3367793 2026-03-09 23:23:06,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:23:06,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb35446f7ae8ff3367793 HTTP/1.1" 200 None 2026-03-09 23:23:06,039 - lava_callback - DEBUG - Processing node 69acb35446f7ae8ff3367793: baseline-arm64-baylibre 2026-03-09 23:23:06,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb35446f7ae8ff3367793 2026-03-09 23:23:06,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:23:06,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb35446f7ae8ff3367793 HTTP/1.1" 200 None 2026-03-09 23:23:06,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:23:06,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 23:23:06,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:23:06,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb35446f7ae8ff3367793 HTTP/1.1" 200 None 2026-03-09 23:23:06,127 - lava_callback - ERROR - Error processing callback for node 69acb35446f7ae8ff3367793: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39114 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:28:35,213 - lava_callback - INFO - Received callback for node 69acb49a46f7ae8ff33677e2 INFO: 172.21.0.1:39592 - "POST /node/69acb49a46f7ae8ff33677e2 HTTP/1.0" 202 Accepted 2026-03-09 23:28:35,218 - lava_callback - INFO - Starting async job submit for node 69acb49a46f7ae8ff33677e2 2026-03-09 23:28:35,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:28:35,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb49a46f7ae8ff33677e2 HTTP/1.1" 200 None 2026-03-09 23:28:35,235 - lava_callback - DEBUG - Processing node 69acb49a46f7ae8ff33677e2: baseline-arm64-clabbe 2026-03-09 23:28:35,236 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb49a46f7ae8ff33677e2 2026-03-09 23:28:35,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:28:35,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb49a46f7ae8ff33677e2 HTTP/1.1" 200 None 2026-03-09 23:28:35,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:28:35,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 23:28:35,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:28:35,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb49a46f7ae8ff33677e2 HTTP/1.1" 200 None 2026-03-09 23:28:35,300 - lava_callback - ERROR - Error processing callback for node 69acb49a46f7ae8ff33677e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60058 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:29:55,299 - lava_callback - INFO - Received callback for node 69acb4ed46f7ae8ff3367801 INFO: 172.21.0.1:35084 - "POST /node/69acb4ed46f7ae8ff3367801 HTTP/1.0" 202 Accepted 2026-03-09 23:29:55,305 - lava_callback - INFO - Starting async job submit for node 69acb4ed46f7ae8ff3367801 2026-03-09 23:29:55,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:29:55,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb4ed46f7ae8ff3367801 HTTP/1.1" 200 None 2026-03-09 23:29:55,321 - lava_callback - DEBUG - Processing node 69acb4ed46f7ae8ff3367801: baseline-arm64-clabbe 2026-03-09 23:29:55,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb4ed46f7ae8ff3367801 2026-03-09 23:29:55,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:29:55,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb4ed46f7ae8ff3367801 HTTP/1.1" 200 None 2026-03-09 23:29:55,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:29:55,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 23:29:55,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:29:55,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb4ed46f7ae8ff3367801 HTTP/1.1" 200 None 2026-03-09 23:29:55,403 - lava_callback - ERROR - Error processing callback for node 69acb4ed46f7ae8ff3367801: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34034 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1] INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:52474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57502 - "GET /node/699448525ae459b9df3dad23%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:47284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42322 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:39:43,781 - lava_callback - INFO - Received callback for node 69ac54c346f7ae8ff33651d9 2026-03-09 16:39:43,787 - lava_callback - INFO - Starting async job submit for node 69ac54c346f7ae8ff33651d9 INFO: 172.21.0.1:36950 - "POST /node/69ac54c346f7ae8ff33651d9 HTTP/1.0" 202 Accepted 2026-03-09 16:39:43,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:39:43,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac54c346f7ae8ff33651d9 HTTP/1.1" 200 None 2026-03-09 16:39:43,810 - lava_callback - DEBUG - Processing node 69ac54c346f7ae8ff33651d9: baseline-arm-baylibre 2026-03-09 16:39:43,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac54c346f7ae8ff33651d9 2026-03-09 16:39:43,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:39:43,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac54c346f7ae8ff33651d9 HTTP/1.1" 200 None 2026-03-09 16:39:43,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:39:43,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a4fe31b554803095c08 HTTP/1.1" 200 None 2026-03-09 16:39:43,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:39:43,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac54c346f7ae8ff33651d9 HTTP/1.1" 200 None 2026-03-09 16:39:43,864 - lava_callback - ERROR - Error processing callback for node 69ac54c346f7ae8ff33651d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42092 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:40:03,292 - lava_callback - INFO - Received callback for node 69aef766ce46d8ec186abbfa INFO: 172.21.0.1:52294 - "POST /node/69aef766ce46d8ec186abbfa HTTP/1.0" 202 Accepted 2026-03-09 16:40:03,299 - lava_callback - INFO - Starting async job submit for node 69aef766ce46d8ec186abbfa 2026-03-09 16:40:03,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:03,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef766ce46d8ec186abbfa HTTP/1.1" 200 None 2026-03-09 16:40:03,353 - lava_callback - DEBUG - Processing node 69aef766ce46d8ec186abbfa: baseline-arm 2026-03-09 16:40:03,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:40:03,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:40:03,700 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aef766ce46d8ec186abbfa/log.txt.gz 2026-03-09 16:40:03,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:40:04,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:40:04,017 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef766ce46d8ec186abbfa/lava-logs.yaml 2026-03-09 16:40:04,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:40:04,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:40:04,200 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef766ce46d8ec186abbfa/lava_callback.json.gz 2026-03-09 16:40:04,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:04,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef766ce46d8ec186abbfa HTTP/1.1" 200 None 2026-03-09 16:40:04,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:04,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef42ace46d8ec186ab814 HTTP/1.1" 200 None 2026-03-09 16:40:04,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:04,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef766ce46d8ec186abbfa HTTP/1.1" 200 None 2026-03-09 16:40:04,292 - lava_callback - INFO - Completed processing callback for node 69aef766ce46d8ec186abbfa INFO: 172.21.0.1:52306 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:40:13,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:13,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42054 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:40:23,752 - lava_callback - INFO - Received callback for node 69ac54e446f7ae8ff33651e7 INFO: 172.21.0.1:33144 - "POST /node/69ac54e446f7ae8ff33651e7 HTTP/1.0" 202 Accepted 2026-03-09 16:40:23,757 - lava_callback - INFO - Starting async job submit for node 69ac54e446f7ae8ff33651e7 2026-03-09 16:40:23,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:23,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac54e446f7ae8ff33651e7 HTTP/1.1" 200 None 2026-03-09 16:40:23,774 - lava_callback - DEBUG - Processing node 69ac54e446f7ae8ff33651e7: baseline-arm-baylibre 2026-03-09 16:40:23,774 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac54e446f7ae8ff33651e7 2026-03-09 16:40:23,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:23,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac54e446f7ae8ff33651e7 HTTP/1.1" 200 None 2026-03-09 16:40:23,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:23,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a4fe31b554803095c08 HTTP/1.1" 200 None 2026-03-09 16:40:23,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:23,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac54e446f7ae8ff33651e7 HTTP/1.1" 200 None 2026-03-09 16:40:23,845 - lava_callback - ERROR - Error processing callback for node 69ac54e446f7ae8ff33651e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54248 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:40:43,777 - lava_callback - INFO - Received callback for node 69ac550846f7ae8ff33651f6 INFO: 172.21.0.1:41056 - "POST /node/69ac550846f7ae8ff33651f6 HTTP/1.0" 202 Accepted 2026-03-09 16:40:43,783 - lava_callback - INFO - Starting async job submit for node 69ac550846f7ae8ff33651f6 2026-03-09 16:40:43,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:43,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac550846f7ae8ff33651f6 HTTP/1.1" 200 None 2026-03-09 16:40:43,799 - lava_callback - DEBUG - Processing node 69ac550846f7ae8ff33651f6: baseline-arm-baylibre 2026-03-09 16:40:43,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac550846f7ae8ff33651f6 2026-03-09 16:40:43,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:43,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac550846f7ae8ff33651f6 HTTP/1.1" 200 None 2026-03-09 16:40:43,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:43,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a4fe31b554803095c08 HTTP/1.1" 200 None 2026-03-09 16:40:43,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:40:43,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac550846f7ae8ff33651f6 HTTP/1.1" 200 None 2026-03-09 16:40:43,863 - lava_callback - ERROR - Error processing callback for node 69ac550846f7ae8ff33651f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35750 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:41:23,876 - lava_callback - INFO - Received callback for node 69ac552c46f7ae8ff3365211 INFO: 172.21.0.1:54994 - "POST /node/69ac552c46f7ae8ff3365211 HTTP/1.0" 202 Accepted 2026-03-09 16:41:23,881 - lava_callback - INFO - Starting async job submit for node 69ac552c46f7ae8ff3365211 2026-03-09 16:41:23,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:41:23,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac552c46f7ae8ff3365211 HTTP/1.1" 200 None 2026-03-09 16:41:23,915 - lava_callback - DEBUG - Processing node 69ac552c46f7ae8ff3365211: baseline-arm-baylibre 2026-03-09 16:41:23,916 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac552c46f7ae8ff3365211 2026-03-09 16:41:23,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:41:23,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac552c46f7ae8ff3365211 HTTP/1.1" 200 None 2026-03-09 16:41:23,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:41:23,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:41:23,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:41:23,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac552c46f7ae8ff3365211 HTTP/1.1" 200 None 2026-03-09 16:41:23,990 - lava_callback - ERROR - Error processing callback for node 69ac552c46f7ae8ff3365211: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56146 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:42:03,826 - lava_callback - INFO - Received callback for node 69ac555146f7ae8ff3365230 INFO: 172.21.0.1:39730 - "POST /node/69ac555146f7ae8ff3365230 HTTP/1.0" 202 Accepted 2026-03-09 16:42:03,835 - lava_callback - INFO - Starting async job submit for node 69ac555146f7ae8ff3365230 2026-03-09 16:42:03,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:03,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac555146f7ae8ff3365230 HTTP/1.1" 200 None 2026-03-09 16:42:03,867 - lava_callback - DEBUG - Processing node 69ac555146f7ae8ff3365230: baseline-arm-baylibre 2026-03-09 16:42:03,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac555146f7ae8ff3365230 2026-03-09 16:42:03,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:03,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac555146f7ae8ff3365230 HTTP/1.1" 200 None 2026-03-09 16:42:03,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:03,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:42:03,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:03,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac555146f7ae8ff3365230 HTTP/1.1" 200 None 2026-03-09 16:42:03,986 - lava_callback - ERROR - Error processing callback for node 69ac555146f7ae8ff3365230: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60968 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:42:44,171 - lava_callback - INFO - Received callback for node 69ac557746f7ae8ff3365244 INFO: 172.21.0.1:43144 - "POST /node/69ac557746f7ae8ff3365244 HTTP/1.0" 202 Accepted 2026-03-09 16:42:44,176 - lava_callback - INFO - Starting async job submit for node 69ac557746f7ae8ff3365244 2026-03-09 16:42:44,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:44,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac557746f7ae8ff3365244 HTTP/1.1" 200 None 2026-03-09 16:42:44,192 - lava_callback - DEBUG - Processing node 69ac557746f7ae8ff3365244: baseline-arm-baylibre 2026-03-09 16:42:44,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac557746f7ae8ff3365244 2026-03-09 16:42:44,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:44,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac557746f7ae8ff3365244 HTTP/1.1" 200 None 2026-03-09 16:42:44,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:44,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:42:44,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:42:44,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac557746f7ae8ff3365244 HTTP/1.1" 200 None 2026-03-09 16:42:44,245 - lava_callback - ERROR - Error processing callback for node 69ac557746f7ae8ff3365244: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45982 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:43:03,918 - lava_callback - INFO - Received callback for node 69ac558a46f7ae8ff3365285 INFO: 172.21.0.1:41616 - "POST /node/69ac558a46f7ae8ff3365285 HTTP/1.0" 202 Accepted 2026-03-09 16:43:03,924 - lava_callback - INFO - Starting async job submit for node 69ac558a46f7ae8ff3365285 2026-03-09 16:43:03,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:03,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558a46f7ae8ff3365285 HTTP/1.1" 200 None 2026-03-09 16:43:03,941 - lava_callback - DEBUG - Processing node 69ac558a46f7ae8ff3365285: baseline-arm-baylibre 2026-03-09 16:43:03,941 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558a46f7ae8ff3365285 2026-03-09 16:43:03,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:03,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558a46f7ae8ff3365285 HTTP/1.1" 200 None 2026-03-09 16:43:03,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:03,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:03,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:03,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558a46f7ae8ff3365285 HTTP/1.1" 200 None 2026-03-09 16:43:03,999 - lava_callback - ERROR - Error processing callback for node 69ac558a46f7ae8ff3365285: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:04,576 - lava_callback - INFO - Received callback for node 69ac558d46f7ae8ff3365294 INFO: 172.21.0.1:41628 - "POST /node/69ac558d46f7ae8ff3365294 HTTP/1.0" 202 Accepted 2026-03-09 16:43:04,582 - lava_callback - INFO - Starting async job submit for node 69ac558d46f7ae8ff3365294 2026-03-09 16:43:04,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:04,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558d46f7ae8ff3365294 HTTP/1.1" 200 None 2026-03-09 16:43:04,598 - lava_callback - DEBUG - Processing node 69ac558d46f7ae8ff3365294: baseline-arm-baylibre 2026-03-09 16:43:04,598 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558d46f7ae8ff3365294 2026-03-09 16:43:04,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:04,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558d46f7ae8ff3365294 HTTP/1.1" 200 None 2026-03-09 16:43:04,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:04,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:04,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:04,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558d46f7ae8ff3365294 HTTP/1.1" 200 None 2026-03-09 16:43:04,651 - lava_callback - ERROR - Error processing callback for node 69ac558d46f7ae8ff3365294: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:05,261 - lava_callback - INFO - Received callback for node 69ac558746f7ae8ff336526a INFO: 172.21.0.1:41632 - "POST /node/69ac558746f7ae8ff336526a HTTP/1.0" 202 Accepted 2026-03-09 16:43:05,269 - lava_callback - INFO - Starting async job submit for node 69ac558746f7ae8ff336526a 2026-03-09 16:43:05,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558746f7ae8ff336526a HTTP/1.1" 200 None 2026-03-09 16:43:05,290 - lava_callback - DEBUG - Processing node 69ac558746f7ae8ff336526a: baseline-arm-baylibre 2026-03-09 16:43:05,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558746f7ae8ff336526a 2026-03-09 16:43:05,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558746f7ae8ff336526a HTTP/1.1" 200 None 2026-03-09 16:43:05,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:05,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558746f7ae8ff336526a HTTP/1.1" 200 None 2026-03-09 16:43:05,346 - lava_callback - ERROR - Error processing callback for node 69ac558746f7ae8ff336526a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:05,925 - lava_callback - INFO - Received callback for node 69ac558846f7ae8ff336526d INFO: 172.21.0.1:41646 - "POST /node/69ac558846f7ae8ff336526d HTTP/1.0" 202 Accepted 2026-03-09 16:43:05,931 - lava_callback - INFO - Starting async job submit for node 69ac558846f7ae8ff336526d 2026-03-09 16:43:05,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558846f7ae8ff336526d HTTP/1.1" 200 None 2026-03-09 16:43:05,956 - lava_callback - DEBUG - Processing node 69ac558846f7ae8ff336526d: baseline-arm-baylibre 2026-03-09 16:43:05,956 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558846f7ae8ff336526d 2026-03-09 16:43:05,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558846f7ae8ff336526d HTTP/1.1" 200 None 2026-03-09 16:43:05,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:05,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:05,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:06,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558846f7ae8ff336526d HTTP/1.1" 200 None 2026-03-09 16:43:06,014 - lava_callback - ERROR - Error processing callback for node 69ac558846f7ae8ff336526d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41662 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:43:06,612 - lava_callback - INFO - Received callback for node 69ac558b46f7ae8ff3365288 INFO: 172.21.0.1:41678 - "POST /node/69ac558b46f7ae8ff3365288 HTTP/1.0" 202 Accepted 2026-03-09 16:43:06,617 - lava_callback - INFO - Starting async job submit for node 69ac558b46f7ae8ff3365288 2026-03-09 16:43:06,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:06,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558b46f7ae8ff3365288 HTTP/1.1" 200 None 2026-03-09 16:43:06,635 - lava_callback - DEBUG - Processing node 69ac558b46f7ae8ff3365288: baseline-arm-baylibre 2026-03-09 16:43:06,635 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558b46f7ae8ff3365288 2026-03-09 16:43:06,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:06,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558b46f7ae8ff3365288 HTTP/1.1" 200 None 2026-03-09 16:43:06,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:06,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:06,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:06,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558b46f7ae8ff3365288 HTTP/1.1" 200 None 2026-03-09 16:43:06,688 - lava_callback - ERROR - Error processing callback for node 69ac558b46f7ae8ff3365288: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:07,296 - lava_callback - INFO - Received callback for node 69ac558f46f7ae8ff3365298 INFO: 172.21.0.1:41694 - "POST /node/69ac558f46f7ae8ff3365298 HTTP/1.0" 202 Accepted 2026-03-09 16:43:07,301 - lava_callback - INFO - Starting async job submit for node 69ac558f46f7ae8ff3365298 2026-03-09 16:43:07,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:07,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558f46f7ae8ff3365298 HTTP/1.1" 200 None 2026-03-09 16:43:07,325 - lava_callback - DEBUG - Processing node 69ac558f46f7ae8ff3365298: baseline-arm-baylibre 2026-03-09 16:43:07,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac558f46f7ae8ff3365298 2026-03-09 16:43:07,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:07,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac558f46f7ae8ff3365298 HTTP/1.1" 200 None 2026-03-09 16:43:07,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:07,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:07,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:07,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac558f46f7ae8ff3365298 HTTP/1.1" 200 None 2026-03-09 16:43:07,381 - lava_callback - ERROR - Error processing callback for node 69ac558f46f7ae8ff3365298: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40362 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:43:17,380 - lava_callback - INFO - Received callback for node 69ac559b46f7ae8ff33652bf INFO: 172.21.0.1:40368 - "POST /node/69ac559b46f7ae8ff33652bf HTTP/1.0" 202 Accepted 2026-03-09 16:43:17,385 - lava_callback - INFO - Starting async job submit for node 69ac559b46f7ae8ff33652bf 2026-03-09 16:43:17,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:17,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559b46f7ae8ff33652bf HTTP/1.1" 200 None 2026-03-09 16:43:17,401 - lava_callback - DEBUG - Processing node 69ac559b46f7ae8ff33652bf: baseline-arm-clabbe 2026-03-09 16:43:17,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac559b46f7ae8ff33652bf 2026-03-09 16:43:17,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:17,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559b46f7ae8ff33652bf HTTP/1.1" 200 None 2026-03-09 16:43:17,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:17,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:17,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:17,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac559b46f7ae8ff33652bf HTTP/1.1" 200 None 2026-03-09 16:43:17,461 - lava_callback - ERROR - Error processing callback for node 69ac559b46f7ae8ff33652bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:18,047 - lava_callback - INFO - Received callback for node 69ac559c46f7ae8ff33652c4 INFO: 172.21.0.1:40382 - "POST /node/69ac559c46f7ae8ff33652c4 HTTP/1.0" 202 Accepted 2026-03-09 16:43:18,052 - lava_callback - INFO - Starting async job submit for node 69ac559c46f7ae8ff33652c4 2026-03-09 16:43:18,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559c46f7ae8ff33652c4 HTTP/1.1" 200 None 2026-03-09 16:43:18,073 - lava_callback - DEBUG - Processing node 69ac559c46f7ae8ff33652c4: baseline-arm-clabbe 2026-03-09 16:43:18,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac559c46f7ae8ff33652c4 2026-03-09 16:43:18,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559c46f7ae8ff33652c4 HTTP/1.1" 200 None 2026-03-09 16:43:18,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:18,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac559c46f7ae8ff33652c4 HTTP/1.1" 200 None 2026-03-09 16:43:18,133 - lava_callback - ERROR - Error processing callback for node 69ac559c46f7ae8ff33652c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:18,707 - lava_callback - INFO - Received callback for node 69ac55a046f7ae8ff33652d1 INFO: 172.21.0.1:40384 - "POST /node/69ac55a046f7ae8ff33652d1 HTTP/1.0" 202 Accepted 2026-03-09 16:43:18,712 - lava_callback - INFO - Starting async job submit for node 69ac55a046f7ae8ff33652d1 2026-03-09 16:43:18,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55a046f7ae8ff33652d1 HTTP/1.1" 200 None 2026-03-09 16:43:18,734 - lava_callback - DEBUG - Processing node 69ac55a046f7ae8ff33652d1: baseline-arm-clabbe 2026-03-09 16:43:18,734 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac55a046f7ae8ff33652d1 2026-03-09 16:43:18,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55a046f7ae8ff33652d1 HTTP/1.1" 200 None 2026-03-09 16:43:18,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 16:43:18,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:18,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac55a046f7ae8ff33652d1 HTTP/1.1" 200 None 2026-03-09 16:43:18,809 - lava_callback - ERROR - Error processing callback for node 69ac55a046f7ae8ff33652d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:43:23,891 - lava_callback - INFO - Received callback for node 69ac559b46f7ae8ff33652c1 INFO: 172.21.0.1:35426 - "POST /node/69ac559b46f7ae8ff33652c1 HTTP/1.0" 202 Accepted 2026-03-09 16:43:23,897 - lava_callback - INFO - Starting async job submit for node 69ac559b46f7ae8ff33652c1 2026-03-09 16:43:23,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:23,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559b46f7ae8ff33652c1 HTTP/1.1" 200 None 2026-03-09 16:43:23,913 - lava_callback - DEBUG - Processing node 69ac559b46f7ae8ff33652c1: baseline-arm-baylibre 2026-03-09 16:43:23,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac559b46f7ae8ff33652c1 2026-03-09 16:43:23,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:23,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac559b46f7ae8ff33652c1 HTTP/1.1" 200 None 2026-03-09 16:43:23,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:23,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:43:23,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:23,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac559b46f7ae8ff33652c1 HTTP/1.1" 200 None 2026-03-09 16:43:23,965 - lava_callback - ERROR - Error processing callback for node 69ac559b46f7ae8ff33652c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43238 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:43:57,367 - lava_callback - INFO - Received callback for node 69ac55c146f7ae8ff336532f INFO: 172.21.0.1:43252 - "POST /node/69ac55c146f7ae8ff336532f HTTP/1.0" 202 Accepted 2026-03-09 16:43:57,373 - lava_callback - INFO - Starting async job submit for node 69ac55c146f7ae8ff336532f 2026-03-09 16:43:57,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:57,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55c146f7ae8ff336532f HTTP/1.1" 200 None 2026-03-09 16:43:57,389 - lava_callback - DEBUG - Processing node 69ac55c146f7ae8ff336532f: baseline-arm-clabbe 2026-03-09 16:43:57,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac55c146f7ae8ff336532f 2026-03-09 16:43:57,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:57,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55c146f7ae8ff336532f HTTP/1.1" 200 None 2026-03-09 16:43:57,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:57,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:43:57,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:43:57,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac55c146f7ae8ff336532f HTTP/1.1" 200 None 2026-03-09 16:43:57,458 - lava_callback - ERROR - Error processing callback for node 69ac55c146f7ae8ff336532f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54246 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:44:37,412 - lava_callback - INFO - Received callback for node 69ac55e946f7ae8ff3365397 INFO: 172.21.0.1:54250 - "POST /node/69ac55e946f7ae8ff3365397 HTTP/1.0" 202 Accepted 2026-03-09 16:44:37,417 - lava_callback - INFO - Starting async job submit for node 69ac55e946f7ae8ff3365397 2026-03-09 16:44:37,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:44:37,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55e946f7ae8ff3365397 HTTP/1.1" 200 None 2026-03-09 16:44:37,434 - lava_callback - DEBUG - Processing node 69ac55e946f7ae8ff3365397: baseline-arm-clabbe 2026-03-09 16:44:37,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac55e946f7ae8ff3365397 2026-03-09 16:44:37,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:44:37,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac55e946f7ae8ff3365397 HTTP/1.1" 200 None 2026-03-09 16:44:37,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:44:37,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:44:37,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:44:37,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac55e946f7ae8ff3365397 HTTP/1.1" 200 None 2026-03-09 16:44:37,503 - lava_callback - ERROR - Error processing callback for node 69ac55e946f7ae8ff3365397: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57904 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:45:17,372 - lava_callback - INFO - Received callback for node 69ac561046f7ae8ff33653d9 INFO: 172.21.0.1:57908 - "POST /node/69ac561046f7ae8ff33653d9 HTTP/1.0" 202 Accepted 2026-03-09 16:45:17,377 - lava_callback - INFO - Starting async job submit for node 69ac561046f7ae8ff33653d9 2026-03-09 16:45:17,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:17,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac561046f7ae8ff33653d9 HTTP/1.1" 200 None 2026-03-09 16:45:17,393 - lava_callback - DEBUG - Processing node 69ac561046f7ae8ff33653d9: baseline-arm-clabbe 2026-03-09 16:45:17,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac561046f7ae8ff33653d9 2026-03-09 16:45:17,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:17,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac561046f7ae8ff33653d9 HTTP/1.1" 200 None 2026-03-09 16:45:17,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:17,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 16:45:17,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:17,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac561046f7ae8ff33653d9 HTTP/1.1" 200 None 2026-03-09 16:45:17,444 - lava_callback - ERROR - Error processing callback for node 69ac561046f7ae8ff33653d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52878 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:45:44,179 - lava_callback - INFO - Received callback for node 69ac563446f7ae8ff33653fa INFO: 172.21.0.1:38932 - "POST /node/69ac563446f7ae8ff33653fa HTTP/1.0" 202 Accepted 2026-03-09 16:45:44,185 - lava_callback - INFO - Starting async job submit for node 69ac563446f7ae8ff33653fa 2026-03-09 16:45:44,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:44,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac563446f7ae8ff33653fa HTTP/1.1" 200 None 2026-03-09 16:45:44,201 - lava_callback - DEBUG - Processing node 69ac563446f7ae8ff33653fa: baseline-arm-baylibre 2026-03-09 16:45:44,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac563446f7ae8ff33653fa 2026-03-09 16:45:44,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:44,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac563446f7ae8ff33653fa HTTP/1.1" 200 None 2026-03-09 16:45:44,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:44,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a4fe31b554803095c08 HTTP/1.1" 200 None 2026-03-09 16:45:44,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:45:44,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac563446f7ae8ff33653fa HTTP/1.1" 200 None 2026-03-09 16:45:44,274 - lava_callback - ERROR - Error processing callback for node 69ac563446f7ae8ff33653fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48726 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:49:16,778 - lava_callback - INFO - Received callback for node 69aef9fdce46d8ec186abe8e INFO: 172.21.0.1:48736 - "POST /node/69aef9fdce46d8ec186abe8e HTTP/1.0" 202 Accepted 2026-03-09 16:49:16,784 - lava_callback - INFO - Starting async job submit for node 69aef9fdce46d8ec186abe8e 2026-03-09 16:49:16,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:16,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fdce46d8ec186abe8e HTTP/1.1" 200 None 2026-03-09 16:49:16,813 - lava_callback - DEBUG - Processing node 69aef9fdce46d8ec186abe8e: baseline-arm 2026-03-09 16:49:16,818 - lava_callback - WARNING - Failed to upload log 2026-03-09 16:49:16,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:16,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:16,986 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fdce46d8ec186abe8e/lava-logs.yaml 2026-03-09 16:49:16,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:17,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:17,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fdce46d8ec186abe8e/lava_callback.json.gz 2026-03-09 16:49:17,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:17,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fdce46d8ec186abe8e HTTP/1.1" 200 None 2026-03-09 16:49:17,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:17,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:49:17,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:17,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef9fdce46d8ec186abe8e HTTP/1.1" 200 None 2026-03-09 16:49:17,229 - lava_callback - INFO - Completed processing callback for node 69aef9fdce46d8ec186abe8e INFO: 172.21.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46698 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:49:43,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:43,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60146 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:49:51,816 - lava_callback - INFO - Received callback for node 69aefa00ce46d8ec186abec0 INFO: 172.21.0.1:41862 - "POST /node/69aefa00ce46d8ec186abec0 HTTP/1.0" 202 Accepted 2026-03-09 16:49:51,865 - lava_callback - INFO - Starting async job submit for node 69aefa00ce46d8ec186abec0 2026-03-09 16:49:51,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:51,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa00ce46d8ec186abec0 HTTP/1.1" 200 None 2026-03-09 16:49:51,898 - lava_callback - DEBUG - Processing node 69aefa00ce46d8ec186abec0: baseline-arm 2026-03-09 16:49:52,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:52,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:52,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aefa00ce46d8ec186abec0/log.txt.gz 2026-03-09 16:49:52,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:52,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:52,398 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aefa00ce46d8ec186abec0/lava-logs.yaml 2026-03-09 16:49:52,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:52,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:52,545 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aefa00ce46d8ec186abec0/lava_callback.json.gz 2026-03-09 16:49:52,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:52,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa00ce46d8ec186abec0 HTTP/1.1" 200 None 2026-03-09 16:49:52,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:52,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:49:52,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:52,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa00ce46d8ec186abec0 HTTP/1.1" 200 None 2026-03-09 16:49:52,638 - lava_callback - INFO - Completed processing callback for node 69aefa00ce46d8ec186abec0 2026-03-09 16:49:53,241 - lava_callback - INFO - Received callback for node 69aef9fdce46d8ec186abe91 INFO: 172.21.0.1:41866 - "POST /node/69aef9fdce46d8ec186abe91 HTTP/1.0" 202 Accepted 2026-03-09 16:49:53,290 - lava_callback - INFO - Starting async job submit for node 69aef9fdce46d8ec186abe91 2026-03-09 16:49:53,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:53,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fdce46d8ec186abe91 HTTP/1.1" 200 None 2026-03-09 16:49:53,323 - lava_callback - DEBUG - Processing node 69aef9fdce46d8ec186abe91: baseline-arm 2026-03-09 16:49:53,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:53,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:53,589 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fdce46d8ec186abe91/log.txt.gz 2026-03-09 16:49:53,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:53,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:53,833 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fdce46d8ec186abe91/lava-logs.yaml 2026-03-09 16:49:53,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:49:54,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:49:54,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fdce46d8ec186abe91/lava_callback.json.gz 2026-03-09 16:49:54,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:54,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fdce46d8ec186abe91 HTTP/1.1" 200 None 2026-03-09 16:49:54,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:54,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:49:54,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:49:54,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef9fdce46d8ec186abe91 HTTP/1.1" 200 None 2026-03-09 16:49:54,147 - lava_callback - INFO - Completed processing callback for node 69aef9fdce46d8ec186abe91 INFO: 172.21.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44748 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:50:13,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:13,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51754 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:50:26,987 - lava_callback - INFO - Received callback for node 69aef9ffce46d8ec186abebd INFO: 172.21.0.1:51770 - "POST /node/69aef9ffce46d8ec186abebd HTTP/1.0" 202 Accepted 2026-03-09 16:50:26,993 - lava_callback - INFO - Starting async job submit for node 69aef9ffce46d8ec186abebd 2026-03-09 16:50:27,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:27,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9ffce46d8ec186abebd HTTP/1.1" 200 None 2026-03-09 16:50:27,033 - lava_callback - DEBUG - Processing node 69aef9ffce46d8ec186abebd: baseline-arm 2026-03-09 16:50:27,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:27,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:27,414 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9ffce46d8ec186abebd/log.txt.gz 2026-03-09 16:50:27,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:27,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:27,765 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9ffce46d8ec186abebd/lava-logs.yaml 2026-03-09 16:50:27,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:27,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:27,918 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9ffce46d8ec186abebd/lava_callback.json.gz 2026-03-09 16:50:27,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:27,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9ffce46d8ec186abebd HTTP/1.1" 200 None 2026-03-09 16:50:27,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:27,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:27,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:28,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef9ffce46d8ec186abebd HTTP/1.1" 200 None 2026-03-09 16:50:28,006 - lava_callback - INFO - Completed processing callback for node 69aef9ffce46d8ec186abebd INFO: 172.21.0.1:52604 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:50:40,789 - lava_callback - INFO - Received callback for node 69aef9fcce46d8ec186abe8b INFO: 172.21.0.1:44378 - "POST /node/69aef9fcce46d8ec186abe8b HTTP/1.0" 202 Accepted 2026-03-09 16:50:40,795 - lava_callback - INFO - Starting async job submit for node 69aef9fcce46d8ec186abe8b 2026-03-09 16:50:40,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:40,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fcce46d8ec186abe8b HTTP/1.1" 200 None 2026-03-09 16:50:40,831 - lava_callback - DEBUG - Processing node 69aef9fcce46d8ec186abe8b: baseline-arm 2026-03-09 16:50:41,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:41,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:41,160 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fcce46d8ec186abe8b/log.txt.gz 2026-03-09 16:50:41,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:41,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:41,455 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fcce46d8ec186abe8b/lava-logs.yaml 2026-03-09 16:50:41,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:41,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:41,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fcce46d8ec186abe8b/lava_callback.json.gz 2026-03-09 16:50:41,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:41,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fcce46d8ec186abe8b HTTP/1.1" 200 None 2026-03-09 16:50:41,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:41,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:41,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:41,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef9fcce46d8ec186abe8b HTTP/1.1" 200 None 2026-03-09 16:50:41,693 - lava_callback - INFO - Completed processing callback for node 69aef9fcce46d8ec186abe8b 2026-03-09 16:50:41,811 - lava_callback - INFO - Received callback for node 69aefa0dce46d8ec186abedf INFO: 172.21.0.1:44388 - "POST /node/69aefa0dce46d8ec186abedf HTTP/1.0" 202 Accepted 2026-03-09 16:50:41,818 - lava_callback - INFO - Starting async job submit for node 69aefa0dce46d8ec186abedf 2026-03-09 16:50:41,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:41,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa0dce46d8ec186abedf HTTP/1.1" 200 None 2026-03-09 16:50:41,867 - lava_callback - DEBUG - Processing node 69aefa0dce46d8ec186abedf: baseline-arm-broonie 2026-03-09 16:50:42,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:42,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:42,247 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0dce46d8ec186abedf/log.txt.gz 2026-03-09 16:50:42,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:42,396 - lava_callback - INFO - Received callback for node 69aefa1dce46d8ec186abf53 2026-03-09 16:50:42,402 - lava_callback - INFO - Starting async job submit for node 69aefa1dce46d8ec186abf53 INFO: 172.21.0.1:44394 - "POST /node/69aefa1dce46d8ec186abf53 HTTP/1.0" 202 Accepted 2026-03-09 16:50:42,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:42,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1dce46d8ec186abf53 HTTP/1.1" 200 None 2026-03-09 16:50:42,454 - lava_callback - DEBUG - Processing node 69aefa1dce46d8ec186abf53: baseline-arm-pengutronix 2026-03-09 16:50:42,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:42,600 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0dce46d8ec186abedf/lava-logs.yaml 2026-03-09 16:50:42,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:42,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:42,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:42,723 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1dce46d8ec186abf53/log.txt.gz 2026-03-09 16:50:42,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:42,828 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0dce46d8ec186abedf/lava_callback.json.gz 2026-03-09 16:50:42,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:42,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:42,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa0dce46d8ec186abedf HTTP/1.1" 200 None 2026-03-09 16:50:42,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:42,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:42,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:42,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa0dce46d8ec186abedf HTTP/1.1" 200 None 2026-03-09 16:50:42,921 - lava_callback - INFO - Completed processing callback for node 69aefa0dce46d8ec186abedf 2026-03-09 16:50:43,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:43,059 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1dce46d8ec186abf53/lava-logs.yaml 2026-03-09 16:50:43,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:43,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:43,215 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1dce46d8ec186abf53/lava_callback.json.gz 2026-03-09 16:50:43,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:43,234 - lava_callback - INFO - Received callback for node 69aefa0ece46d8ec186abee5 INFO: 172.21.0.1:44396 - "POST /node/69aefa0ece46d8ec186abee5 HTTP/1.0" 202 Accepted 2026-03-09 16:50:43,240 - lava_callback - INFO - Starting async job submit for node 69aefa0ece46d8ec186abee5 2026-03-09 16:50:43,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:43,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa0ece46d8ec186abee5 HTTP/1.1" 200 None 2026-03-09 16:50:43,282 - lava_callback - DEBUG - Processing node 69aefa0ece46d8ec186abee5: baseline-arm-broonie 2026-03-09 16:50:43,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1dce46d8ec186abf53 HTTP/1.1" 200 None 2026-03-09 16:50:43,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:43,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:43,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:43,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa1dce46d8ec186abf53 HTTP/1.1" 200 None 2026-03-09 16:50:43,554 - lava_callback - INFO - Completed processing callback for node 69aefa1dce46d8ec186abf53 2026-03-09 16:50:43,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:43,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:43,802 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0ece46d8ec186abee5/log.txt.gz 2026-03-09 16:50:43,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:43,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:43,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-09 16:50:44,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:44,115 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0ece46d8ec186abee5/lava-logs.yaml 2026-03-09 16:50:44,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:44,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:44,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa0ece46d8ec186abee5/lava_callback.json.gz 2026-03-09 16:50:44,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:44,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa0ece46d8ec186abee5 HTTP/1.1" 200 None 2026-03-09 16:50:44,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:44,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:44,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:44,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa0ece46d8ec186abee5 HTTP/1.1" 200 None 2026-03-09 16:50:44,350 - lava_callback - INFO - Completed processing callback for node 69aefa0ece46d8ec186abee5 INFO: 172.21.0.1:44398 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:50:48,532 - lava_callback - INFO - Received callback for node 69aec8aefeb96b28aa34b74d INFO: 172.21.0.1:44406 - "POST /node/69aec8aefeb96b28aa34b74d HTTP/1.0" 202 Accepted 2026-03-09 16:50:48,538 - lava_callback - INFO - Starting async job submit for node 69aec8aefeb96b28aa34b74d 2026-03-09 16:50:48,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:48,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec8aefeb96b28aa34b74d HTTP/1.1" 200 None 2026-03-09 16:50:48,588 - lava_callback - DEBUG - Processing node 69aec8aefeb96b28aa34b74d: baseline-arm64-broonie 2026-03-09 16:50:48,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:49,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:49,141 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aec8aefeb96b28aa34b74d/log.txt.gz 2026-03-09 16:50:49,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:49,568 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:49,570 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aec8aefeb96b28aa34b74d/lava-logs.yaml 2026-03-09 16:50:49,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:49,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:49,734 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aec8aefeb96b28aa34b74d/lava_callback.json.gz DEBUG: Setup failed for 69aec8aefeb96b28aa34b74d. Transitting job node result: incomplete -> fail 2026-03-09 16:50:49,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:49,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec8aefeb96b28aa34b74d HTTP/1.1" 200 None 2026-03-09 16:50:49,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:49,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-09 16:50:49,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:49,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aec8aefeb96b28aa34b74d HTTP/1.1" 200 None 2026-03-09 16:50:49,934 - lava_callback - INFO - Completed processing callback for node 69aec8aefeb96b28aa34b74d 2026-03-09 16:50:52,093 - lava_callback - INFO - Received callback for node 69aefa21ce46d8ec186abf5c INFO: 172.21.0.1:49112 - "POST /node/69aefa21ce46d8ec186abf5c HTTP/1.0" 202 Accepted 2026-03-09 16:50:52,141 - lava_callback - INFO - Starting async job submit for node 69aefa21ce46d8ec186abf5c 2026-03-09 16:50:52,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:52,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa21ce46d8ec186abf5c HTTP/1.1" 200 None 2026-03-09 16:50:52,176 - lava_callback - DEBUG - Processing node 69aefa21ce46d8ec186abf5c: baseline-arm-pengutronix 2026-03-09 16:50:52,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:52,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:52,397 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa21ce46d8ec186abf5c/log.txt.gz 2026-03-09 16:50:52,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:52,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:52,657 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa21ce46d8ec186abf5c/lava-logs.yaml 2026-03-09 16:50:52,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:50:52,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:50:52,781 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa21ce46d8ec186abf5c/lava_callback.json.gz 2026-03-09 16:50:52,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:52,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa21ce46d8ec186abf5c HTTP/1.1" 200 None 2026-03-09 16:50:52,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:52,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:50:52,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:50:52,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa21ce46d8ec186abf5c HTTP/1.1" 200 None 2026-03-09 16:50:52,866 - lava_callback - INFO - Completed processing callback for node 69aefa21ce46d8ec186abf5c INFO: 172.21.0.1:49126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36884 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:51:08,280 - lava_callback - INFO - Received callback for node 69aefa1ace46d8ec186abf42 INFO: 172.21.0.1:36896 - "POST /node/69aefa1ace46d8ec186abf42 HTTP/1.0" 202 Accepted 2026-03-09 16:51:08,286 - lava_callback - INFO - Starting async job submit for node 69aefa1ace46d8ec186abf42 2026-03-09 16:51:08,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:08,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1ace46d8ec186abf42 HTTP/1.1" 200 None 2026-03-09 16:51:08,322 - lava_callback - DEBUG - Processing node 69aefa1ace46d8ec186abf42: baseline-arm-kontron 2026-03-09 16:51:08,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:08,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:08,632 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69aefa1ace46d8ec186abf42/log.txt.gz 2026-03-09 16:51:08,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:08,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:08,900 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69aefa1ace46d8ec186abf42/lava-logs.yaml 2026-03-09 16:51:08,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:09,030 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:09,032 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69aefa1ace46d8ec186abf42/lava_callback.json.gz 2026-03-09 16:51:09,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:09,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1ace46d8ec186abf42 HTTP/1.1" 200 None 2026-03-09 16:51:09,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:09,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:51:09,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:09,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa1ace46d8ec186abf42 HTTP/1.1" 200 None 2026-03-09 16:51:09,123 - lava_callback - INFO - Completed processing callback for node 69aefa1ace46d8ec186abf42 2026-03-09 16:51:09,152 - lava_callback - INFO - Received callback for node 69aefa12ce46d8ec186abef7 INFO: 172.21.0.1:36910 - "POST /node/69aefa12ce46d8ec186abef7 HTTP/1.0" 202 Accepted 2026-03-09 16:51:09,162 - lava_callback - INFO - Starting async job submit for node 69aefa12ce46d8ec186abef7 2026-03-09 16:51:09,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:09,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa12ce46d8ec186abef7 HTTP/1.1" 200 None 2026-03-09 16:51:09,231 - lava_callback - DEBUG - Processing node 69aefa12ce46d8ec186abef7: baseline-arm-clabbe 2026-03-09 16:51:09,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:09,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:09,646 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa12ce46d8ec186abef7/log.txt.gz 2026-03-09 16:51:09,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:09,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:09,970 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa12ce46d8ec186abef7/lava-logs.yaml 2026-03-09 16:51:09,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:10,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:10,179 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa12ce46d8ec186abef7/lava_callback.json.gz 2026-03-09 16:51:10,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:10,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa12ce46d8ec186abef7 HTTP/1.1" 200 None 2026-03-09 16:51:10,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:10,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:51:10,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:10,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa12ce46d8ec186abef7 HTTP/1.1" 200 None 2026-03-09 16:51:10,257 - lava_callback - INFO - Completed processing callback for node 69aefa12ce46d8ec186abef7 2026-03-09 16:51:13,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:13,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:49012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37966 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:51:41,441 - lava_callback - INFO - Received callback for node 69aefa1ece46d8ec186abf56 INFO: 172.21.0.1:49440 - "POST /node/69aefa1ece46d8ec186abf56 HTTP/1.0" 202 Accepted 2026-03-09 16:51:41,447 - lava_callback - INFO - Starting async job submit for node 69aefa1ece46d8ec186abf56 2026-03-09 16:51:41,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:41,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1ece46d8ec186abf56 HTTP/1.1" 200 None 2026-03-09 16:51:41,484 - lava_callback - DEBUG - Processing node 69aefa1ece46d8ec186abf56: baseline-arm-pengutronix 2026-03-09 16:51:41,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:41,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:41,761 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1ece46d8ec186abf56/log.txt.gz 2026-03-09 16:51:41,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:42,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:42,042 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1ece46d8ec186abf56/lava-logs.yaml 2026-03-09 16:51:42,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:42,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:42,214 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1ece46d8ec186abf56/lava_callback.json.gz 2026-03-09 16:51:42,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:42,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1ece46d8ec186abf56 HTTP/1.1" 200 None 2026-03-09 16:51:42,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:42,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:51:42,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:42,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa1ece46d8ec186abf56 HTTP/1.1" 200 None 2026-03-09 16:51:42,395 - lava_callback - INFO - Completed processing callback for node 69aefa1ece46d8ec186abf56 2026-03-09 16:51:43,830 - lava_callback - INFO - Received callback for node 69aefa36ce46d8ec186abfa1 INFO: 172.21.0.1:49448 - "POST /node/69aefa36ce46d8ec186abfa1 HTTP/1.0" 202 Accepted 2026-03-09 16:51:43,836 - lava_callback - INFO - Starting async job submit for node 69aefa36ce46d8ec186abfa1 2026-03-09 16:51:43,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:43,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa36ce46d8ec186abfa1 HTTP/1.1" 200 None 2026-03-09 16:51:43,888 - lava_callback - DEBUG - Processing node 69aefa36ce46d8ec186abfa1: ltp-dio 2026-03-09 16:51:43,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:44,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 16:51:44,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:44,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:44,398 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69aefa36ce46d8ec186abfa1/log.txt.gz 2026-03-09 16:51:44,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:44,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:44,844 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69aefa36ce46d8ec186abfa1/lava-logs.yaml 2026-03-09 16:51:44,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:51:45,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:51:45,041 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69aefa36ce46d8ec186abfa1/lava_callback.json.gz 2026-03-09 16:51:45,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:45,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa36ce46d8ec186abfa1 HTTP/1.1" 200 None 2026-03-09 16:51:45,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:45,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:51:45,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:51:45,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa36ce46d8ec186abfa1 HTTP/1.1" 200 None 2026-03-09 16:51:45,139 - lava_callback - INFO - Completed processing callback for node 69aefa36ce46d8ec186abfa1 INFO: 172.21.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50370 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:52:08,414 - lava_callback - INFO - Received callback for node 69aefa10ce46d8ec186abee8 INFO: 172.21.0.1:50372 - "POST /node/69aefa10ce46d8ec186abee8 HTTP/1.0" 202 Accepted 2026-03-09 16:52:08,420 - lava_callback - INFO - Starting async job submit for node 69aefa10ce46d8ec186abee8 2026-03-09 16:52:08,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:08,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa10ce46d8ec186abee8 HTTP/1.1" 200 None 2026-03-09 16:52:08,458 - lava_callback - DEBUG - Processing node 69aefa10ce46d8ec186abee8: baseline-arm-broonie 2026-03-09 16:52:08,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:52:08,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:08,876 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa10ce46d8ec186abee8/log.txt.gz 2026-03-09 16:52:09,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:52:09,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:09,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa10ce46d8ec186abee8/lava-logs.yaml 2026-03-09 16:52:09,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:52:09,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:09,589 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefa10ce46d8ec186abee8/lava_callback.json.gz 2026-03-09 16:52:09,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:09,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa10ce46d8ec186abee8 HTTP/1.1" 200 None 2026-03-09 16:52:09,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:09,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:52:09,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:09,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa10ce46d8ec186abee8 HTTP/1.1" 200 None 2026-03-09 16:52:09,803 - lava_callback - INFO - Completed processing callback for node 69aefa10ce46d8ec186abee8 2026-03-09 16:52:14,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:14,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39038 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:52:56,234 - lava_callback - INFO - Received callback for node 69aefa13ce46d8ec186abefb INFO: 172.21.0.1:39042 - "POST /node/69aefa13ce46d8ec186abefb HTTP/1.0" 202 Accepted 2026-03-09 16:52:56,240 - lava_callback - INFO - Starting async job submit for node 69aefa13ce46d8ec186abefb 2026-03-09 16:52:56,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:56,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa13ce46d8ec186abefb HTTP/1.1" 200 None 2026-03-09 16:52:56,277 - lava_callback - DEBUG - Processing node 69aefa13ce46d8ec186abefb: baseline-arm-clabbe 2026-03-09 16:52:56,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:39048 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:52:56,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:56,623 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa13ce46d8ec186abefb/log.txt.gz 2026-03-09 16:52:56,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:52:56,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:56,925 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa13ce46d8ec186abefb/lava-logs.yaml 2026-03-09 16:52:56,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:52:57,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:52:57,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa13ce46d8ec186abefb/lava_callback.json.gz 2026-03-09 16:52:57,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:57,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa13ce46d8ec186abefb HTTP/1.1" 200 None 2026-03-09 16:52:57,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:57,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:52:57,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:52:57,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa13ce46d8ec186abefb HTTP/1.1" 200 None 2026-03-09 16:52:57,145 - lava_callback - INFO - Completed processing callback for node 69aefa13ce46d8ec186abefb INFO: 172.21.0.1:55306 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:53:14,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:53:14,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40948 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:54:35,354 - lava_callback - INFO - Received callback for node 69aefa17ce46d8ec186abf07 INFO: 172.21.0.1:40110 - "POST /node/69aefa17ce46d8ec186abf07 HTTP/1.0" 202 Accepted 2026-03-09 16:54:35,367 - lava_callback - INFO - Starting async job submit for node 69aefa17ce46d8ec186abf07 2026-03-09 16:54:35,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:35,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa17ce46d8ec186abf07 HTTP/1.1" 200 None 2026-03-09 16:54:35,422 - lava_callback - DEBUG - Processing node 69aefa17ce46d8ec186abf07: baseline-arm-clabbe 2026-03-09 16:54:35,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:54:36,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:54:36,010 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa17ce46d8ec186abf07/log.txt.gz 2026-03-09 16:54:36,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40112 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:54:36,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:54:36,682 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa17ce46d8ec186abf07/lava-logs.yaml 2026-03-09 16:54:36,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:54:36,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:54:36,961 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa17ce46d8ec186abf07/lava_callback.json.gz 2026-03-09 16:54:36,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:36,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa17ce46d8ec186abf07 HTTP/1.1" 200 None 2026-03-09 16:54:36,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:37,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:54:37,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:37,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa17ce46d8ec186abf07 HTTP/1.1" 200 None 2026-03-09 16:54:37,046 - lava_callback - INFO - Completed processing callback for node 69aefa17ce46d8ec186abf07 2026-03-09 16:54:44,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:44,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32858 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:54:54,937 - lava_callback - INFO - Received callback for node 69aefb48ce46d8ec186ac13a INFO: 172.21.0.1:39816 - "POST /node/69aefb48ce46d8ec186ac13a HTTP/1.0" 202 Accepted 2026-03-09 16:54:54,947 - lava_callback - INFO - Starting async job submit for node 69aefb48ce46d8ec186ac13a 2026-03-09 16:54:54,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:54,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefb48ce46d8ec186ac13a HTTP/1.1" 200 None 2026-03-09 16:54:54,978 - lava_callback - DEBUG - Processing node 69aefb48ce46d8ec186ac13a: baseline-arm 2026-03-09 16:54:54,988 - lava_callback - WARNING - Failed to upload log 2026-03-09 16:54:54,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:54:55,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:54:55,140 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aefb48ce46d8ec186ac13a/lava-logs.yaml 2026-03-09 16:54:55,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:54:55,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:54:55,287 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aefb48ce46d8ec186ac13a/lava_callback.json.gz 2026-03-09 16:54:55,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:55,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefb48ce46d8ec186ac13a HTTP/1.1" 200 None 2026-03-09 16:54:55,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:55,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:54:55,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:54:55,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefb48ce46d8ec186ac13a HTTP/1.1" 200 None 2026-03-09 16:54:55,381 - lava_callback - INFO - Completed processing callback for node 69aefb48ce46d8ec186ac13a INFO: 172.21.0.1:39820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32956 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:55:14,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:14,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39028 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:55:28,993 - lava_callback - INFO - Received callback for node 69aefa3bce46d8ec186abfbb INFO: 172.21.0.1:39030 - "POST /node/69aefa3bce46d8ec186abfbb HTTP/1.0" 202 Accepted 2026-03-09 16:55:29,000 - lava_callback - INFO - Starting async job submit for node 69aefa3bce46d8ec186abfbb 2026-03-09 16:55:29,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:29,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa3bce46d8ec186abfbb HTTP/1.1" 200 None 2026-03-09 16:55:29,039 - lava_callback - DEBUG - Processing node 69aefa3bce46d8ec186abfbb: ltp-syscalls 2026-03-09 16:55:29,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:55:29,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:55:29,967 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69aefa3bce46d8ec186abfbb/log.txt.gz 2026-03-09 16:55:30,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:55:30,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:55:30,662 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69aefa3bce46d8ec186abfbb/lava-logs.yaml 2026-03-09 16:55:30,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:55:30,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:55:30,874 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69aefa3bce46d8ec186abfbb/lava_callback.json.gz 2026-03-09 16:55:30,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:30,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa3bce46d8ec186abfbb HTTP/1.1" 200 None 2026-03-09 16:55:30,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:30,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:55:30,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:30,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa3bce46d8ec186abfbb HTTP/1.1" 200 None 2026-03-09 16:55:30,982 - lava_callback - INFO - Completed processing callback for node 69aefa3bce46d8ec186abfbb INFO: 172.21.0.1:52856 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:55:44,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:55:44,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34778 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:56:04,778 - lava_callback - INFO - Received callback for node 69ac589146f7ae8ff3365632 INFO: 172.21.0.1:44624 - "POST /node/69ac589146f7ae8ff3365632 HTTP/1.0" 202 Accepted 2026-03-09 16:56:04,783 - lava_callback - INFO - Starting async job submit for node 69ac589146f7ae8ff3365632 2026-03-09 16:56:04,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:04,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589146f7ae8ff3365632 HTTP/1.1" 200 None 2026-03-09 16:56:04,799 - lava_callback - DEBUG - Processing node 69ac589146f7ae8ff3365632: baseline-arm64-baylibre 2026-03-09 16:56:04,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589146f7ae8ff3365632 2026-03-09 16:56:04,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:04,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589146f7ae8ff3365632 HTTP/1.1" 200 None 2026-03-09 16:56:04,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:04,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:04,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:04,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589146f7ae8ff3365632 HTTP/1.1" 200 None 2026-03-09 16:56:04,857 - lava_callback - ERROR - Error processing callback for node 69ac589146f7ae8ff3365632: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:05,444 - lava_callback - INFO - Received callback for node 69ac589246f7ae8ff3365635 INFO: 172.21.0.1:44634 - "POST /node/69ac589246f7ae8ff3365635 HTTP/1.0" 202 Accepted 2026-03-09 16:56:05,450 - lava_callback - INFO - Starting async job submit for node 69ac589246f7ae8ff3365635 2026-03-09 16:56:05,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:05,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589246f7ae8ff3365635 HTTP/1.1" 200 None 2026-03-09 16:56:05,466 - lava_callback - DEBUG - Processing node 69ac589246f7ae8ff3365635: baseline-arm64-baylibre 2026-03-09 16:56:05,466 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589246f7ae8ff3365635 2026-03-09 16:56:05,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:05,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589246f7ae8ff3365635 HTTP/1.1" 200 None 2026-03-09 16:56:05,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:05,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:05,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:05,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589246f7ae8ff3365635 HTTP/1.1" 200 None 2026-03-09 16:56:05,519 - lava_callback - ERROR - Error processing callback for node 69ac589246f7ae8ff3365635: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:06,113 - lava_callback - INFO - Received callback for node 69ac589446f7ae8ff3365645 INFO: 172.21.0.1:44640 - "POST /node/69ac589446f7ae8ff3365645 HTTP/1.0" 202 Accepted 2026-03-09 16:56:06,119 - lava_callback - INFO - Starting async job submit for node 69ac589446f7ae8ff3365645 2026-03-09 16:56:06,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589446f7ae8ff3365645 HTTP/1.1" 200 None 2026-03-09 16:56:06,141 - lava_callback - DEBUG - Processing node 69ac589446f7ae8ff3365645: baseline-arm64-baylibre 2026-03-09 16:56:06,141 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589446f7ae8ff3365645 2026-03-09 16:56:06,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589446f7ae8ff3365645 HTTP/1.1" 200 None 2026-03-09 16:56:06,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:06,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589446f7ae8ff3365645 HTTP/1.1" 200 None 2026-03-09 16:56:06,195 - lava_callback - ERROR - Error processing callback for node 69ac589446f7ae8ff3365645: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44652 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:56:06,787 - lava_callback - INFO - Received callback for node 69ac589646f7ae8ff3365668 INFO: 172.21.0.1:44666 - "POST /node/69ac589646f7ae8ff3365668 HTTP/1.0" 202 Accepted 2026-03-09 16:56:06,794 - lava_callback - INFO - Starting async job submit for node 69ac589646f7ae8ff3365668 2026-03-09 16:56:06,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589646f7ae8ff3365668 HTTP/1.1" 200 None 2026-03-09 16:56:06,824 - lava_callback - DEBUG - Processing node 69ac589646f7ae8ff3365668: baseline-arm64-baylibre 2026-03-09 16:56:06,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589646f7ae8ff3365668 2026-03-09 16:56:06,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589646f7ae8ff3365668 HTTP/1.1" 200 None 2026-03-09 16:56:06,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:06,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:06,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589646f7ae8ff3365668 HTTP/1.1" 200 None 2026-03-09 16:56:06,885 - lava_callback - ERROR - Error processing callback for node 69ac589646f7ae8ff3365668: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:07,433 - lava_callback - INFO - Received callback for node 69ac589846f7ae8ff336566d INFO: 172.21.0.1:44672 - "POST /node/69ac589846f7ae8ff336566d HTTP/1.0" 202 Accepted 2026-03-09 16:56:07,442 - lava_callback - INFO - Starting async job submit for node 69ac589846f7ae8ff336566d 2026-03-09 16:56:07,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:07,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589846f7ae8ff336566d HTTP/1.1" 200 None 2026-03-09 16:56:07,472 - lava_callback - DEBUG - Processing node 69ac589846f7ae8ff336566d: baseline-arm64-baylibre 2026-03-09 16:56:07,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589846f7ae8ff336566d 2026-03-09 16:56:07,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:07,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589846f7ae8ff336566d HTTP/1.1" 200 None 2026-03-09 16:56:07,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:07,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:07,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:07,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589846f7ae8ff336566d HTTP/1.1" 200 None 2026-03-09 16:56:07,547 - lava_callback - ERROR - Error processing callback for node 69ac589846f7ae8ff336566d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:08,099 - lava_callback - INFO - Received callback for node 69ac589946f7ae8ff3365674 INFO: 172.21.0.1:44682 - "POST /node/69ac589946f7ae8ff3365674 HTTP/1.0" 202 Accepted 2026-03-09 16:56:08,108 - lava_callback - INFO - Starting async job submit for node 69ac589946f7ae8ff3365674 2026-03-09 16:56:08,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:08,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589946f7ae8ff3365674 HTTP/1.1" 200 None 2026-03-09 16:56:08,143 - lava_callback - DEBUG - Processing node 69ac589946f7ae8ff3365674: baseline-arm64-baylibre 2026-03-09 16:56:08,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac589946f7ae8ff3365674 2026-03-09 16:56:08,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:08,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac589946f7ae8ff3365674 HTTP/1.1" 200 None 2026-03-09 16:56:08,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:08,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:08,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:08,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac589946f7ae8ff3365674 HTTP/1.1" 200 None 2026-03-09 16:56:08,219 - lava_callback - ERROR - Error processing callback for node 69ac589946f7ae8ff3365674: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57778 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:56:17,836 - lava_callback - INFO - Received callback for node 69ac58ab46f7ae8ff33656ab INFO: 172.21.0.1:57780 - "POST /node/69ac58ab46f7ae8ff33656ab HTTP/1.0" 202 Accepted 2026-03-09 16:56:17,845 - lava_callback - INFO - Starting async job submit for node 69ac58ab46f7ae8ff33656ab 2026-03-09 16:56:17,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:17,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac58ab46f7ae8ff33656ab HTTP/1.1" 200 None 2026-03-09 16:56:17,873 - lava_callback - DEBUG - Processing node 69ac58ab46f7ae8ff33656ab: baseline-arm64-clabbe 2026-03-09 16:56:17,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac58ab46f7ae8ff33656ab 2026-03-09 16:56:17,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:17,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac58ab46f7ae8ff33656ab HTTP/1.1" 200 None 2026-03-09 16:56:17,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:17,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:17,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:17,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac58ab46f7ae8ff33656ab HTTP/1.1" 200 None 2026-03-09 16:56:17,976 - lava_callback - ERROR - Error processing callback for node 69ac58ab46f7ae8ff33656ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:18,524 - lava_callback - INFO - Received callback for node 69aefa18ce46d8ec186abf0a INFO: 172.21.0.1:57796 - "POST /node/69aefa18ce46d8ec186abf0a HTTP/1.0" 202 Accepted 2026-03-09 16:56:18,533 - lava_callback - INFO - Starting async job submit for node 69aefa18ce46d8ec186abf0a 2026-03-09 16:56:18,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:18,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa18ce46d8ec186abf0a HTTP/1.1" 200 None 2026-03-09 16:56:18,593 - lava_callback - DEBUG - Processing node 69aefa18ce46d8ec186abf0a: baseline-arm-clabbe 2026-03-09 16:56:18,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:56:19,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:56:19,079 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa18ce46d8ec186abf0a/log.txt.gz 2026-03-09 16:56:19,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:56:19,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:56:19,719 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa18ce46d8ec186abf0a/lava-logs.yaml 2026-03-09 16:56:19,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:56:19,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:56:19,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69aefa18ce46d8ec186abf0a/lava_callback.json.gz 2026-03-09 16:56:19,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:19,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa18ce46d8ec186abf0a HTTP/1.1" 200 None 2026-03-09 16:56:19,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:19,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 16:56:19,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:19,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa18ce46d8ec186abf0a HTTP/1.1" 200 None 2026-03-09 16:56:19,959 - lava_callback - INFO - Completed processing callback for node 69aefa18ce46d8ec186abf0a INFO: 172.21.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58860 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:56:37,868 - lava_callback - INFO - Received callback for node 69ac58b246f7ae8ff33656c2 INFO: 172.21.0.1:58862 - "POST /node/69ac58b246f7ae8ff33656c2 HTTP/1.0" 202 Accepted 2026-03-09 16:56:37,874 - lava_callback - INFO - Starting async job submit for node 69ac58b246f7ae8ff33656c2 2026-03-09 16:56:37,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:37,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac58b246f7ae8ff33656c2 HTTP/1.1" 200 None 2026-03-09 16:56:37,898 - lava_callback - DEBUG - Processing node 69ac58b246f7ae8ff33656c2: baseline-arm64-clabbe 2026-03-09 16:56:37,898 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac58b246f7ae8ff33656c2 2026-03-09 16:56:37,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:37,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac58b246f7ae8ff33656c2 HTTP/1.1" 200 None 2026-03-09 16:56:37,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:37,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 16:56:37,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:37,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac58b246f7ae8ff33656c2 HTTP/1.1" 200 None 2026-03-09 16:56:37,962 - lava_callback - ERROR - Error processing callback for node 69ac58b246f7ae8ff33656c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 16:56:44,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:56:44,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36670 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:57:22,579 - lava_callback - INFO - Received callback for node 69aef767ce46d8ec186abbfd INFO: 172.21.0.1:37972 - "POST /node/69aef767ce46d8ec186abbfd HTTP/1.0" 202 Accepted 2026-03-09 16:57:22,586 - lava_callback - INFO - Starting async job submit for node 69aef767ce46d8ec186abbfd 2026-03-09 16:57:22,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:57:22,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef767ce46d8ec186abbfd HTTP/1.1" 200 None 2026-03-09 16:57:22,660 - lava_callback - DEBUG - Processing node 69aef767ce46d8ec186abbfd: ltp-smoke 2026-03-09 16:57:23,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:57:23,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:57:23,329 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69aef767ce46d8ec186abbfd/log.txt.gz 2026-03-09 16:57:23,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:57:23,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:57:23,855 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69aef767ce46d8ec186abbfd/lava-logs.yaml 2026-03-09 16:57:23,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 16:57:24,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 16:57:24,033 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69aef767ce46d8ec186abbfd/lava_callback.json.gz 2026-03-09 16:57:24,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:57:24,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef767ce46d8ec186abbfd HTTP/1.1" 200 None 2026-03-09 16:57:24,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:57:24,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef42ace46d8ec186ab814 HTTP/1.1" 200 None 2026-03-09 16:57:24,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:57:24,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef767ce46d8ec186abbfd HTTP/1.1" 200 None 2026-03-09 16:57:24,340 - lava_callback - INFO - Completed processing callback for node 69aef767ce46d8ec186abbfd INFO: 172.21.0.1:37980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40106 - "GET / HTTP/1.0" 200 OK 2026-03-09 16:57:44,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 16:57:44,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52708 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:00:38,146 - lava_callback - INFO - Received callback for node 69aefc65ce46d8ec186ac289 INFO: 172.21.0.1:52724 - "POST /node/69aefc65ce46d8ec186ac289 HTTP/1.0" 202 Accepted 2026-03-09 17:00:38,156 - lava_callback - INFO - Starting async job submit for node 69aefc65ce46d8ec186ac289 2026-03-09 17:00:38,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:00:38,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc65ce46d8ec186ac289 HTTP/1.1" 200 None 2026-03-09 17:00:38,197 - lava_callback - DEBUG - Processing node 69aefc65ce46d8ec186ac289: baseline-arm-pengutronix 2026-03-09 17:00:38,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:00:38,562 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:00:38,564 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefc65ce46d8ec186ac289/log.txt.gz 2026-03-09 17:00:38,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:00:38,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:00:38,874 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefc65ce46d8ec186ac289/lava-logs.yaml 2026-03-09 17:00:38,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:00:39,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:00:39,041 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefc65ce46d8ec186ac289/lava_callback.json.gz 2026-03-09 17:00:39,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:00:39,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc65ce46d8ec186ac289 HTTP/1.1" 200 None 2026-03-09 17:00:39,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:00:39,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:00:39,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:00:39,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefc65ce46d8ec186ac289 HTTP/1.1" 200 None 2026-03-09 17:00:39,145 - lava_callback - INFO - Completed processing callback for node 69aefc65ce46d8ec186ac289 2026-03-09 17:00:44,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:00:44,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50918 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:01:28,723 - lava_callback - INFO - Received callback for node 69aefc89ce46d8ec186ac2a5 INFO: 172.21.0.1:50932 - "POST /node/69aefc89ce46d8ec186ac2a5 HTTP/1.0" 202 Accepted 2026-03-09 17:01:28,729 - lava_callback - INFO - Starting async job submit for node 69aefc89ce46d8ec186ac2a5 2026-03-09 17:01:28,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:28,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc89ce46d8ec186ac2a5 HTTP/1.1" 200 None 2026-03-09 17:01:28,767 - lava_callback - DEBUG - Processing node 69aefc89ce46d8ec186ac2a5: baseline-arm-broonie 2026-03-09 17:01:28,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:01:29,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:01:29,120 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefc89ce46d8ec186ac2a5/log.txt.gz 2026-03-09 17:01:29,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:01:29,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:01:29,450 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefc89ce46d8ec186ac2a5/lava-logs.yaml 2026-03-09 17:01:29,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:01:29,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:01:29,599 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69aefc89ce46d8ec186ac2a5/lava_callback.json.gz 2026-03-09 17:01:29,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:29,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc89ce46d8ec186ac2a5 HTTP/1.1" 200 None 2026-03-09 17:01:29,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:29,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:01:29,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:29,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefc89ce46d8ec186ac2a5 HTTP/1.1" 200 None 2026-03-09 17:01:29,688 - lava_callback - INFO - Completed processing callback for node 69aefc89ce46d8ec186ac2a5 INFO: 172.21.0.1:44718 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:01:44,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:44,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58528 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:01:57,531 - lava_callback - INFO - Received callback for node 69aefcf8ce46d8ec186ac33c INFO: 172.21.0.1:58534 - "POST /node/69aefcf8ce46d8ec186ac33c HTTP/1.0" 202 Accepted 2026-03-09 17:01:57,537 - lava_callback - INFO - Starting async job submit for node 69aefcf8ce46d8ec186ac33c 2026-03-09 17:01:57,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:57,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefcf8ce46d8ec186ac33c HTTP/1.1" 200 None 2026-03-09 17:01:57,555 - lava_callback - DEBUG - Processing node 69aefcf8ce46d8ec186ac33c: baseline-arm 2026-03-09 17:01:57,560 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:01:57,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:01:57,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:01:57,702 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aefcf8ce46d8ec186ac33c/lava-logs.yaml 2026-03-09 17:01:57,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:01:57,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:01:57,859 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aefcf8ce46d8ec186ac33c/lava_callback.json.gz 2026-03-09 17:01:57,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:57,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefcf8ce46d8ec186ac33c HTTP/1.1" 200 None 2026-03-09 17:01:57,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:57,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:01:57,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:01:57,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefcf8ce46d8ec186ac33c HTTP/1.1" 200 None 2026-03-09 17:01:57,924 - lava_callback - INFO - Completed processing callback for node 69aefcf8ce46d8ec186ac33c INFO: 172.21.0.1:56352 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:02:14,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:02:14,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42530 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:05:25,243 - lava_callback - INFO - Received callback for node 69ac5ac946f7ae8ff336590e INFO: 172.21.0.1:56880 - "POST /node/69ac5ac946f7ae8ff336590e HTTP/1.0" 202 Accepted 2026-03-09 17:05:25,252 - lava_callback - INFO - Starting async job submit for node 69ac5ac946f7ae8ff336590e 2026-03-09 17:05:25,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:05:25,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5ac946f7ae8ff336590e HTTP/1.1" 200 None 2026-03-09 17:05:25,268 - lava_callback - DEBUG - Processing node 69ac5ac946f7ae8ff336590e: baseline-arm64-baylibre 2026-03-09 17:05:25,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5ac946f7ae8ff336590e 2026-03-09 17:05:25,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:05:25,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5ac946f7ae8ff336590e HTTP/1.1" 200 None 2026-03-09 17:05:25,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:05:25,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a64e31b554803095c1a HTTP/1.1" 200 None 2026-03-09 17:05:25,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:05:25,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5ac946f7ae8ff336590e HTTP/1.1" 200 None 2026-03-09 17:05:25,365 - lava_callback - ERROR - Error processing callback for node 69ac5ac946f7ae8ff336590e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:06:06,930 - lava_callback - INFO - Received callback for node 69aef9fece46d8ec186abe94 INFO: 172.21.0.1:44030 - "POST /node/69aef9fece46d8ec186abe94 HTTP/1.0" 202 Accepted 2026-03-09 17:06:06,937 - lava_callback - INFO - Starting async job submit for node 69aef9fece46d8ec186abe94 2026-03-09 17:06:06,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:06:07,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fece46d8ec186abe94 HTTP/1.1" 200 None 2026-03-09 17:06:07,011 - lava_callback - DEBUG - Processing node 69aef9fece46d8ec186abe94: baseline-arm 2026-03-09 17:06:07,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:06:07,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:06:07,967 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fece46d8ec186abe94/log.txt.gz 2026-03-09 17:06:08,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:06:08,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:06:08,696 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fece46d8ec186abe94/lava-logs.yaml 2026-03-09 17:06:08,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:06:08,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:06:08,923 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69aef9fece46d8ec186abe94/lava_callback.json.gz DEBUG: Setup failed for 69aef9fece46d8ec186abe94. Transitting job node result: incomplete -> fail 2026-03-09 17:06:09,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:06:09,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef9fece46d8ec186abe94 HTTP/1.1" 200 None 2026-03-09 17:06:09,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:06:09,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:06:09,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:06:09,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef9fece46d8ec186abe94 HTTP/1.1" 200 None 2026-03-09 17:06:09,283 - lava_callback - INFO - Completed processing callback for node 69aef9fece46d8ec186abe94 2026-03-09 17:06:14,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:06:14,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40398 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:07:45,324 - lava_callback - INFO - Received callback for node 69ac5b5c46f7ae8ff3365944 INFO: 172.21.0.1:50242 - "POST /node/69ac5b5c46f7ae8ff3365944 HTTP/1.0" 202 Accepted 2026-03-09 17:07:45,335 - lava_callback - INFO - Starting async job submit for node 69ac5b5c46f7ae8ff3365944 2026-03-09 17:07:45,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:07:45,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5b5c46f7ae8ff3365944 HTTP/1.1" 200 None 2026-03-09 17:07:45,368 - lava_callback - DEBUG - Processing node 69ac5b5c46f7ae8ff3365944: baseline-arm64-baylibre 2026-03-09 17:07:45,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5b5c46f7ae8ff3365944 2026-03-09 17:07:45,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:07:45,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5b5c46f7ae8ff3365944 HTTP/1.1" 200 None 2026-03-09 17:07:45,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:07:45,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a64e31b554803095c1a HTTP/1.1" 200 None 2026-03-09 17:07:45,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:07:45,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5b5c46f7ae8ff3365944 HTTP/1.1" 200 None 2026-03-09 17:07:45,596 - lava_callback - ERROR - Error processing callback for node 69ac5b5c46f7ae8ff3365944: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60556 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:12:05,595 - lava_callback - INFO - Received callback for node 69ac5c5e46f7ae8ff33659a2 INFO: 172.21.0.1:60896 - "POST /node/69ac5c5e46f7ae8ff33659a2 HTTP/1.0" 202 Accepted 2026-03-09 17:12:05,601 - lava_callback - INFO - Starting async job submit for node 69ac5c5e46f7ae8ff33659a2 2026-03-09 17:12:05,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:12:05,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5c5e46f7ae8ff33659a2 HTTP/1.1" 200 None 2026-03-09 17:12:05,644 - lava_callback - DEBUG - Processing node 69ac5c5e46f7ae8ff33659a2: baseline-arm64-baylibre 2026-03-09 17:12:05,644 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5c5e46f7ae8ff33659a2 2026-03-09 17:12:05,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:12:05,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5c5e46f7ae8ff33659a2 HTTP/1.1" 200 None 2026-03-09 17:12:05,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:12:05,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 17:12:05,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:12:05,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5c5e46f7ae8ff33659a2 HTTP/1.1" 200 None 2026-03-09 17:12:05,719 - lava_callback - ERROR - Error processing callback for node 69ac5c5e46f7ae8ff33659a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35200 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:13:25,652 - lava_callback - INFO - Received callback for node 69ac5caa46f7ae8ff33659ae INFO: 172.21.0.1:46330 - "POST /node/69ac5caa46f7ae8ff33659ae HTTP/1.0" 202 Accepted 2026-03-09 17:13:25,658 - lava_callback - INFO - Starting async job submit for node 69ac5caa46f7ae8ff33659ae 2026-03-09 17:13:25,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:13:25,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5caa46f7ae8ff33659ae HTTP/1.1" 200 None 2026-03-09 17:13:25,676 - lava_callback - DEBUG - Processing node 69ac5caa46f7ae8ff33659ae: baseline-arm64-baylibre 2026-03-09 17:13:25,676 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5caa46f7ae8ff33659ae 2026-03-09 17:13:25,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:13:25,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5caa46f7ae8ff33659ae HTTP/1.1" 200 None 2026-03-09 17:13:25,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:13:25,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 17:13:25,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:13:25,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5caa46f7ae8ff33659ae HTTP/1.1" 200 None 2026-03-09 17:13:25,732 - lava_callback - ERROR - Error processing callback for node 69ac5caa46f7ae8ff33659ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54232 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:11,717 - lava_callback - INFO - Received callback for node 69aeffb8ce46d8ec186ac440 INFO: 172.21.0.1:41512 - "POST /node/69aeffb8ce46d8ec186ac440 HTTP/1.0" 202 Accepted 2026-03-09 17:14:11,723 - lava_callback - INFO - Starting async job submit for node 69aeffb8ce46d8ec186ac440 2026-03-09 17:14:11,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:11,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb8ce46d8ec186ac440 HTTP/1.1" 200 None 2026-03-09 17:14:11,758 - lava_callback - DEBUG - Processing node 69aeffb8ce46d8ec186ac440: baseline-x86 2026-03-09 17:14:11,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:12,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:12,099 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac440/log.txt.gz 2026-03-09 17:14:12,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:12,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:12,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac440/lava-logs.yaml 2026-03-09 17:14:12,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:12,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:12,590 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac440/lava_callback.json.gz 2026-03-09 17:14:12,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:12,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb8ce46d8ec186ac440 HTTP/1.1" 200 None 2026-03-09 17:14:12,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:12,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef449ce46d8ec186ab83b HTTP/1.1" 200 None 2026-03-09 17:14:12,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:12,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aeffb8ce46d8ec186ac440 HTTP/1.1" 200 None 2026-03-09 17:14:12,672 - lava_callback - INFO - Completed processing callback for node 69aeffb8ce46d8ec186ac440 2026-03-09 17:14:14,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:14,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41522 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:14:16,090 - lava_callback - INFO - Received callback for node 69aeffb6ce46d8ec186ac435 INFO: 172.21.0.1:41528 - "POST /node/69aeffb6ce46d8ec186ac435 HTTP/1.0" 202 Accepted 2026-03-09 17:14:16,096 - lava_callback - INFO - Starting async job submit for node 69aeffb6ce46d8ec186ac435 2026-03-09 17:14:16,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:16,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb6ce46d8ec186ac435 HTTP/1.1" 200 None 2026-03-09 17:14:16,129 - lava_callback - DEBUG - Processing node 69aeffb6ce46d8ec186ac435: baseline-x86 2026-03-09 17:14:16,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:16,454 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:16,456 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb6ce46d8ec186ac435/log.txt.gz INFO: 172.21.0.1:41544 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:16,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:16,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:16,751 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb6ce46d8ec186ac435/lava-logs.yaml 2026-03-09 17:14:16,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:16,911 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:16,913 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb6ce46d8ec186ac435/lava_callback.json.gz 2026-03-09 17:14:16,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:16,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb6ce46d8ec186ac435 HTTP/1.1" 200 None 2026-03-09 17:14:16,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:16,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef449ce46d8ec186ab83b HTTP/1.1" 200 None 2026-03-09 17:14:16,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:17,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aeffb6ce46d8ec186ac435 HTTP/1.1" 200 None 2026-03-09 17:14:17,004 - lava_callback - INFO - Completed processing callback for node 69aeffb6ce46d8ec186ac435 INFO: 172.21.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33004 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:40,263 - lava_callback - INFO - Received callback for node 69aeffb7ce46d8ec186ac438 INFO: 172.21.0.1:36044 - "POST /node/69aeffb7ce46d8ec186ac438 HTTP/1.0" 202 Accepted 2026-03-09 17:14:40,269 - lava_callback - INFO - Starting async job submit for node 69aeffb7ce46d8ec186ac438 2026-03-09 17:14:40,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:40,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb7ce46d8ec186ac438 HTTP/1.1" 200 None 2026-03-09 17:14:40,303 - lava_callback - DEBUG - Processing node 69aeffb7ce46d8ec186ac438: baseline-x86 2026-03-09 17:14:40,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:40,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:40,725 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb7ce46d8ec186ac438/log.txt.gz 2026-03-09 17:14:40,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:41,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:41,145 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb7ce46d8ec186ac438/lava-logs.yaml 2026-03-09 17:14:41,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:41,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:41,297 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb7ce46d8ec186ac438/lava_callback.json.gz 2026-03-09 17:14:41,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:41,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb7ce46d8ec186ac438 HTTP/1.1" 200 None 2026-03-09 17:14:41,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:41,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef449ce46d8ec186ab83b HTTP/1.1" 200 None 2026-03-09 17:14:41,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:41,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aeffb7ce46d8ec186ac438 HTTP/1.1" 200 None 2026-03-09 17:14:41,381 - lava_callback - INFO - Completed processing callback for node 69aeffb7ce46d8ec186ac438 2026-03-09 17:14:44,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:44,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 17:14:45,708 - lava_callback - INFO - Received callback for node 69ac5cf446f7ae8ff33659cc INFO: 172.21.0.1:36048 - "POST /node/69ac5cf446f7ae8ff33659cc HTTP/1.0" 202 Accepted 2026-03-09 17:14:45,713 - lava_callback - INFO - Starting async job submit for node 69ac5cf446f7ae8ff33659cc 2026-03-09 17:14:45,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:45,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5cf446f7ae8ff33659cc HTTP/1.1" 200 None 2026-03-09 17:14:45,731 - lava_callback - DEBUG - Processing node 69ac5cf446f7ae8ff33659cc: baseline-arm64-baylibre 2026-03-09 17:14:45,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5cf446f7ae8ff33659cc 2026-03-09 17:14:45,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:45,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5cf446f7ae8ff33659cc HTTP/1.1" 200 None 2026-03-09 17:14:45,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:45,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 17:14:45,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:45,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5cf446f7ae8ff33659cc HTTP/1.1" 200 None 2026-03-09 17:14:45,804 - lava_callback - ERROR - Error processing callback for node 69ac5cf446f7ae8ff33659cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36058 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:56,030 - lava_callback - INFO - Received callback for node 69aeffb8ce46d8ec186ac43b INFO: 172.21.0.1:52418 - "POST /node/69aeffb8ce46d8ec186ac43b HTTP/1.0" 202 Accepted 2026-03-09 17:14:56,036 - lava_callback - INFO - Starting async job submit for node 69aeffb8ce46d8ec186ac43b 2026-03-09 17:14:56,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:56,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb8ce46d8ec186ac43b HTTP/1.1" 200 None 2026-03-09 17:14:56,070 - lava_callback - DEBUG - Processing node 69aeffb8ce46d8ec186ac43b: baseline-x86 INFO: 172.21.0.1:52428 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:56,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:56,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:56,468 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac43b/log.txt.gz INFO: 172.21.0.1:52434 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:14:56,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:56,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:56,798 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac43b/lava-logs.yaml 2026-03-09 17:14:56,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:14:56,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:14:56,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffb8ce46d8ec186ac43b/lava_callback.json.gz 2026-03-09 17:14:56,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:56,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffb8ce46d8ec186ac43b HTTP/1.1" 200 None 2026-03-09 17:14:56,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:57,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef449ce46d8ec186ab83b HTTP/1.1" 200 None 2026-03-09 17:14:57,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:14:57,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aeffb8ce46d8ec186ac43b HTTP/1.1" 200 None 2026-03-09 17:14:57,037 - lava_callback - INFO - Completed processing callback for node 69aeffb8ce46d8ec186ac43b INFO: 172.21.0.1:41534 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:15:14,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:15:14,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52370 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:15:45,732 - lava_callback - INFO - Received callback for node 69ac5d3f46f7ae8ff33659e3 INFO: 172.21.0.1:51174 - "POST /node/69ac5d3f46f7ae8ff33659e3 HTTP/1.0" 202 Accepted 2026-03-09 17:15:45,737 - lava_callback - INFO - Starting async job submit for node 69ac5d3f46f7ae8ff33659e3 2026-03-09 17:15:45,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:15:45,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5d3f46f7ae8ff33659e3 HTTP/1.1" 200 None 2026-03-09 17:15:45,755 - lava_callback - DEBUG - Processing node 69ac5d3f46f7ae8ff33659e3: baseline-arm64-baylibre 2026-03-09 17:15:45,755 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5d3f46f7ae8ff33659e3 2026-03-09 17:15:45,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:15:45,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5d3f46f7ae8ff33659e3 HTTP/1.1" 200 None 2026-03-09 17:15:45,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:15:45,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 17:15:45,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:15:45,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5d3f46f7ae8ff33659e3 HTTP/1.1" 200 None 2026-03-09 17:15:45,813 - lava_callback - ERROR - Error processing callback for node 69ac5d3f46f7ae8ff33659e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59694 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:17:18,500 - lava_callback - INFO - Received callback for node 69af008fce46d8ec186ac515 INFO: 172.21.0.1:59704 - "POST /node/69af008fce46d8ec186ac515 HTTP/1.0" 202 Accepted 2026-03-09 17:17:18,509 - lava_callback - INFO - Starting async job submit for node 69af008fce46d8ec186ac515 2026-03-09 17:17:18,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af008fce46d8ec186ac515 HTTP/1.1" 200 None 2026-03-09 17:17:18,528 - lava_callback - DEBUG - Processing node 69af008fce46d8ec186ac515: baseline-arm 2026-03-09 17:17:18,533 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:17:18,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:17:18,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:17:18,713 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af008fce46d8ec186ac515/lava-logs.yaml 2026-03-09 17:17:18,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:17:18,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:17:18,849 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af008fce46d8ec186ac515/lava_callback.json.gz 2026-03-09 17:17:18,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,859 - lava_callback - INFO - Received callback for node 69ac5d8e46f7ae8ff33659f9 INFO: 172.21.0.1:59716 - "POST /node/69ac5d8e46f7ae8ff33659f9 HTTP/1.0" 202 Accepted 2026-03-09 17:17:18,867 - lava_callback - INFO - Starting async job submit for node 69ac5d8e46f7ae8ff33659f9 2026-03-09 17:17:18,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af008fce46d8ec186ac515 HTTP/1.1" 200 None 2026-03-09 17:17:18,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5d8e46f7ae8ff33659f9 HTTP/1.1" 200 None 2026-03-09 17:17:18,884 - lava_callback - DEBUG - Processing node 69ac5d8e46f7ae8ff33659f9: baseline-arm64-clabbe 2026-03-09 17:17:18,884 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5d8e46f7ae8ff33659f9 2026-03-09 17:17:18,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:17:18,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5d8e46f7ae8ff33659f9 HTTP/1.1" 200 None 2026-03-09 17:17:18,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:18,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af008fce46d8ec186ac515 HTTP/1.1" 200 None 2026-03-09 17:17:18,930 - lava_callback - INFO - Completed processing callback for node 69af008fce46d8ec186ac515 2026-03-09 17:17:18,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 17:17:18,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:19,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5d8e46f7ae8ff33659f9 HTTP/1.1" 200 None 2026-03-09 17:17:19,009 - lava_callback - ERROR - Error processing callback for node 69ac5d8e46f7ae8ff33659f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37910 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:17:44,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:17:44,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44672 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:21:32,721 - lava_callback - INFO - Received callback for node 69af018fce46d8ec186ac613 INFO: 172.21.0.1:38332 - "POST /node/69af018fce46d8ec186ac613 HTTP/1.0" 202 Accepted 2026-03-09 17:21:32,727 - lava_callback - INFO - Starting async job submit for node 69af018fce46d8ec186ac613 2026-03-09 17:21:32,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:32,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018fce46d8ec186ac613 HTTP/1.1" 200 None 2026-03-09 17:21:32,747 - lava_callback - DEBUG - Processing node 69af018fce46d8ec186ac613: baseline-arm64 2026-03-09 17:21:32,751 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:21:32,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:32,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:32,921 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018fce46d8ec186ac613/lava-logs.yaml 2026-03-09 17:21:32,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:33,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:33,072 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018fce46d8ec186ac613/lava_callback.json.gz 2026-03-09 17:21:33,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:33,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018fce46d8ec186ac613 HTTP/1.1" 200 None 2026-03-09 17:21:33,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:33,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:21:33,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:33,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af018fce46d8ec186ac613 HTTP/1.1" 200 None 2026-03-09 17:21:33,160 - lava_callback - INFO - Completed processing callback for node 69af018fce46d8ec186ac613 2026-03-09 17:21:34,890 - lava_callback - INFO - Received callback for node 69af018fce46d8ec186ac610 INFO: 172.21.0.1:38338 - "POST /node/69af018fce46d8ec186ac610 HTTP/1.0" 202 Accepted 2026-03-09 17:21:34,897 - lava_callback - INFO - Starting async job submit for node 69af018fce46d8ec186ac610 2026-03-09 17:21:34,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:34,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018fce46d8ec186ac610 HTTP/1.1" 200 None 2026-03-09 17:21:34,919 - lava_callback - DEBUG - Processing node 69af018fce46d8ec186ac610: baseline-arm64 2026-03-09 17:21:34,923 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:21:34,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:35,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:35,049 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018fce46d8ec186ac610/lava-logs.yaml 2026-03-09 17:21:35,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:35,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:35,175 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018fce46d8ec186ac610/lava_callback.json.gz 2026-03-09 17:21:35,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:35,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018fce46d8ec186ac610 HTTP/1.1" 200 None 2026-03-09 17:21:35,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:35,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:21:35,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:35,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af018fce46d8ec186ac610 HTTP/1.1" 200 None 2026-03-09 17:21:35,245 - lava_callback - INFO - Completed processing callback for node 69af018fce46d8ec186ac610 INFO: 172.21.0.1:38354 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:21:44,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:44,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:21:57,949 - lava_callback - INFO - Received callback for node 69af019bce46d8ec186ac631 INFO: 172.21.0.1:45814 - "POST /node/69af019bce46d8ec186ac631 HTTP/1.0" 202 Accepted 2026-03-09 17:21:57,959 - lava_callback - INFO - Starting async job submit for node 69af019bce46d8ec186ac631 2026-03-09 17:21:57,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:57,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019bce46d8ec186ac631 HTTP/1.1" 200 None 2026-03-09 17:21:57,989 - lava_callback - DEBUG - Processing node 69af019bce46d8ec186ac631: baseline-arm64 2026-03-09 17:21:57,998 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:21:58,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:58,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:58,227 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019bce46d8ec186ac631/lava-logs.yaml 2026-03-09 17:21:58,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:21:58,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:21:58,395 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019bce46d8ec186ac631/lava_callback.json.gz 2026-03-09 17:21:58,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:58,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019bce46d8ec186ac631 HTTP/1.1" 200 None 2026-03-09 17:21:58,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:58,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:21:58,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:21:58,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af019bce46d8ec186ac631 HTTP/1.1" 200 None 2026-03-09 17:21:58,480 - lava_callback - INFO - Completed processing callback for node 69af019bce46d8ec186ac631 2026-03-09 17:22:01,147 - lava_callback - INFO - Received callback for node 69af018ece46d8ec186ac60d INFO: 172.21.0.1:51838 - "POST /node/69af018ece46d8ec186ac60d HTTP/1.0" 202 Accepted 2026-03-09 17:22:01,158 - lava_callback - INFO - Starting async job submit for node 69af018ece46d8ec186ac60d 2026-03-09 17:22:01,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:01,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018ece46d8ec186ac60d HTTP/1.1" 200 None 2026-03-09 17:22:01,223 - lava_callback - DEBUG - Processing node 69af018ece46d8ec186ac60d: baseline-arm64 2026-03-09 17:22:01,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:01,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:01,606 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018ece46d8ec186ac60d/log.txt.gz 2026-03-09 17:22:01,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:01,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:01,882 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018ece46d8ec186ac60d/lava-logs.yaml 2026-03-09 17:22:01,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:02,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:02,059 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af018ece46d8ec186ac60d/lava_callback.json.gz 2026-03-09 17:22:02,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:02,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af018ece46d8ec186ac60d HTTP/1.1" 200 None 2026-03-09 17:22:02,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:02,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:02,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:02,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af018ece46d8ec186ac60d HTTP/1.1" 200 None 2026-03-09 17:22:02,143 - lava_callback - INFO - Completed processing callback for node 69af018ece46d8ec186ac60d INFO: 172.21.0.1:51850 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:07,192 - lava_callback - INFO - Received callback for node 69af0193ce46d8ec186ac61c INFO: 172.21.0.1:51854 - "POST /node/69af0193ce46d8ec186ac61c HTTP/1.0" 202 Accepted 2026-03-09 17:22:07,199 - lava_callback - INFO - Starting async job submit for node 69af0193ce46d8ec186ac61c 2026-03-09 17:22:07,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:07,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0193ce46d8ec186ac61c HTTP/1.1" 200 None 2026-03-09 17:22:07,234 - lava_callback - DEBUG - Processing node 69af0193ce46d8ec186ac61c: baseline-arm64 2026-03-09 17:22:07,295 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:07,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:07,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:07,462 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0193ce46d8ec186ac61c/lava-logs.yaml 2026-03-09 17:22:07,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:07,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:07,583 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0193ce46d8ec186ac61c/lava_callback.json.gz 2026-03-09 17:22:07,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:07,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0193ce46d8ec186ac61c HTTP/1.1" 200 None 2026-03-09 17:22:07,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:07,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:07,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:07,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0193ce46d8ec186ac61c HTTP/1.1" 200 None 2026-03-09 17:22:07,789 - lava_callback - INFO - Completed processing callback for node 69af0193ce46d8ec186ac61c 2026-03-09 17:22:14,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:14,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 17:22:14,829 - lava_callback - INFO - Received callback for node 69af01aace46d8ec186ac673 INFO: 172.21.0.1:42660 - "POST /node/69af01aace46d8ec186ac673 HTTP/1.0" 202 Accepted 2026-03-09 17:22:14,835 - lava_callback - INFO - Starting async job submit for node 69af01aace46d8ec186ac673 2026-03-09 17:22:14,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:14,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01aace46d8ec186ac673 HTTP/1.1" 200 None 2026-03-09 17:22:14,853 - lava_callback - DEBUG - Processing node 69af01aace46d8ec186ac673: baseline-nfs-arm64 2026-03-09 17:22:14,855 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:14,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:15,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:15,039 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01aace46d8ec186ac673/lava-logs.yaml 2026-03-09 17:22:15,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:42672 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:22:15,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:15,177 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01aace46d8ec186ac673/lava_callback.json.gz 2026-03-09 17:22:15,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:15,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01aace46d8ec186ac673 HTTP/1.1" 200 None 2026-03-09 17:22:15,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:15,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:15,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:15,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01aace46d8ec186ac673 HTTP/1.1" 200 None 2026-03-09 17:22:15,245 - lava_callback - INFO - Completed processing callback for node 69af01aace46d8ec186ac673 2026-03-09 17:22:16,128 - lava_callback - INFO - Received callback for node 69af01a1ce46d8ec186ac65c INFO: 172.21.0.1:42676 - "POST /node/69af01a1ce46d8ec186ac65c HTTP/1.0" 202 Accepted 2026-03-09 17:22:16,135 - lava_callback - INFO - Starting async job submit for node 69af01a1ce46d8ec186ac65c 2026-03-09 17:22:16,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:16,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a1ce46d8ec186ac65c HTTP/1.1" 200 None 2026-03-09 17:22:16,168 - lava_callback - DEBUG - Processing node 69af01a1ce46d8ec186ac65c: baseline-arm64 2026-03-09 17:22:16,219 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:16,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:16,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:16,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af01a1ce46d8ec186ac65c/lava-logs.yaml 2026-03-09 17:22:16,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:42682 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:16,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:16,539 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af01a1ce46d8ec186ac65c/lava_callback.json.gz 2026-03-09 17:22:16,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:16,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a1ce46d8ec186ac65c HTTP/1.1" 200 None 2026-03-09 17:22:16,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:16,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:16,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:16,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a1ce46d8ec186ac65c HTTP/1.1" 200 None 2026-03-09 17:22:16,710 - lava_callback - INFO - Completed processing callback for node 69af01a1ce46d8ec186ac65c 2026-03-09 17:22:18,263 - lava_callback - INFO - Received callback for node 69af019ace46d8ec186ac62e INFO: 172.21.0.1:42688 - "POST /node/69af019ace46d8ec186ac62e HTTP/1.0" 202 Accepted 2026-03-09 17:22:18,270 - lava_callback - INFO - Starting async job submit for node 69af019ace46d8ec186ac62e 2026-03-09 17:22:18,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:18,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019ace46d8ec186ac62e HTTP/1.1" 200 None 2026-03-09 17:22:18,303 - lava_callback - DEBUG - Processing node 69af019ace46d8ec186ac62e: baseline-arm64 2026-03-09 17:22:18,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:18,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:18,617 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ace46d8ec186ac62e/log.txt.gz 2026-03-09 17:22:18,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:18,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:18,926 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ace46d8ec186ac62e/lava-logs.yaml 2026-03-09 17:22:18,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:19,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:19,140 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ace46d8ec186ac62e/lava_callback.json.gz 2026-03-09 17:22:19,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:19,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019ace46d8ec186ac62e HTTP/1.1" 200 None 2026-03-09 17:22:19,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:19,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:19,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:19,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af019ace46d8ec186ac62e HTTP/1.1" 200 None 2026-03-09 17:22:19,226 - lava_callback - INFO - Completed processing callback for node 69af019ace46d8ec186ac62e 2026-03-09 17:22:20,446 - lava_callback - INFO - Received callback for node 69af0191ce46d8ec186ac619 INFO: 172.21.0.1:42562 - "POST /node/69af0191ce46d8ec186ac619 HTTP/1.0" 202 Accepted 2026-03-09 17:22:20,456 - lava_callback - INFO - Starting async job submit for node 69af0191ce46d8ec186ac619 2026-03-09 17:22:20,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:20,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0191ce46d8ec186ac619 HTTP/1.1" 200 None 2026-03-09 17:22:20,492 - lava_callback - DEBUG - Processing node 69af0191ce46d8ec186ac619: baseline-arm64 2026-03-09 17:22:20,553 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:20,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:20,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:20,726 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0191ce46d8ec186ac619/lava-logs.yaml 2026-03-09 17:22:20,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:20,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:20,882 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0191ce46d8ec186ac619/lava_callback.json.gz 2026-03-09 17:22:20,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:20,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0191ce46d8ec186ac619 HTTP/1.1" 200 None 2026-03-09 17:22:20,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:20,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:20,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:20,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0191ce46d8ec186ac619 HTTP/1.1" 200 None 2026-03-09 17:22:21,023 - lava_callback - INFO - Completed processing callback for node 69af0191ce46d8ec186ac619 INFO: 172.21.0.1:42570 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:32,996 - lava_callback - INFO - Received callback for node 69af01abce46d8ec186ac676 INFO: 172.21.0.1:45484 - "POST /node/69af01abce46d8ec186ac676 HTTP/1.0" 202 Accepted 2026-03-09 17:22:33,002 - lava_callback - INFO - Starting async job submit for node 69af01abce46d8ec186ac676 2026-03-09 17:22:33,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:33,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01abce46d8ec186ac676 HTTP/1.1" 200 None 2026-03-09 17:22:33,029 - lava_callback - DEBUG - Processing node 69af01abce46d8ec186ac676: baseline-nfs-arm64 2026-03-09 17:22:33,035 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:33,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:33,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:33,158 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01abce46d8ec186ac676/lava-logs.yaml 2026-03-09 17:22:33,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:33,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:33,314 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01abce46d8ec186ac676/lava_callback.json.gz 2026-03-09 17:22:33,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:33,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01abce46d8ec186ac676 HTTP/1.1" 200 None 2026-03-09 17:22:33,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:33,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:33,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:33,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01abce46d8ec186ac676 HTTP/1.1" 200 None 2026-03-09 17:22:33,438 - lava_callback - INFO - Completed processing callback for node 69af01abce46d8ec186ac676 INFO: 172.21.0.1:45498 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:44,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:44,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36350 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:50,741 - lava_callback - INFO - Received callback for node 69af01dace46d8ec186ac715 INFO: 172.21.0.1:34958 - "POST /node/69af01dace46d8ec186ac715 HTTP/1.0" 202 Accepted 2026-03-09 17:22:50,747 - lava_callback - INFO - Starting async job submit for node 69af01dace46d8ec186ac715 2026-03-09 17:22:50,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:50,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01dace46d8ec186ac715 HTTP/1.1" 200 None 2026-03-09 17:22:50,794 - lava_callback - DEBUG - Processing node 69af01dace46d8ec186ac715: baseline-arm64-foundriesio 2026-03-09 17:22:50,846 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:50,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:51,056 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:51,058 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af01dace46d8ec186ac715/lava-logs.yaml 2026-03-09 17:22:51,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:51,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:51,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af01dace46d8ec186ac715/lava_callback.json.gz 2026-03-09 17:22:51,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:51,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01dace46d8ec186ac715 HTTP/1.1" 200 None 2026-03-09 17:22:51,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:51,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:51,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:51,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01dace46d8ec186ac715 HTTP/1.1" 200 None 2026-03-09 17:22:51,330 - lava_callback - INFO - Completed processing callback for node 69af01dace46d8ec186ac715 INFO: 172.21.0.1:34968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34984 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:22:57,169 - lava_callback - INFO - Received callback for node 69af01d1ce46d8ec186ac6fd INFO: 172.21.0.1:34988 - "POST /node/69af01d1ce46d8ec186ac6fd HTTP/1.0" 202 Accepted 2026-03-09 17:22:57,174 - lava_callback - INFO - Starting async job submit for node 69af01d1ce46d8ec186ac6fd 2026-03-09 17:22:57,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:57,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d1ce46d8ec186ac6fd HTTP/1.1" 200 None 2026-03-09 17:22:57,192 - lava_callback - DEBUG - Processing node 69af01d1ce46d8ec186ac6fd: baseline-arm64-clabbe 2026-03-09 17:22:57,197 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:22:57,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:57,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:57,327 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af01d1ce46d8ec186ac6fd/lava-logs.yaml 2026-03-09 17:22:57,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:22:57,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:22:57,600 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af01d1ce46d8ec186ac6fd/lava_callback.json.gz 2026-03-09 17:22:57,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:57,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d1ce46d8ec186ac6fd HTTP/1.1" 200 None 2026-03-09 17:22:57,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:57,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:22:57,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:22:57,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01d1ce46d8ec186ac6fd HTTP/1.1" 200 None 2026-03-09 17:22:57,689 - lava_callback - INFO - Completed processing callback for node 69af01d1ce46d8ec186ac6fd 2026-03-09 17:23:03,103 - lava_callback - INFO - Received callback for node 69af01ccce46d8ec186ac6ec INFO: 172.21.0.1:51706 - "POST /node/69af01ccce46d8ec186ac6ec HTTP/1.0" 202 Accepted 2026-03-09 17:23:03,111 - lava_callback - INFO - Starting async job submit for node 69af01ccce46d8ec186ac6ec 2026-03-09 17:23:03,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:03,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01ccce46d8ec186ac6ec HTTP/1.1" 200 None 2026-03-09 17:23:03,137 - lava_callback - DEBUG - Processing node 69af01ccce46d8ec186ac6ec: baseline-arm64-broonie 2026-03-09 17:23:03,146 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:23:03,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:03,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:03,286 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01ccce46d8ec186ac6ec/lava-logs.yaml 2026-03-09 17:23:03,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:03,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:03,696 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01ccce46d8ec186ac6ec/lava_callback.json.gz 2026-03-09 17:23:03,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:03,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01ccce46d8ec186ac6ec HTTP/1.1" 200 None 2026-03-09 17:23:03,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:03,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:03,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:03,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01ccce46d8ec186ac6ec HTTP/1.1" 200 None 2026-03-09 17:23:03,818 - lava_callback - INFO - Completed processing callback for node 69af01ccce46d8ec186ac6ec INFO: 172.21.0.1:51712 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:23:13,895 - lava_callback - INFO - Received callback for node 69af0198ce46d8ec186ac628 INFO: 172.21.0.1:49500 - "POST /node/69af0198ce46d8ec186ac628 HTTP/1.0" 202 Accepted 2026-03-09 17:23:13,902 - lava_callback - INFO - Starting async job submit for node 69af0198ce46d8ec186ac628 2026-03-09 17:23:13,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:13,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0198ce46d8ec186ac628 HTTP/1.1" 200 None 2026-03-09 17:23:13,981 - lava_callback - DEBUG - Processing node 69af0198ce46d8ec186ac628: baseline-arm64 2026-03-09 17:23:14,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:14,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:14,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 17:23:14,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:14,938 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0198ce46d8ec186ac628/log.txt.gz INFO: 172.21.0.1:49502 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:23:15,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:15,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:15,710 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0198ce46d8ec186ac628/lava-logs.yaml 2026-03-09 17:23:15,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:15,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:15,839 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0198ce46d8ec186ac628/lava_callback.json.gz DEBUG: Setup failed for 69af0198ce46d8ec186ac628. Transitting job node result: incomplete -> fail 2026-03-09 17:23:15,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:15,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0198ce46d8ec186ac628 HTTP/1.1" 200 None 2026-03-09 17:23:15,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:15,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:15,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:16,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0198ce46d8ec186ac628 HTTP/1.1" 200 None 2026-03-09 17:23:16,179 - lava_callback - INFO - Completed processing callback for node 69af0198ce46d8ec186ac628 INFO: 172.21.0.1:49508 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:23:20,993 - lava_callback - INFO - Received callback for node 69af0196ce46d8ec186ac625 INFO: 172.21.0.1:34424 - "POST /node/69af0196ce46d8ec186ac625 HTTP/1.0" 202 Accepted 2026-03-09 17:23:21,001 - lava_callback - INFO - Starting async job submit for node 69af0196ce46d8ec186ac625 2026-03-09 17:23:21,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:21,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0196ce46d8ec186ac625 HTTP/1.1" 200 None 2026-03-09 17:23:21,051 - lava_callback - DEBUG - Processing node 69af0196ce46d8ec186ac625: baseline-arm64 2026-03-09 17:23:22,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:22,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:22,411 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0196ce46d8ec186ac625/log.txt.gz 2026-03-09 17:23:23,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:23,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:23,388 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0196ce46d8ec186ac625/lava-logs.yaml 2026-03-09 17:23:23,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:23,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:23,799 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0196ce46d8ec186ac625/lava_callback.json.gz DEBUG: Setup failed for 69af0196ce46d8ec186ac625. Transitting job node result: incomplete -> fail 2026-03-09 17:23:23,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:23,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0196ce46d8ec186ac625 HTTP/1.1" 200 None 2026-03-09 17:23:23,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:23,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:23,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:23,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0196ce46d8ec186ac625 HTTP/1.1" 200 None 2026-03-09 17:23:24,034 - lava_callback - INFO - Completed processing callback for node 69af0196ce46d8ec186ac625 INFO: 172.21.0.1:34438 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:23:27,005 - lava_callback - INFO - Received callback for node 69af01a0ce46d8ec186ac659 INFO: 172.21.0.1:34446 - "POST /node/69af01a0ce46d8ec186ac659 HTTP/1.0" 202 Accepted 2026-03-09 17:23:27,016 - lava_callback - INFO - Starting async job submit for node 69af01a0ce46d8ec186ac659 2026-03-09 17:23:27,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:27,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a0ce46d8ec186ac659 HTTP/1.1" 200 None 2026-03-09 17:23:27,097 - lava_callback - DEBUG - Processing node 69af01a0ce46d8ec186ac659: baseline-arm64 2026-03-09 17:23:27,206 - lava_callback - INFO - Received callback for node 69af019ece46d8ec186ac639 INFO: 172.21.0.1:34454 - "POST /node/69af019ece46d8ec186ac639 HTTP/1.0" 202 Accepted 2026-03-09 17:23:27,228 - lava_callback - INFO - Starting async job submit for node 69af019ece46d8ec186ac639 2026-03-09 17:23:27,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:27,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019ece46d8ec186ac639 HTTP/1.1" 200 None 2026-03-09 17:23:27,429 - lava_callback - DEBUG - Processing node 69af019ece46d8ec186ac639: baseline-arm64 2026-03-09 17:23:28,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:28,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:28,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:28,218 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af01a0ce46d8ec186ac659/log.txt.gz 2026-03-09 17:23:28,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:28,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:28,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ece46d8ec186ac639/log.txt.gz 2026-03-09 17:23:28,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:28,952 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af01a0ce46d8ec186ac659/lava-logs.yaml 2026-03-09 17:23:28,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:28,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:29,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:29,480 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af01a0ce46d8ec186ac659/lava_callback.json.gz 2026-03-09 17:23:29,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:29,592 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ece46d8ec186ac639/lava-logs.yaml DEBUG: Setup failed for 69af01a0ce46d8ec186ac659. Transitting job node result: incomplete -> fail 2026-03-09 17:23:29,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:29,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:29,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a0ce46d8ec186ac659 HTTP/1.1" 200 None 2026-03-09 17:23:29,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:29,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:29,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:29,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a0ce46d8ec186ac659 HTTP/1.1" 200 None 2026-03-09 17:23:29,824 - lava_callback - INFO - Completed processing callback for node 69af01a0ce46d8ec186ac659 2026-03-09 17:23:30,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:30,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019ece46d8ec186ac639/lava_callback.json.gz DEBUG: Setup failed for 69af019ece46d8ec186ac639. Transitting job node result: incomplete -> fail 2026-03-09 17:23:30,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:30,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019ece46d8ec186ac639 HTTP/1.1" 200 None 2026-03-09 17:23:30,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:30,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:30,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:30,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af019ece46d8ec186ac639 HTTP/1.1" 200 None 2026-03-09 17:23:30,604 - lava_callback - INFO - Completed processing callback for node 69af019ece46d8ec186ac639 INFO: 172.21.0.1:52748 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:23:36,738 - lava_callback - INFO - Received callback for node 69af01d5ce46d8ec186ac706 INFO: 172.21.0.1:52762 - "POST /node/69af01d5ce46d8ec186ac706 HTTP/1.0" 202 Accepted 2026-03-09 17:23:36,747 - lava_callback - INFO - Starting async job submit for node 69af01d5ce46d8ec186ac706 2026-03-09 17:23:36,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:36,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d5ce46d8ec186ac706 HTTP/1.1" 200 None 2026-03-09 17:23:36,806 - lava_callback - DEBUG - Processing node 69af01d5ce46d8ec186ac706: baseline-arm64-pengutronix 2026-03-09 17:23:36,872 - lava_callback - INFO - Received callback for node 69af0199ce46d8ec186ac62b INFO: 172.21.0.1:52768 - "POST /node/69af0199ce46d8ec186ac62b HTTP/1.0" 202 Accepted 2026-03-09 17:23:36,899 - lava_callback - INFO - Starting async job submit for node 69af0199ce46d8ec186ac62b 2026-03-09 17:23:37,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:37,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0199ce46d8ec186ac62b HTTP/1.1" 200 None 2026-03-09 17:23:37,089 - lava_callback - DEBUG - Processing node 69af0199ce46d8ec186ac62b: baseline-arm64 2026-03-09 17:23:37,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:37,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:37,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d5ce46d8ec186ac706/log.txt.gz 2026-03-09 17:23:38,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:38,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:38,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:38,728 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0199ce46d8ec186ac62b/log.txt.gz 2026-03-09 17:23:38,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:38,833 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d5ce46d8ec186ac706/lava-logs.yaml 2026-03-09 17:23:38,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:39,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:39,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:39,131 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d5ce46d8ec186ac706/lava_callback.json.gz 2026-03-09 17:23:39,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:39,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d5ce46d8ec186ac706 HTTP/1.1" 200 None 2026-03-09 17:23:39,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:39,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:39,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:39,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01d5ce46d8ec186ac706 HTTP/1.1" 200 None 2026-03-09 17:23:39,265 - lava_callback - INFO - Completed processing callback for node 69af01d5ce46d8ec186ac706 2026-03-09 17:23:39,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:39,742 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0199ce46d8ec186ac62b/lava-logs.yaml 2026-03-09 17:23:39,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:39,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:39,974 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0199ce46d8ec186ac62b/lava_callback.json.gz DEBUG: Setup failed for 69af0199ce46d8ec186ac62b. Transitting job node result: incomplete -> fail 2026-03-09 17:23:40,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:40,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0199ce46d8ec186ac62b HTTP/1.1" 200 None 2026-03-09 17:23:40,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:40,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:40,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:40,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0199ce46d8ec186ac62b HTTP/1.1" 200 None 2026-03-09 17:23:40,287 - lava_callback - INFO - Completed processing callback for node 69af0199ce46d8ec186ac62b 2026-03-09 17:23:42,241 - lava_callback - INFO - Received callback for node 69af01a8ce46d8ec186ac66b INFO: 172.21.0.1:47594 - "POST /node/69af01a8ce46d8ec186ac66b HTTP/1.0" 202 Accepted 2026-03-09 17:23:42,248 - lava_callback - INFO - Starting async job submit for node 69af01a8ce46d8ec186ac66b 2026-03-09 17:23:42,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:42,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a8ce46d8ec186ac66b HTTP/1.1" 200 None 2026-03-09 17:23:42,325 - lava_callback - DEBUG - Processing node 69af01a8ce46d8ec186ac66b: baseline-nfs-arm64 2026-03-09 17:23:42,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:43,165 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:43,167 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a8ce46d8ec186ac66b/log.txt.gz 2026-03-09 17:23:43,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:43,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:43,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a8ce46d8ec186ac66b/lava-logs.yaml 2026-03-09 17:23:43,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:44,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:44,218 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a8ce46d8ec186ac66b/lava_callback.json.gz DEBUG: Setup failed for 69af01a8ce46d8ec186ac66b. Transitting job node result: incomplete -> fail 2026-03-09 17:23:44,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:44,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a8ce46d8ec186ac66b HTTP/1.1" 200 None 2026-03-09 17:23:44,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:44,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:44,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:44,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a8ce46d8ec186ac66b HTTP/1.1" 200 None 2026-03-09 17:23:44,429 - lava_callback - INFO - Completed processing callback for node 69af01a8ce46d8ec186ac66b 2026-03-09 17:23:44,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:44,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:47600 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:23:48,418 - lava_callback - INFO - Received callback for node 69af01d6ce46d8ec186ac709 INFO: 172.21.0.1:47612 - "POST /node/69af01d6ce46d8ec186ac709 HTTP/1.0" 202 Accepted 2026-03-09 17:23:48,427 - lava_callback - INFO - Starting async job submit for node 69af01d6ce46d8ec186ac709 2026-03-09 17:23:48,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:48,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d6ce46d8ec186ac709 HTTP/1.1" 200 None 2026-03-09 17:23:48,502 - lava_callback - DEBUG - Processing node 69af01d6ce46d8ec186ac709: baseline-arm64-pengutronix 2026-03-09 17:23:48,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:48,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:48,989 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d6ce46d8ec186ac709/log.txt.gz 2026-03-09 17:23:49,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:49,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:49,576 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d6ce46d8ec186ac709/lava-logs.yaml 2026-03-09 17:23:49,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:23:49,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:23:49,755 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d6ce46d8ec186ac709/lava_callback.json.gz 2026-03-09 17:23:49,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:49,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d6ce46d8ec186ac709 HTTP/1.1" 200 None 2026-03-09 17:23:49,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:49,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:23:49,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:23:49,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01d6ce46d8ec186ac709 HTTP/1.1" 200 None 2026-03-09 17:23:49,849 - lava_callback - INFO - Completed processing callback for node 69af01d6ce46d8ec186ac709 INFO: 172.21.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48906 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:24:01,573 - lava_callback - INFO - Received callback for node 69af01d9ce46d8ec186ac710 INFO: 172.21.0.1:41442 - "POST /node/69af01d9ce46d8ec186ac710 HTTP/1.0" 202 Accepted 2026-03-09 17:24:01,582 - lava_callback - INFO - Starting async job submit for node 69af01d9ce46d8ec186ac710 2026-03-09 17:24:01,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:01,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d9ce46d8ec186ac710 HTTP/1.1" 200 None 2026-03-09 17:24:01,683 - lava_callback - DEBUG - Processing node 69af01d9ce46d8ec186ac710: baseline-arm64-foundriesio 2026-03-09 17:24:02,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:02,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:02,201 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af01d9ce46d8ec186ac710/log.txt.gz 2026-03-09 17:24:02,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:02,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:02,733 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af01d9ce46d8ec186ac710/lava-logs.yaml 2026-03-09 17:24:02,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:02,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:02,919 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af01d9ce46d8ec186ac710/lava_callback.json.gz DEBUG: Setup failed for 69af01d9ce46d8ec186ac710. Transitting job node result: incomplete -> fail 2026-03-09 17:24:02,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:03,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d9ce46d8ec186ac710 HTTP/1.1" 200 None 2026-03-09 17:24:03,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:03,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:03,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:03,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01d9ce46d8ec186ac710 HTTP/1.1" 200 None 2026-03-09 17:24:03,217 - lava_callback - INFO - Completed processing callback for node 69af01d9ce46d8ec186ac710 2026-03-09 17:24:06,366 - lava_callback - INFO - Received callback for node 69af01aece46d8ec186ac67e INFO: 172.21.0.1:41456 - "POST /node/69af01aece46d8ec186ac67e HTTP/1.0" 202 Accepted 2026-03-09 17:24:06,376 - lava_callback - INFO - Starting async job submit for node 69af01aece46d8ec186ac67e 2026-03-09 17:24:06,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:06,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01aece46d8ec186ac67e HTTP/1.1" 200 None 2026-03-09 17:24:06,469 - lava_callback - DEBUG - Processing node 69af01aece46d8ec186ac67e: baseline-nfs-arm64 INFO: 172.21.0.1:41462 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:24:07,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:07,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:07,227 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01aece46d8ec186ac67e/log.txt.gz 2026-03-09 17:24:07,436 - lava_callback - INFO - Received callback for node 69af01a7ce46d8ec186ac668 INFO: 172.21.0.1:41466 - "POST /node/69af01a7ce46d8ec186ac668 HTTP/1.0" 202 Accepted 2026-03-09 17:24:07,469 - lava_callback - INFO - Starting async job submit for node 69af01a7ce46d8ec186ac668 2026-03-09 17:24:07,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:07,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:07,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a7ce46d8ec186ac668 HTTP/1.1" 200 None 2026-03-09 17:24:07,599 - lava_callback - DEBUG - Processing node 69af01a7ce46d8ec186ac668: baseline-nfs-arm64 2026-03-09 17:24:08,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:08,052 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01aece46d8ec186ac67e/lava-logs.yaml 2026-03-09 17:24:08,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:08,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:08,741 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01aece46d8ec186ac67e/lava_callback.json.gz DEBUG: Setup failed for 69af01aece46d8ec186ac67e. Transitting job node result: incomplete -> fail 2026-03-09 17:24:08,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:08,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:08,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01aece46d8ec186ac67e HTTP/1.1" 200 None 2026-03-09 17:24:08,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:09,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:09,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:09,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01aece46d8ec186ac67e HTTP/1.1" 200 None 2026-03-09 17:24:09,208 - lava_callback - INFO - Completed processing callback for node 69af01aece46d8ec186ac67e 2026-03-09 17:24:09,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:09,498 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a7ce46d8ec186ac668/log.txt.gz 2026-03-09 17:24:10,094 - lava_callback - INFO - Received callback for node 69af01d8ce46d8ec186ac70d INFO: 172.21.0.1:59216 - "POST /node/69af01d8ce46d8ec186ac70d HTTP/1.0" 202 Accepted 2026-03-09 17:24:10,117 - lava_callback - INFO - Starting async job submit for node 69af01d8ce46d8ec186ac70d 2026-03-09 17:24:10,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:10,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:10,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d8ce46d8ec186ac70d HTTP/1.1" 200 None 2026-03-09 17:24:10,255 - lava_callback - DEBUG - Processing node 69af01d8ce46d8ec186ac70d: baseline-arm64-pengutronix 2026-03-09 17:24:10,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:10,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:10,891 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d8ce46d8ec186ac70d/log.txt.gz 2026-03-09 17:24:10,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:10,919 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a7ce46d8ec186ac668/lava-logs.yaml 2026-03-09 17:24:11,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:11,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:11,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:11,237 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a7ce46d8ec186ac668/lava_callback.json.gz DEBUG: Setup failed for 69af01a7ce46d8ec186ac668. Transitting job node result: incomplete -> fail 2026-03-09 17:24:11,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:11,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a7ce46d8ec186ac668 HTTP/1.1" 200 None 2026-03-09 17:24:11,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:11,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:11,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:11,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a7ce46d8ec186ac668 HTTP/1.1" 200 None 2026-03-09 17:24:11,562 - lava_callback - INFO - Completed processing callback for node 69af01a7ce46d8ec186ac668 2026-03-09 17:24:11,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:11,772 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d8ce46d8ec186ac70d/lava-logs.yaml 2026-03-09 17:24:11,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:11,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:11,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69af01d8ce46d8ec186ac70d/lava_callback.json.gz 2026-03-09 17:24:11,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:11,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01d8ce46d8ec186ac70d HTTP/1.1" 200 None 2026-03-09 17:24:11,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:11,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:11,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:12,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01d8ce46d8ec186ac70d HTTP/1.1" 200 None 2026-03-09 17:24:12,028 - lava_callback - INFO - Completed processing callback for node 69af01d8ce46d8ec186ac70d 2026-03-09 17:24:14,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:14,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:59218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59220 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:24:21,322 - lava_callback - INFO - Received callback for node 69af01a9ce46d8ec186ac670 INFO: 172.21.0.1:34734 - "POST /node/69af01a9ce46d8ec186ac670 HTTP/1.0" 202 Accepted 2026-03-09 17:24:21,329 - lava_callback - INFO - Starting async job submit for node 69af01a9ce46d8ec186ac670 2026-03-09 17:24:21,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:21,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a9ce46d8ec186ac670 HTTP/1.1" 200 None 2026-03-09 17:24:21,394 - lava_callback - DEBUG - Processing node 69af01a9ce46d8ec186ac670: baseline-nfs-arm64 2026-03-09 17:24:22,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:22,454 - lava_callback - INFO - Received callback for node 69af01b1ce46d8ec186ac688 INFO: 172.21.0.1:34740 - "POST /node/69af01b1ce46d8ec186ac688 HTTP/1.0" 202 Accepted 2026-03-09 17:24:22,466 - lava_callback - INFO - Starting async job submit for node 69af01b1ce46d8ec186ac688 2026-03-09 17:24:22,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:22,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01b1ce46d8ec186ac688 HTTP/1.1" 200 None 2026-03-09 17:24:22,552 - lava_callback - DEBUG - Processing node 69af01b1ce46d8ec186ac688: baseline-nfs-arm64 2026-03-09 17:24:22,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:22,577 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a9ce46d8ec186ac670/log.txt.gz 2026-03-09 17:24:23,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:23,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:23,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:23,952 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01b1ce46d8ec186ac688/log.txt.gz 2026-03-09 17:24:24,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:24,151 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a9ce46d8ec186ac670/lava-logs.yaml 2026-03-09 17:24:24,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:24,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:24,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:24,711 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a9ce46d8ec186ac670/lava_callback.json.gz 2026-03-09 17:24:24,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 DEBUG: Setup failed for 69af01a9ce46d8ec186ac670. Transitting job node result: incomplete -> fail 2026-03-09 17:24:24,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:24,839 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01b1ce46d8ec186ac688/lava-logs.yaml 2026-03-09 17:24:24,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:24:24,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a9ce46d8ec186ac670 HTTP/1.1" 200 None 2026-03-09 17:24:24,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:24,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:24,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:24,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a9ce46d8ec186ac670 HTTP/1.1" 200 None 2026-03-09 17:24:24,998 - lava_callback - INFO - Completed processing callback for node 69af01a9ce46d8ec186ac670 2026-03-09 17:24:25,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:24:25,099 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01b1ce46d8ec186ac688/lava_callback.json.gz DEBUG: Setup failed for 69af01b1ce46d8ec186ac688. Transitting job node result: incomplete -> fail 2026-03-09 17:24:25,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:25,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01b1ce46d8ec186ac688 HTTP/1.1" 200 None 2026-03-09 17:24:25,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:25,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:24:25,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:25,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01b1ce46d8ec186ac688 HTTP/1.1" 200 None 2026-03-09 17:24:25,471 - lava_callback - INFO - Completed processing callback for node 69af01b1ce46d8ec186ac688 2026-03-09 17:24:26,247 - lava_callback - INFO - Received callback for node 69ac5f4646f7ae8ff3365a91 INFO: 172.21.0.1:34748 - "POST /node/69ac5f4646f7ae8ff3365a91 HTTP/1.0" 202 Accepted 2026-03-09 17:24:26,256 - lava_callback - INFO - Starting async job submit for node 69ac5f4646f7ae8ff3365a91 2026-03-09 17:24:26,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:26,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5f4646f7ae8ff3365a91 HTTP/1.1" 200 None 2026-03-09 17:24:26,286 - lava_callback - DEBUG - Processing node 69ac5f4646f7ae8ff3365a91: baseline-arm64-baylibre 2026-03-09 17:24:26,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5f4646f7ae8ff3365a91 2026-03-09 17:24:26,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:26,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5f4646f7ae8ff3365a91 HTTP/1.1" 200 None 2026-03-09 17:24:26,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:26,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70a64e31b554803095c1a HTTP/1.1" 200 None 2026-03-09 17:24:26,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:26,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5f4646f7ae8ff3365a91 HTTP/1.1" 200 None 2026-03-09 17:24:26,377 - lava_callback - ERROR - Error processing callback for node 69ac5f4646f7ae8ff3365a91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34224 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:24:44,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:24:44,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58812 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:25:03,393 - lava_callback - INFO - Received callback for node 69af01c1ce46d8ec186ac6ca INFO: 172.21.0.1:60404 - "POST /node/69af01c1ce46d8ec186ac6ca HTTP/1.0" 202 Accepted 2026-03-09 17:25:03,400 - lava_callback - INFO - Starting async job submit for node 69af01c1ce46d8ec186ac6ca 2026-03-09 17:25:03,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:03,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c1ce46d8ec186ac6ca HTTP/1.1" 200 None 2026-03-09 17:25:03,434 - lava_callback - DEBUG - Processing node 69af01c1ce46d8ec186ac6ca: baseline-arm64-broonie 2026-03-09 17:25:03,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:03,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:03,918 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c1ce46d8ec186ac6ca/log.txt.gz 2026-03-09 17:25:04,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:04,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:04,330 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c1ce46d8ec186ac6ca/lava-logs.yaml 2026-03-09 17:25:04,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:04,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:04,486 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c1ce46d8ec186ac6ca/lava_callback.json.gz 2026-03-09 17:25:04,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:04,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c1ce46d8ec186ac6ca HTTP/1.1" 200 None 2026-03-09 17:25:04,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:04,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:25:04,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:04,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c1ce46d8ec186ac6ca HTTP/1.1" 200 None 2026-03-09 17:25:04,616 - lava_callback - INFO - Completed processing callback for node 69af01c1ce46d8ec186ac6ca INFO: 172.21.0.1:60410 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:25:09,059 - lava_callback - INFO - Received callback for node 69af0190ce46d8ec186ac616 INFO: 172.21.0.1:60412 - "POST /node/69af0190ce46d8ec186ac616 HTTP/1.0" 202 Accepted 2026-03-09 17:25:09,065 - lava_callback - INFO - Starting async job submit for node 69af0190ce46d8ec186ac616 2026-03-09 17:25:09,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:09,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0190ce46d8ec186ac616 HTTP/1.1" 200 None 2026-03-09 17:25:09,111 - lava_callback - DEBUG - Processing node 69af0190ce46d8ec186ac616: baseline-arm64 2026-03-09 17:25:09,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:09,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:09,650 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0190ce46d8ec186ac616/log.txt.gz 2026-03-09 17:25:09,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:10,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:10,066 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0190ce46d8ec186ac616/lava-logs.yaml 2026-03-09 17:25:10,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:10,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:10,206 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0190ce46d8ec186ac616/lava_callback.json.gz DEBUG: Setup failed for 69af0190ce46d8ec186ac616. Transitting job node result: incomplete -> fail 2026-03-09 17:25:10,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:10,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0190ce46d8ec186ac616 HTTP/1.1" 200 None 2026-03-09 17:25:10,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:10,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:25:10,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:10,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0190ce46d8ec186ac616 HTTP/1.1" 200 None 2026-03-09 17:25:10,386 - lava_callback - INFO - Completed processing callback for node 69af0190ce46d8ec186ac616 2026-03-09 17:25:11,596 - lava_callback - INFO - Received callback for node 69af01afce46d8ec186ac681 INFO: 172.21.0.1:35462 - "POST /node/69af01afce46d8ec186ac681 HTTP/1.0" 202 Accepted 2026-03-09 17:25:11,602 - lava_callback - INFO - Starting async job submit for node 69af01afce46d8ec186ac681 2026-03-09 17:25:11,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:11,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01afce46d8ec186ac681 HTTP/1.1" 200 None 2026-03-09 17:25:11,653 - lava_callback - DEBUG - Processing node 69af01afce46d8ec186ac681: baseline-nfs-arm64 2026-03-09 17:25:12,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:12,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:12,176 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01afce46d8ec186ac681/log.txt.gz 2026-03-09 17:25:12,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:12,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:12,625 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01afce46d8ec186ac681/lava-logs.yaml 2026-03-09 17:25:12,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:12,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:12,773 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01afce46d8ec186ac681/lava_callback.json.gz DEBUG: Setup failed for 69af01afce46d8ec186ac681. Transitting job node result: incomplete -> fail 2026-03-09 17:25:12,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:12,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01afce46d8ec186ac681 HTTP/1.1" 200 None 2026-03-09 17:25:12,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:12,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:25:12,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:12,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01afce46d8ec186ac681 HTTP/1.1" 200 None 2026-03-09 17:25:12,960 - lava_callback - INFO - Completed processing callback for node 69af01afce46d8ec186ac681 2026-03-09 17:25:14,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:14,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35486 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:25:26,230 - lava_callback - INFO - Received callback for node 69ac5f8246f7ae8ff3365a9e INFO: 172.21.0.1:38966 - "POST /node/69ac5f8246f7ae8ff3365a9e HTTP/1.0" 202 Accepted 2026-03-09 17:25:26,235 - lava_callback - INFO - Starting async job submit for node 69ac5f8246f7ae8ff3365a9e 2026-03-09 17:25:26,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:26,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5f8246f7ae8ff3365a9e HTTP/1.1" 200 None 2026-03-09 17:25:26,251 - lava_callback - DEBUG - Processing node 69ac5f8246f7ae8ff3365a9e: baseline-arm-baylibre 2026-03-09 17:25:26,252 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac5f8246f7ae8ff3365a9e 2026-03-09 17:25:26,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:26,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac5f8246f7ae8ff3365a9e HTTP/1.1" 200 None 2026-03-09 17:25:26,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:26,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None 2026-03-09 17:25:26,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:26,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac5f8246f7ae8ff3365a9e HTTP/1.1" 200 None 2026-03-09 17:25:26,337 - lava_callback - ERROR - Error processing callback for node 69ac5f8246f7ae8ff3365a9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:25:45,399 - lava_callback - INFO - Received callback for node 69af01a3ce46d8ec186ac65f INFO: 172.21.0.1:55494 - "POST /node/69af01a3ce46d8ec186ac65f HTTP/1.0" 202 Accepted 2026-03-09 17:25:45,406 - lava_callback - INFO - Starting async job submit for node 69af01a3ce46d8ec186ac65f 2026-03-09 17:25:45,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:45,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a3ce46d8ec186ac65f HTTP/1.1" 200 None 2026-03-09 17:25:45,456 - lava_callback - DEBUG - Processing node 69af01a3ce46d8ec186ac65f: baseline-nfs-arm64 2026-03-09 17:25:45,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:45,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:45,970 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a3ce46d8ec186ac65f/log.txt.gz 2026-03-09 17:25:46,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:25:46,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:46,431 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a3ce46d8ec186ac65f/lava-logs.yaml 2026-03-09 17:25:46,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:55498 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:25:46,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:25:46,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a3ce46d8ec186ac65f/lava_callback.json.gz DEBUG: Setup failed for 69af01a3ce46d8ec186ac65f. Transitting job node result: incomplete -> fail 2026-03-09 17:25:46,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:46,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a3ce46d8ec186ac65f HTTP/1.1" 200 None 2026-03-09 17:25:46,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:46,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:25:46,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:25:46,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a3ce46d8ec186ac65f HTTP/1.1" 200 None 2026-03-09 17:25:46,835 - lava_callback - INFO - Completed processing callback for node 69af01a3ce46d8ec186ac65f INFO: 172.21.0.1:39296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50804 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:26:14,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:14,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50940 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:26:20,703 - lava_callback - INFO - Received callback for node 69af019fce46d8ec186ac655 INFO: 172.21.0.1:40100 - "POST /node/69af019fce46d8ec186ac655 HTTP/1.0" 202 Accepted 2026-03-09 17:26:20,709 - lava_callback - INFO - Starting async job submit for node 69af019fce46d8ec186ac655 2026-03-09 17:26:20,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:20,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019fce46d8ec186ac655 HTTP/1.1" 200 None 2026-03-09 17:26:20,800 - lava_callback - DEBUG - Processing node 69af019fce46d8ec186ac655: baseline-arm64 2026-03-09 17:26:21,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:26:21,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:26:21,264 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019fce46d8ec186ac655/log.txt.gz 2026-03-09 17:26:21,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:26:21,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:26:21,657 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019fce46d8ec186ac655/lava-logs.yaml 2026-03-09 17:26:21,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:26:21,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:26:21,827 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019fce46d8ec186ac655/lava_callback.json.gz DEBUG: Setup failed for 69af019fce46d8ec186ac655. Transitting job node result: incomplete -> fail 2026-03-09 17:26:21,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:21,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019fce46d8ec186ac655 HTTP/1.1" 200 None 2026-03-09 17:26:21,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:21,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:26:21,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:21,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af019fce46d8ec186ac655 HTTP/1.1" 200 None 2026-03-09 17:26:22,029 - lava_callback - INFO - Completed processing callback for node 69af019fce46d8ec186ac655 INFO: 172.21.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51816 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:26:44,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:26:44,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51124 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:28:45,121 - lava_callback - INFO - Received callback for node 69af01bfce46d8ec186ac6c5 INFO: 172.21.0.1:39584 - "POST /node/69af01bfce46d8ec186ac6c5 HTTP/1.0" 202 Accepted 2026-03-09 17:28:45,136 - lava_callback - INFO - Starting async job submit for node 69af01bfce46d8ec186ac6c5 2026-03-09 17:28:45,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:45,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01bfce46d8ec186ac6c5 HTTP/1.1" 200 None 2026-03-09 17:28:45,241 - lava_callback - DEBUG - Processing node 69af01bfce46d8ec186ac6c5: baseline-arm64-broonie 2026-03-09 17:28:45,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:28:45,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:28:45,767 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bfce46d8ec186ac6c5/log.txt.gz 2026-03-09 17:28:45,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:28:46,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:28:46,223 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bfce46d8ec186ac6c5/lava-logs.yaml 2026-03-09 17:28:46,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:28:46,426 - lava_callback - INFO - Received callback for node 69ac603a46f7ae8ff3365aed INFO: 172.21.0.1:39592 - "POST /node/69ac603a46f7ae8ff3365aed HTTP/1.0" 202 Accepted 2026-03-09 17:28:46,432 - lava_callback - INFO - Starting async job submit for node 69ac603a46f7ae8ff3365aed 2026-03-09 17:28:46,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:28:46,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,436 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bfce46d8ec186ac6c5/lava_callback.json.gz 2026-03-09 17:28:46,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac603a46f7ae8ff3365aed HTTP/1.1" 200 None DEBUG: Setup failed for 69af01bfce46d8ec186ac6c5. Transitting job node result: incomplete -> fail 2026-03-09 17:28:46,518 - lava_callback - DEBUG - Processing node 69ac603a46f7ae8ff3365aed: baseline-arm-baylibre 2026-03-09 17:28:46,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac603a46f7ae8ff3365aed 2026-03-09 17:28:46,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:39602 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:28:46,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01bfce46d8ec186ac6c5 HTTP/1.1" 200 None 2026-03-09 17:28:46,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac603a46f7ae8ff3365aed HTTP/1.1" 200 None 2026-03-09 17:28:46,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:28:46,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None 2026-03-09 17:28:46,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:28:46,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01bfce46d8ec186ac6c5 HTTP/1.1" 200 None 2026-03-09 17:28:46,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac603a46f7ae8ff3365aed HTTP/1.1" 200 None 2026-03-09 17:28:46,646 - lava_callback - ERROR - Error processing callback for node 69ac603a46f7ae8ff3365aed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 17:28:46,774 - lava_callback - INFO - Completed processing callback for node 69af01bfce46d8ec186ac6c5 INFO: 172.21.0.1:49706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49722 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:29:06,409 - lava_callback - INFO - Received callback for node 69ac606046f7ae8ff3365b05 INFO: 172.21.0.1:39574 - "POST /node/69ac606046f7ae8ff3365b05 HTTP/1.0" 202 Accepted 2026-03-09 17:29:06,415 - lava_callback - INFO - Starting async job submit for node 69ac606046f7ae8ff3365b05 2026-03-09 17:29:06,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:29:06,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac606046f7ae8ff3365b05 HTTP/1.1" 200 None 2026-03-09 17:29:06,440 - lava_callback - DEBUG - Processing node 69ac606046f7ae8ff3365b05: baseline-arm-baylibre 2026-03-09 17:29:06,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac606046f7ae8ff3365b05 2026-03-09 17:29:06,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:29:06,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac606046f7ae8ff3365b05 HTTP/1.1" 200 None 2026-03-09 17:29:06,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:29:06,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None INFO: 172.21.0.1:39588 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:29:06,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:29:06,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac606046f7ae8ff3365b05 HTTP/1.1" 200 None 2026-03-09 17:29:06,542 - lava_callback - ERROR - Error processing callback for node 69ac606046f7ae8ff3365b05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 17:29:14,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:29:14,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44186 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:30:26,551 - lava_callback - INFO - Received callback for node 69ac60af46f7ae8ff3365b35 INFO: 172.21.0.1:44190 - "POST /node/69ac60af46f7ae8ff3365b35 HTTP/1.0" 202 Accepted 2026-03-09 17:30:26,556 - lava_callback - INFO - Starting async job submit for node 69ac60af46f7ae8ff3365b35 2026-03-09 17:30:26,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:26,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac60af46f7ae8ff3365b35 HTTP/1.1" 200 None 2026-03-09 17:30:26,572 - lava_callback - DEBUG - Processing node 69ac60af46f7ae8ff3365b35: baseline-arm-baylibre 2026-03-09 17:30:26,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac60af46f7ae8ff3365b35 2026-03-09 17:30:26,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:26,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac60af46f7ae8ff3365b35 HTTP/1.1" 200 None 2026-03-09 17:30:26,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:26,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None 2026-03-09 17:30:26,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:26,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac60af46f7ae8ff3365b35 HTTP/1.1" 200 None 2026-03-09 17:30:26,638 - lava_callback - ERROR - Error processing callback for node 69ac60af46f7ae8ff3365b35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50450 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:30:43,827 - lava_callback - INFO - Received callback for node 69af01c5ce46d8ec186ac6d3 INFO: 172.21.0.1:35374 - "POST /node/69af01c5ce46d8ec186ac6d3 HTTP/1.0" 202 Accepted 2026-03-09 17:30:43,833 - lava_callback - INFO - Starting async job submit for node 69af01c5ce46d8ec186ac6d3 2026-03-09 17:30:43,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:43,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c5ce46d8ec186ac6d3 HTTP/1.1" 200 None 2026-03-09 17:30:43,888 - lava_callback - DEBUG - Processing node 69af01c5ce46d8ec186ac6d3: baseline-arm64-broonie 2026-03-09 17:30:44,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:30:44,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:30:44,373 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c5ce46d8ec186ac6d3/log.txt.gz 2026-03-09 17:30:44,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:30:44,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:30:44,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c5ce46d8ec186ac6d3/lava-logs.yaml 2026-03-09 17:30:44,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:30:44,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:30:44,984 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c5ce46d8ec186ac6d3/lava_callback.json.gz DEBUG: Setup failed for 69af01c5ce46d8ec186ac6d3. Transitting job node result: incomplete -> fail 2026-03-09 17:30:45,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:45,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c5ce46d8ec186ac6d3 HTTP/1.1" 200 None 2026-03-09 17:30:45,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:45,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:30:45,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:30:45,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c5ce46d8ec186ac6d3 HTTP/1.1" 200 None 2026-03-09 17:30:45,208 - lava_callback - INFO - Completed processing callback for node 69af01c5ce46d8ec186ac6d3 INFO: 172.21.0.1:35378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52606 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:31:14,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:14,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50910 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:31:32,506 - lava_callback - INFO - Received callback for node 69af03eace46d8ec186ac862 INFO: 172.21.0.1:34926 - "POST /node/69af03eace46d8ec186ac862 HTTP/1.0" 202 Accepted 2026-03-09 17:31:32,512 - lava_callback - INFO - Starting async job submit for node 69af03eace46d8ec186ac862 2026-03-09 17:31:32,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:32,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af03eace46d8ec186ac862 HTTP/1.1" 200 None 2026-03-09 17:31:32,531 - lava_callback - DEBUG - Processing node 69af03eace46d8ec186ac862: baseline-arm64 2026-03-09 17:31:32,536 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:31:32,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:31:32,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:31:32,693 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af03eace46d8ec186ac862/lava-logs.yaml 2026-03-09 17:31:32,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:31:32,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:31:32,805 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af03eace46d8ec186ac862/lava_callback.json.gz 2026-03-09 17:31:32,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:32,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af03eace46d8ec186ac862 HTTP/1.1" 200 None 2026-03-09 17:31:32,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:32,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:31:32,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:32,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af03eace46d8ec186ac862 HTTP/1.1" 200 None 2026-03-09 17:31:32,882 - lava_callback - INFO - Completed processing callback for node 69af03eace46d8ec186ac862 INFO: 172.21.0.1:34928 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:31:44,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:31:44,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34792 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:32:26,667 - lava_callback - INFO - Received callback for node 69ac612046f7ae8ff3365b68 INFO: 172.21.0.1:34806 - "POST /node/69ac612046f7ae8ff3365b68 HTTP/1.0" 202 Accepted 2026-03-09 17:32:26,673 - lava_callback - INFO - Starting async job submit for node 69ac612046f7ae8ff3365b68 2026-03-09 17:32:26,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:26,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac612046f7ae8ff3365b68 HTTP/1.1" 200 None 2026-03-09 17:32:26,688 - lava_callback - DEBUG - Processing node 69ac612046f7ae8ff3365b68: baseline-arm-baylibre 2026-03-09 17:32:26,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac612046f7ae8ff3365b68 2026-03-09 17:32:26,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:26,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac612046f7ae8ff3365b68 HTTP/1.1" 200 None 2026-03-09 17:32:26,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:26,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None 2026-03-09 17:32:26,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:26,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac612046f7ae8ff3365b68 HTTP/1.1" 200 None 2026-03-09 17:32:26,764 - lava_callback - ERROR - Error processing callback for node 69ac612046f7ae8ff3365b68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 17:32:30,101 - lava_callback - INFO - Received callback for node 69af0195ce46d8ec186ac622 INFO: 172.21.0.1:45448 - "POST /node/69af0195ce46d8ec186ac622 HTTP/1.0" 202 Accepted 2026-03-09 17:32:30,114 - lava_callback - INFO - Starting async job submit for node 69af0195ce46d8ec186ac622 2026-03-09 17:32:30,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:30,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0195ce46d8ec186ac622 HTTP/1.1" 200 None 2026-03-09 17:32:30,173 - lava_callback - DEBUG - Processing node 69af0195ce46d8ec186ac622: baseline-arm64 2026-03-09 17:32:30,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:30,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:30,629 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0195ce46d8ec186ac622/log.txt.gz 2026-03-09 17:32:30,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:31,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:31,052 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0195ce46d8ec186ac622/lava-logs.yaml 2026-03-09 17:32:31,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:31,292 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:31,294 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0195ce46d8ec186ac622/lava_callback.json.gz 2026-03-09 17:32:31,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:31,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0195ce46d8ec186ac622 HTTP/1.1" 200 None 2026-03-09 17:32:31,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:31,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:32:31,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:31,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0195ce46d8ec186ac622 HTTP/1.1" 200 None 2026-03-09 17:32:31,452 - lava_callback - INFO - Completed processing callback for node 69af0195ce46d8ec186ac622 INFO: 172.21.0.1:45456 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:32:38,198 - lava_callback - INFO - Received callback for node 69af019cce46d8ec186ac634 INFO: 172.21.0.1:45470 - "POST /node/69af019cce46d8ec186ac634 HTTP/1.0" 202 Accepted 2026-03-09 17:32:38,204 - lava_callback - INFO - Starting async job submit for node 69af019cce46d8ec186ac634 2026-03-09 17:32:38,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:38,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019cce46d8ec186ac634 HTTP/1.1" 200 None 2026-03-09 17:32:38,243 - lava_callback - DEBUG - Processing node 69af019cce46d8ec186ac634: baseline-arm64 2026-03-09 17:32:38,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:38,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:38,934 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019cce46d8ec186ac634/log.txt.gz 2026-03-09 17:32:39,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:39,426 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:39,428 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019cce46d8ec186ac634/lava-logs.yaml 2026-03-09 17:32:39,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:39,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:39,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af019cce46d8ec186ac634/lava_callback.json.gz 2026-03-09 17:32:39,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:39,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af019cce46d8ec186ac634 HTTP/1.1" 200 None 2026-03-09 17:32:39,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:39,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:32:39,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:39,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af019cce46d8ec186ac634 HTTP/1.1" 200 None 2026-03-09 17:32:39,754 - lava_callback - INFO - Completed processing callback for node 69af019cce46d8ec186ac634 2026-03-09 17:32:44,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:44,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:32:46,778 - lava_callback - INFO - Received callback for node 69af01adce46d8ec186ac679 INFO: 172.21.0.1:43968 - "POST /node/69af01adce46d8ec186ac679 HTTP/1.0" 202 Accepted 2026-03-09 17:32:46,785 - lava_callback - INFO - Starting async job submit for node 69af01adce46d8ec186ac679 2026-03-09 17:32:46,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:46,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01adce46d8ec186ac679 HTTP/1.1" 200 None 2026-03-09 17:32:46,825 - lava_callback - DEBUG - Processing node 69af01adce46d8ec186ac679: baseline-nfs-arm64 2026-03-09 17:32:47,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:47,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:47,453 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01adce46d8ec186ac679/log.txt.gz 2026-03-09 17:32:47,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:47,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:47,922 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01adce46d8ec186ac679/lava-logs.yaml 2026-03-09 17:32:47,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:32:48,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:32:48,100 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01adce46d8ec186ac679/lava_callback.json.gz 2026-03-09 17:32:48,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:48,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01adce46d8ec186ac679 HTTP/1.1" 200 None 2026-03-09 17:32:48,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:48,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:32:48,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:32:48,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01adce46d8ec186ac679 HTTP/1.1" 200 None 2026-03-09 17:32:48,250 - lava_callback - INFO - Completed processing callback for node 69af01adce46d8ec186ac679 INFO: 172.21.0.1:50366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50374 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:33:01,182 - lava_callback - INFO - Received callback for node 69af0437ce46d8ec186ac876 INFO: 172.21.0.1:39358 - "POST /node/69af0437ce46d8ec186ac876 HTTP/1.0" 202 Accepted 2026-03-09 17:33:01,188 - lava_callback - INFO - Starting async job submit for node 69af0437ce46d8ec186ac876 2026-03-09 17:33:01,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:01,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0437ce46d8ec186ac876 HTTP/1.1" 200 None 2026-03-09 17:33:01,206 - lava_callback - DEBUG - Processing node 69af0437ce46d8ec186ac876: baseline-arm64 2026-03-09 17:33:01,210 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:33:01,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:01,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:01,340 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0437ce46d8ec186ac876/lava-logs.yaml 2026-03-09 17:33:01,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:01,451 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:01,453 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0437ce46d8ec186ac876/lava_callback.json.gz 2026-03-09 17:33:01,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:01,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0437ce46d8ec186ac876 HTTP/1.1" 200 None 2026-03-09 17:33:01,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:01,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:33:01,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:01,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0437ce46d8ec186ac876 HTTP/1.1" 200 None 2026-03-09 17:33:01,546 - lava_callback - INFO - Completed processing callback for node 69af0437ce46d8ec186ac876 2026-03-09 17:33:04,111 - lava_callback - INFO - Received callback for node 69af01a5ce46d8ec186ac665 INFO: 172.21.0.1:39368 - "POST /node/69af01a5ce46d8ec186ac665 HTTP/1.0" 202 Accepted 2026-03-09 17:33:04,118 - lava_callback - INFO - Starting async job submit for node 69af01a5ce46d8ec186ac665 2026-03-09 17:33:04,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:04,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a5ce46d8ec186ac665 HTTP/1.1" 200 None 2026-03-09 17:33:04,170 - lava_callback - DEBUG - Processing node 69af01a5ce46d8ec186ac665: baseline-nfs-arm64 2026-03-09 17:33:04,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:04,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:04,598 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a5ce46d8ec186ac665/log.txt.gz 2026-03-09 17:33:04,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:04,967 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:04,969 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a5ce46d8ec186ac665/lava-logs.yaml 2026-03-09 17:33:04,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:05,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:05,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a5ce46d8ec186ac665/lava_callback.json.gz 2026-03-09 17:33:05,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:05,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a5ce46d8ec186ac665 HTTP/1.1" 200 None 2026-03-09 17:33:05,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:05,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:33:05,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:05,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a5ce46d8ec186ac665 HTTP/1.1" 200 None 2026-03-09 17:33:05,313 - lava_callback - INFO - Completed processing callback for node 69af01a5ce46d8ec186ac665 INFO: 172.21.0.1:39372 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:33:10,058 - lava_callback - INFO - Received callback for node 69af003fce46d8ec186ac4ca INFO: 172.21.0.1:53218 - "POST /node/69af003fce46d8ec186ac4ca HTTP/1.0" 202 Accepted 2026-03-09 17:33:10,065 - lava_callback - INFO - Starting async job submit for node 69af003fce46d8ec186ac4ca 2026-03-09 17:33:10,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:10,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af003fce46d8ec186ac4ca HTTP/1.1" 200 None 2026-03-09 17:33:10,139 - lava_callback - DEBUG - Processing node 69af003fce46d8ec186ac4ca: ltp-smoke 2026-03-09 17:33:10,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:10,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:10,775 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af003fce46d8ec186ac4ca/log.txt.gz 2026-03-09 17:33:11,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:11,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:11,266 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af003fce46d8ec186ac4ca/lava-logs.yaml 2026-03-09 17:33:11,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:33:11,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:33:11,438 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af003fce46d8ec186ac4ca/lava_callback.json.gz 2026-03-09 17:33:11,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:11,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af003fce46d8ec186ac4ca HTTP/1.1" 200 None 2026-03-09 17:33:11,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:11,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef42ace46d8ec186ab814 HTTP/1.1" 200 None 2026-03-09 17:33:11,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:11,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af003fce46d8ec186ac4ca HTTP/1.1" 200 None 2026-03-09 17:33:11,737 - lava_callback - INFO - Completed processing callback for node 69af003fce46d8ec186ac4ca 2026-03-09 17:33:14,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:33:14,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51822 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:34:26,735 - lava_callback - INFO - Received callback for node 69ac619146f7ae8ff3365b93 INFO: 172.21.0.1:51834 - "POST /node/69ac619146f7ae8ff3365b93 HTTP/1.0" 202 Accepted 2026-03-09 17:34:26,741 - lava_callback - INFO - Starting async job submit for node 69ac619146f7ae8ff3365b93 2026-03-09 17:34:26,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:26,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac619146f7ae8ff3365b93 HTTP/1.1" 200 None 2026-03-09 17:34:26,756 - lava_callback - DEBUG - Processing node 69ac619146f7ae8ff3365b93: baseline-arm-baylibre 2026-03-09 17:34:26,757 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac619146f7ae8ff3365b93 2026-03-09 17:34:26,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:26,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac619146f7ae8ff3365b93 HTTP/1.1" 200 None 2026-03-09 17:34:26,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:26,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46664a327b9815668f9bb HTTP/1.1" 200 None 2026-03-09 17:34:26,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:26,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac619146f7ae8ff3365b93 HTTP/1.1" 200 None 2026-03-09 17:34:26,843 - lava_callback - ERROR - Error processing callback for node 69ac619146f7ae8ff3365b93: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60420 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:34:50,384 - lava_callback - INFO - Received callback for node 69af01cace46d8ec186ac6e5 INFO: 172.21.0.1:36684 - "POST /node/69af01cace46d8ec186ac6e5 HTTP/1.0" 202 Accepted 2026-03-09 17:34:50,390 - lava_callback - INFO - Starting async job submit for node 69af01cace46d8ec186ac6e5 2026-03-09 17:34:50,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:50,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01cace46d8ec186ac6e5 HTTP/1.1" 200 None 2026-03-09 17:34:50,408 - lava_callback - DEBUG - Processing node 69af01cace46d8ec186ac6e5: baseline-arm64-broonie 2026-03-09 17:34:50,413 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:34:50,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:34:50,566 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:34:50,568 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01cace46d8ec186ac6e5/lava-logs.yaml 2026-03-09 17:34:50,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:34:50,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:34:50,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01cace46d8ec186ac6e5/lava_callback.json.gz 2026-03-09 17:34:50,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:50,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01cace46d8ec186ac6e5 HTTP/1.1" 200 None 2026-03-09 17:34:50,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:50,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:34:50,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:34:50,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01cace46d8ec186ac6e5 HTTP/1.1" 200 None 2026-03-09 17:34:50,749 - lava_callback - INFO - Completed processing callback for node 69af01cace46d8ec186ac6e5 INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:35:14,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:35:14,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44376 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:36:50,335 - lava_callback - INFO - Received callback for node 69af01c4ce46d8ec186ac6d0 INFO: 172.21.0.1:60544 - "POST /node/69af01c4ce46d8ec186ac6d0 HTTP/1.0" 202 Accepted 2026-03-09 17:36:50,341 - lava_callback - INFO - Starting async job submit for node 69af01c4ce46d8ec186ac6d0 2026-03-09 17:36:50,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:36:50,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c4ce46d8ec186ac6d0 HTTP/1.1" 200 None 2026-03-09 17:36:50,393 - lava_callback - DEBUG - Processing node 69af01c4ce46d8ec186ac6d0: baseline-arm64-broonie 2026-03-09 17:36:50,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:36:51,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:36:51,006 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c4ce46d8ec186ac6d0/log.txt.gz 2026-03-09 17:36:51,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:36:51,486 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:36:51,488 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c4ce46d8ec186ac6d0/lava-logs.yaml 2026-03-09 17:36:51,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:36:51,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:36:51,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c4ce46d8ec186ac6d0/lava_callback.json.gz DEBUG: Setup failed for 69af01c4ce46d8ec186ac6d0. Transitting job node result: incomplete -> fail 2026-03-09 17:36:51,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:36:51,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c4ce46d8ec186ac6d0 HTTP/1.1" 200 None 2026-03-09 17:36:51,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:36:51,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:36:51,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:36:51,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c4ce46d8ec186ac6d0 HTTP/1.1" 200 None 2026-03-09 17:36:51,885 - lava_callback - INFO - Completed processing callback for node 69af01c4ce46d8ec186ac6d0 INFO: 172.21.0.1:60560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60560 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:37:13,886 - lava_callback - INFO - Received callback for node 69af0528ce46d8ec186ac8b3 INFO: 172.21.0.1:56490 - "POST /node/69af0528ce46d8ec186ac8b3 HTTP/1.0" 202 Accepted 2026-03-09 17:37:13,891 - lava_callback - INFO - Starting async job submit for node 69af0528ce46d8ec186ac8b3 2026-03-09 17:37:13,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:37:13,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0528ce46d8ec186ac8b3 HTTP/1.1" 200 None 2026-03-09 17:37:13,909 - lava_callback - DEBUG - Processing node 69af0528ce46d8ec186ac8b3: baseline-arm64 2026-03-09 17:37:13,914 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:37:13,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:37:14,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:37:14,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0528ce46d8ec186ac8b3/lava-logs.yaml 2026-03-09 17:37:14,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:37:14,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:37:14,214 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0528ce46d8ec186ac8b3/lava_callback.json.gz 2026-03-09 17:37:14,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:37:14,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0528ce46d8ec186ac8b3 HTTP/1.1" 200 None 2026-03-09 17:37:14,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:37:14,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:37:14,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:37:14,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0528ce46d8ec186ac8b3 HTTP/1.1" 200 None 2026-03-09 17:37:14,283 - lava_callback - INFO - Completed processing callback for node 69af0528ce46d8ec186ac8b3 2026-03-09 17:37:14,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:37:14,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52258 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:38:22,664 - lava_callback - INFO - Received callback for node 69af0156ce46d8ec186ac5c9 INFO: 172.21.0.1:51472 - "POST /node/69af0156ce46d8ec186ac5c9 HTTP/1.0" 202 Accepted 2026-03-09 17:38:22,670 - lava_callback - INFO - Starting async job submit for node 69af0156ce46d8ec186ac5c9 2026-03-09 17:38:22,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:22,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0156ce46d8ec186ac5c9 HTTP/1.1" 200 None 2026-03-09 17:38:22,738 - lava_callback - DEBUG - Processing node 69af0156ce46d8ec186ac5c9: baseline-arm 2026-03-09 17:38:23,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:23,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:23,484 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af0156ce46d8ec186ac5c9/log.txt.gz 2026-03-09 17:38:23,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:24,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:24,084 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af0156ce46d8ec186ac5c9/lava-logs.yaml 2026-03-09 17:38:24,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:24,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:24,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af0156ce46d8ec186ac5c9/lava_callback.json.gz DEBUG: Setup failed for 69af0156ce46d8ec186ac5c9. Transitting job node result: incomplete -> fail 2026-03-09 17:38:24,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:24,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0156ce46d8ec186ac5c9 HTTP/1.1" 200 None 2026-03-09 17:38:24,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:24,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:38:24,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:24,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0156ce46d8ec186ac5c9 HTTP/1.1" 200 None 2026-03-09 17:38:24,616 - lava_callback - INFO - Completed processing callback for node 69af0156ce46d8ec186ac5c9 INFO: 172.21.0.1:51484 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:38:30,821 - lava_callback - INFO - Received callback for node 69af0194ce46d8ec186ac61f INFO: 172.21.0.1:52838 - "POST /node/69af0194ce46d8ec186ac61f HTTP/1.0" 202 Accepted 2026-03-09 17:38:30,827 - lava_callback - INFO - Starting async job submit for node 69af0194ce46d8ec186ac61f 2026-03-09 17:38:30,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:30,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0194ce46d8ec186ac61f HTTP/1.1" 200 None 2026-03-09 17:38:30,858 - lava_callback - DEBUG - Processing node 69af0194ce46d8ec186ac61f: baseline-arm64 2026-03-09 17:38:31,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:31,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:31,341 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0194ce46d8ec186ac61f/log.txt.gz 2026-03-09 17:38:31,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:31,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:31,754 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0194ce46d8ec186ac61f/lava-logs.yaml 2026-03-09 17:38:31,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:38:31,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:38:31,917 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0194ce46d8ec186ac61f/lava_callback.json.gz 2026-03-09 17:38:31,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:31,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0194ce46d8ec186ac61f HTTP/1.1" 200 None 2026-03-09 17:38:31,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:31,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:38:31,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:31,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0194ce46d8ec186ac61f HTTP/1.1" 200 None 2026-03-09 17:38:31,998 - lava_callback - INFO - Completed processing callback for node 69af0194ce46d8ec186ac61f INFO: 172.21.0.1:52850 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:38:44,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:38:44,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59968 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:39:38,272 - lava_callback - INFO - Received callback for node 69af05bdce46d8ec186ac8de INFO: 172.21.0.1:59970 - "POST /node/69af05bdce46d8ec186ac8de HTTP/1.0" 202 Accepted 2026-03-09 17:39:38,278 - lava_callback - INFO - Starting async job submit for node 69af05bdce46d8ec186ac8de 2026-03-09 17:39:38,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:39:38,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af05bdce46d8ec186ac8de HTTP/1.1" 200 None 2026-03-09 17:39:38,317 - lava_callback - DEBUG - Processing node 69af05bdce46d8ec186ac8de: baseline-arm64 2026-03-09 17:39:38,366 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:39:38,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:39:38,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:39:38,573 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af05bdce46d8ec186ac8de/lava-logs.yaml 2026-03-09 17:39:38,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:39:38,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:39:38,709 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af05bdce46d8ec186ac8de/lava_callback.json.gz 2026-03-09 17:39:38,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:39:38,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af05bdce46d8ec186ac8de HTTP/1.1" 200 None 2026-03-09 17:39:38,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:39:38,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:39:38,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:39:38,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af05bdce46d8ec186ac8de HTTP/1.1" 200 None 2026-03-09 17:39:38,844 - lava_callback - INFO - Completed processing callback for node 69af05bdce46d8ec186ac8de 2026-03-09 17:39:44,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:39:44,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54722 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:40:08,386 - lava_callback - INFO - Received callback for node 69af01a4ce46d8ec186ac662 INFO: 172.21.0.1:54726 - "POST /node/69af01a4ce46d8ec186ac662 HTTP/1.0" 202 Accepted 2026-03-09 17:40:08,392 - lava_callback - INFO - Starting async job submit for node 69af01a4ce46d8ec186ac662 2026-03-09 17:40:08,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:08,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a4ce46d8ec186ac662 HTTP/1.1" 200 None 2026-03-09 17:40:08,430 - lava_callback - DEBUG - Processing node 69af01a4ce46d8ec186ac662: baseline-nfs-arm64 2026-03-09 17:40:08,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:40:09,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:40:09,019 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a4ce46d8ec186ac662/log.txt.gz 2026-03-09 17:40:09,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:40:09,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:40:09,466 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a4ce46d8ec186ac662/lava-logs.yaml 2026-03-09 17:40:09,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:40:09,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:40:09,638 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af01a4ce46d8ec186ac662/lava_callback.json.gz 2026-03-09 17:40:09,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:09,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01a4ce46d8ec186ac662 HTTP/1.1" 200 None 2026-03-09 17:40:09,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:09,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:40:09,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:09,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01a4ce46d8ec186ac662 HTTP/1.1" 200 None 2026-03-09 17:40:09,714 - lava_callback - INFO - Completed processing callback for node 69af01a4ce46d8ec186ac662 2026-03-09 17:40:14,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:14,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50184 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:40:55,328 - lava_callback - INFO - Received callback for node 69af0614ce46d8ec186ac8f3 INFO: 172.21.0.1:50582 - "POST /node/69af0614ce46d8ec186ac8f3 HTTP/1.0" 202 Accepted 2026-03-09 17:40:55,334 - lava_callback - INFO - Starting async job submit for node 69af0614ce46d8ec186ac8f3 2026-03-09 17:40:55,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:55,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0614ce46d8ec186ac8f3 HTTP/1.1" 200 None 2026-03-09 17:40:55,368 - lava_callback - DEBUG - Processing node 69af0614ce46d8ec186ac8f3: baseline-nfs-arm64 2026-03-09 17:40:55,369 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:40:55,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:40:55,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:40:55,519 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0614ce46d8ec186ac8f3/lava-logs.yaml 2026-03-09 17:40:55,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:40:55,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:40:55,666 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0614ce46d8ec186ac8f3/lava_callback.json.gz 2026-03-09 17:40:55,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:55,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0614ce46d8ec186ac8f3 HTTP/1.1" 200 None 2026-03-09 17:40:55,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:55,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:40:55,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:40:55,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0614ce46d8ec186ac8f3 HTTP/1.1" 200 None 2026-03-09 17:40:55,734 - lava_callback - INFO - Completed processing callback for node 69af0614ce46d8ec186ac8f3 INFO: 172.21.0.1:50586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55740 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:41:14,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:41:14,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41236 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:42:26,304 - lava_callback - INFO - Received callback for node 69af065ece46d8ec186ac8fb INFO: 172.21.0.1:58138 - "POST /node/69af065ece46d8ec186ac8fb HTTP/1.0" 202 Accepted 2026-03-09 17:42:26,310 - lava_callback - INFO - Starting async job submit for node 69af065ece46d8ec186ac8fb 2026-03-09 17:42:26,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:42:26,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af065ece46d8ec186ac8fb HTTP/1.1" 200 None 2026-03-09 17:42:26,347 - lava_callback - DEBUG - Processing node 69af065ece46d8ec186ac8fb: baseline-arm64 2026-03-09 17:42:26,398 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:42:26,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:58144 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:42:26,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:42:26,601 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af065ece46d8ec186ac8fb/lava-logs.yaml 2026-03-09 17:42:26,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:42:26,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:42:26,745 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af065ece46d8ec186ac8fb/lava_callback.json.gz 2026-03-09 17:42:26,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:42:26,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af065ece46d8ec186ac8fb HTTP/1.1" 200 None 2026-03-09 17:42:26,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:42:26,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:42:26,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:42:26,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af065ece46d8ec186ac8fb HTTP/1.1" 200 None 2026-03-09 17:42:26,863 - lava_callback - INFO - Completed processing callback for node 69af065ece46d8ec186ac8fb INFO: 172.21.0.1:37042 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:42:44,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:42:44,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36764 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:43:39,526 - lava_callback - INFO - Received callback for node 69af06a7ce46d8ec186ac906 INFO: 172.21.0.1:36766 - "POST /node/69af06a7ce46d8ec186ac906 HTTP/1.0" 202 Accepted 2026-03-09 17:43:39,533 - lava_callback - INFO - Starting async job submit for node 69af06a7ce46d8ec186ac906 2026-03-09 17:43:39,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:43:39,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af06a7ce46d8ec186ac906 HTTP/1.1" 200 None 2026-03-09 17:43:39,565 - lava_callback - DEBUG - Processing node 69af06a7ce46d8ec186ac906: baseline-arm64 2026-03-09 17:43:39,613 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:43:39,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:43:39,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:43:39,852 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af06a7ce46d8ec186ac906/lava-logs.yaml 2026-03-09 17:43:39,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:43:39,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:43:39,995 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af06a7ce46d8ec186ac906/lava_callback.json.gz 2026-03-09 17:43:40,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:43:40,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af06a7ce46d8ec186ac906 HTTP/1.1" 200 None 2026-03-09 17:43:40,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:43:40,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:43:40,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:43:40,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af06a7ce46d8ec186ac906 HTTP/1.1" 200 None 2026-03-09 17:43:40,146 - lava_callback - INFO - Completed processing callback for node 69af06a7ce46d8ec186ac906 2026-03-09 17:43:44,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:43:44,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60784 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:44:07,287 - lava_callback - INFO - Received callback for node 69ac63dc46f7ae8ff3365c96 INFO: 172.21.0.1:60792 - "POST /node/69ac63dc46f7ae8ff3365c96 HTTP/1.0" 202 Accepted 2026-03-09 17:44:07,293 - lava_callback - INFO - Starting async job submit for node 69ac63dc46f7ae8ff3365c96 2026-03-09 17:44:07,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:07,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac63dc46f7ae8ff3365c96 HTTP/1.1" 200 None 2026-03-09 17:44:07,308 - lava_callback - DEBUG - Processing node 69ac63dc46f7ae8ff3365c96: baseline-arm-baylibre 2026-03-09 17:44:07,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac63dc46f7ae8ff3365c96 2026-03-09 17:44:07,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:07,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac63dc46f7ae8ff3365c96 HTTP/1.1" 200 None 2026-03-09 17:44:07,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:07,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:44:07,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:07,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac63dc46f7ae8ff3365c96 HTTP/1.1" 200 None 2026-03-09 17:44:07,386 - lava_callback - ERROR - Error processing callback for node 69ac63dc46f7ae8ff3365c96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42798 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:44:47,264 - lava_callback - INFO - Received callback for node 69ac640046f7ae8ff3365ca8 INFO: 172.21.0.1:42800 - "POST /node/69ac640046f7ae8ff3365ca8 HTTP/1.0" 202 Accepted 2026-03-09 17:44:47,269 - lava_callback - INFO - Starting async job submit for node 69ac640046f7ae8ff3365ca8 2026-03-09 17:44:47,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:47,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac640046f7ae8ff3365ca8 HTTP/1.1" 200 None 2026-03-09 17:44:47,286 - lava_callback - DEBUG - Processing node 69ac640046f7ae8ff3365ca8: baseline-arm-baylibre 2026-03-09 17:44:47,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac640046f7ae8ff3365ca8 2026-03-09 17:44:47,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:47,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac640046f7ae8ff3365ca8 HTTP/1.1" 200 None 2026-03-09 17:44:47,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:47,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:44:47,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:47,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac640046f7ae8ff3365ca8 HTTP/1.1" 200 None 2026-03-09 17:44:47,358 - lava_callback - ERROR - Error processing callback for node 69ac640046f7ae8ff3365ca8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 17:44:54,986 - lava_callback - INFO - Received callback for node 69af06fcce46d8ec186ac913 INFO: 172.21.0.1:41768 - "POST /node/69af06fcce46d8ec186ac913 HTTP/1.0" 202 Accepted 2026-03-09 17:44:54,992 - lava_callback - INFO - Starting async job submit for node 69af06fcce46d8ec186ac913 2026-03-09 17:44:54,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:55,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af06fcce46d8ec186ac913 HTTP/1.1" 200 None 2026-03-09 17:44:55,010 - lava_callback - DEBUG - Processing node 69af06fcce46d8ec186ac913: baseline-nfs-arm64 2026-03-09 17:44:55,016 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:44:55,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:44:55,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:44:55,218 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af06fcce46d8ec186ac913/lava-logs.yaml 2026-03-09 17:44:55,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:44:55,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:44:55,351 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af06fcce46d8ec186ac913/lava_callback.json.gz 2026-03-09 17:44:55,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:55,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af06fcce46d8ec186ac913 HTTP/1.1" 200 None 2026-03-09 17:44:55,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:55,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:44:55,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:44:55,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af06fcce46d8ec186ac913 HTTP/1.1" 200 None 2026-03-09 17:44:55,440 - lava_callback - INFO - Completed processing callback for node 69af06fcce46d8ec186ac913 INFO: 172.21.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40234 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:45:14,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:15,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33970 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:45:27,377 - lava_callback - INFO - Received callback for node 69ac642546f7ae8ff3365cb9 INFO: 172.21.0.1:33980 - "POST /node/69ac642546f7ae8ff3365cb9 HTTP/1.0" 202 Accepted 2026-03-09 17:45:27,383 - lava_callback - INFO - Starting async job submit for node 69ac642546f7ae8ff3365cb9 2026-03-09 17:45:27,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:27,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac642546f7ae8ff3365cb9 HTTP/1.1" 200 None 2026-03-09 17:45:27,399 - lava_callback - DEBUG - Processing node 69ac642546f7ae8ff3365cb9: baseline-arm-baylibre 2026-03-09 17:45:27,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac642546f7ae8ff3365cb9 2026-03-09 17:45:27,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:27,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac642546f7ae8ff3365cb9 HTTP/1.1" 200 None 2026-03-09 17:45:27,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:27,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:45:27,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:27,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac642546f7ae8ff3365cb9 HTTP/1.1" 200 None 2026-03-09 17:45:27,489 - lava_callback - ERROR - Error processing callback for node 69ac642546f7ae8ff3365cb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37840 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:45:47,354 - lava_callback - INFO - Received callback for node 69ac644846f7ae8ff3365cc7 INFO: 172.21.0.1:37854 - "POST /node/69ac644846f7ae8ff3365cc7 HTTP/1.0" 202 Accepted 2026-03-09 17:45:47,359 - lava_callback - INFO - Starting async job submit for node 69ac644846f7ae8ff3365cc7 2026-03-09 17:45:47,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:47,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac644846f7ae8ff3365cc7 HTTP/1.1" 200 None 2026-03-09 17:45:47,380 - lava_callback - DEBUG - Processing node 69ac644846f7ae8ff3365cc7: baseline-arm-baylibre 2026-03-09 17:45:47,380 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac644846f7ae8ff3365cc7 2026-03-09 17:45:47,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:47,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac644846f7ae8ff3365cc7 HTTP/1.1" 200 None 2026-03-09 17:45:47,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:47,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:45:47,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:45:47,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac644846f7ae8ff3365cc7 HTTP/1.1" 200 None 2026-03-09 17:45:47,465 - lava_callback - ERROR - Error processing callback for node 69ac644846f7ae8ff3365cc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50780 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:46:11,694 - lava_callback - INFO - Received callback for node 69af0757ce46d8ec186ac928 INFO: 172.21.0.1:33788 - "POST /node/69af0757ce46d8ec186ac928 HTTP/1.0" 202 Accepted 2026-03-09 17:46:11,700 - lava_callback - INFO - Starting async job submit for node 69af0757ce46d8ec186ac928 2026-03-09 17:46:11,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:11,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0757ce46d8ec186ac928 HTTP/1.1" 200 None 2026-03-09 17:46:11,753 - lava_callback - DEBUG - Processing node 69af0757ce46d8ec186ac928: baseline-arm64-foundriesio 2026-03-09 17:46:11,804 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:46:11,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:46:12,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:46:12,002 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af0757ce46d8ec186ac928/lava-logs.yaml 2026-03-09 17:46:12,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:46:12,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:46:12,129 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af0757ce46d8ec186ac928/lava_callback.json.gz 2026-03-09 17:46:12,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:12,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0757ce46d8ec186ac928 HTTP/1.1" 200 None 2026-03-09 17:46:12,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:12,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:46:12,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:12,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0757ce46d8ec186ac928 HTTP/1.1" 200 None 2026-03-09 17:46:12,251 - lava_callback - INFO - Completed processing callback for node 69af0757ce46d8ec186ac928 2026-03-09 17:46:15,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:15,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33800 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:46:23,831 - lava_callback - INFO - Received callback for node 69af01c6ce46d8ec186ac6db INFO: 172.21.0.1:45726 - "POST /node/69af01c6ce46d8ec186ac6db HTTP/1.0" 202 Accepted 2026-03-09 17:46:23,837 - lava_callback - INFO - Starting async job submit for node 69af01c6ce46d8ec186ac6db 2026-03-09 17:46:23,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:23,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c6ce46d8ec186ac6db HTTP/1.1" 200 None 2026-03-09 17:46:23,890 - lava_callback - DEBUG - Processing node 69af01c6ce46d8ec186ac6db: baseline-arm64-broonie 2026-03-09 17:46:24,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:46:24,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:46:24,319 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c6ce46d8ec186ac6db/log.txt.gz 2026-03-09 17:46:24,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:46:24,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:46:24,689 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c6ce46d8ec186ac6db/lava-logs.yaml 2026-03-09 17:46:24,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:46:24,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:46:24,844 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c6ce46d8ec186ac6db/lava_callback.json.gz DEBUG: Setup failed for 69af01c6ce46d8ec186ac6db. Transitting job node result: incomplete -> fail 2026-03-09 17:46:24,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:24,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c6ce46d8ec186ac6db HTTP/1.1" 200 None 2026-03-09 17:46:24,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:24,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:46:24,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:24,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c6ce46d8ec186ac6db HTTP/1.1" 200 None 2026-03-09 17:46:25,079 - lava_callback - INFO - Completed processing callback for node 69af01c6ce46d8ec186ac6db INFO: 172.21.0.1:45732 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:46:27,381 - lava_callback - INFO - Received callback for node 69ac646d46f7ae8ff3365cd5 INFO: 172.21.0.1:45736 - "POST /node/69ac646d46f7ae8ff3365cd5 HTTP/1.0" 202 Accepted 2026-03-09 17:46:27,386 - lava_callback - INFO - Starting async job submit for node 69ac646d46f7ae8ff3365cd5 2026-03-09 17:46:27,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:27,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac646d46f7ae8ff3365cd5 HTTP/1.1" 200 None 2026-03-09 17:46:27,415 - lava_callback - DEBUG - Processing node 69ac646d46f7ae8ff3365cd5: baseline-arm-baylibre 2026-03-09 17:46:27,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac646d46f7ae8ff3365cd5 2026-03-09 17:46:27,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:27,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac646d46f7ae8ff3365cd5 HTTP/1.1" 200 None 2026-03-09 17:46:27,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:27,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:46:27,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:27,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac646d46f7ae8ff3365cd5 HTTP/1.1" 200 None 2026-03-09 17:46:27,491 - lava_callback - ERROR - Error processing callback for node 69ac646d46f7ae8ff3365cd5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44738 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:46:45,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:46:45,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43914 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:47:07,426 - lava_callback - INFO - Received callback for node 69ac649046f7ae8ff3365ceb INFO: 172.21.0.1:43922 - "POST /node/69ac649046f7ae8ff3365ceb HTTP/1.0" 202 Accepted 2026-03-09 17:47:07,432 - lava_callback - INFO - Starting async job submit for node 69ac649046f7ae8ff3365ceb 2026-03-09 17:47:07,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:07,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac649046f7ae8ff3365ceb HTTP/1.1" 200 None 2026-03-09 17:47:07,448 - lava_callback - DEBUG - Processing node 69ac649046f7ae8ff3365ceb: baseline-arm-baylibre 2026-03-09 17:47:07,448 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac649046f7ae8ff3365ceb 2026-03-09 17:47:07,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:07,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac649046f7ae8ff3365ceb HTTP/1.1" 200 None 2026-03-09 17:47:07,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:07,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70f87e31b554803095edb HTTP/1.1" 200 None 2026-03-09 17:47:07,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:07,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac649046f7ae8ff3365ceb HTTP/1.1" 200 None 2026-03-09 17:47:07,509 - lava_callback - ERROR - Error processing callback for node 69ac649046f7ae8ff3365ceb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57642 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:47:24,145 - lava_callback - INFO - Received callback for node 69af01bece46d8ec186ac6c2 INFO: 172.21.0.1:33466 - "POST /node/69af01bece46d8ec186ac6c2 HTTP/1.0" 202 Accepted 2026-03-09 17:47:24,151 - lava_callback - INFO - Starting async job submit for node 69af01bece46d8ec186ac6c2 2026-03-09 17:47:24,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:24,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01bece46d8ec186ac6c2 HTTP/1.1" 200 None 2026-03-09 17:47:24,202 - lava_callback - DEBUG - Processing node 69af01bece46d8ec186ac6c2: baseline-arm64-broonie 2026-03-09 17:47:24,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:47:24,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:47:24,573 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bece46d8ec186ac6c2/log.txt.gz 2026-03-09 17:47:24,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:47:24,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:47:24,915 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bece46d8ec186ac6c2/lava-logs.yaml 2026-03-09 17:47:24,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:47:25,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:47:25,060 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01bece46d8ec186ac6c2/lava_callback.json.gz DEBUG: Setup failed for 69af01bece46d8ec186ac6c2. Transitting job node result: incomplete -> fail 2026-03-09 17:47:25,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:25,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01bece46d8ec186ac6c2 HTTP/1.1" 200 None 2026-03-09 17:47:25,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:25,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:47:25,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:25,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01bece46d8ec186ac6c2 HTTP/1.1" 200 None 2026-03-09 17:47:25,292 - lava_callback - INFO - Completed processing callback for node 69af01bece46d8ec186ac6c2 INFO: 172.21.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44062 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:47:37,673 - lava_callback - INFO - Received callback for node 69af079fce46d8ec186ac93f INFO: 172.21.0.1:44070 - "POST /node/69af079fce46d8ec186ac93f HTTP/1.0" 202 Accepted 2026-03-09 17:47:37,678 - lava_callback - INFO - Starting async job submit for node 69af079fce46d8ec186ac93f 2026-03-09 17:47:37,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:37,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af079fce46d8ec186ac93f HTTP/1.1" 200 None 2026-03-09 17:47:37,704 - lava_callback - DEBUG - Processing node 69af079fce46d8ec186ac93f: baseline-arm64-clabbe 2026-03-09 17:47:37,708 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:47:37,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:47:37,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:47:37,848 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af079fce46d8ec186ac93f/lava-logs.yaml 2026-03-09 17:47:37,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:47:37,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:47:37,992 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af079fce46d8ec186ac93f/lava_callback.json.gz 2026-03-09 17:47:37,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:38,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af079fce46d8ec186ac93f HTTP/1.1" 200 None 2026-03-09 17:47:38,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:38,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:47:38,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:38,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af079fce46d8ec186ac93f HTTP/1.1" 200 None 2026-03-09 17:47:38,056 - lava_callback - INFO - Completed processing callback for node 69af079fce46d8ec186ac93f 2026-03-09 17:47:45,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:47:45,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39526 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:49:03,502 - lava_callback - INFO - Received callback for node 69af07edce46d8ec186ac94d INFO: 172.21.0.1:59434 - "POST /node/69af07edce46d8ec186ac94d HTTP/1.0" 202 Accepted 2026-03-09 17:49:03,508 - lava_callback - INFO - Starting async job submit for node 69af07edce46d8ec186ac94d 2026-03-09 17:49:03,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:49:03,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af07edce46d8ec186ac94d HTTP/1.1" 200 None 2026-03-09 17:49:03,526 - lava_callback - DEBUG - Processing node 69af07edce46d8ec186ac94d: baseline-arm64-broonie 2026-03-09 17:49:03,532 - lava_callback - WARNING - Failed to upload log 2026-03-09 17:49:03,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:49:03,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:49:03,834 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af07edce46d8ec186ac94d/lava-logs.yaml 2026-03-09 17:49:03,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:49:03,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:49:03,983 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af07edce46d8ec186ac94d/lava_callback.json.gz 2026-03-09 17:49:03,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:49:04,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af07edce46d8ec186ac94d HTTP/1.1" 200 None 2026-03-09 17:49:04,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:49:04,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:49:04,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:49:04,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af07edce46d8ec186ac94d HTTP/1.1" 200 None 2026-03-09 17:49:04,053 - lava_callback - INFO - Completed processing callback for node 69af07edce46d8ec186ac94d INFO: 172.21.0.1:59442 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:49:15,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:49:15,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40486 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:51:01,408 - lava_callback - INFO - Received callback for node 69aefa1fce46d8ec186abf59 INFO: 172.21.0.1:33998 - "POST /node/69aefa1fce46d8ec186abf59 HTTP/1.0" 202 Accepted 2026-03-09 17:51:01,457 - lava_callback - INFO - Starting async job submit for node 69aefa1fce46d8ec186abf59 2026-03-09 17:51:01,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:01,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1fce46d8ec186abf59 HTTP/1.1" 200 None 2026-03-09 17:51:01,496 - lava_callback - DEBUG - Processing node 69aefa1fce46d8ec186abf59: baseline-arm-pengutronix 2026-03-09 17:51:01,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:01,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:01,847 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1fce46d8ec186abf59/log.txt.gz 2026-03-09 17:51:01,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:02,101 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:02,103 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1fce46d8ec186abf59/lava-logs.yaml 2026-03-09 17:51:02,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:02,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:02,251 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69aefa1fce46d8ec186abf59/lava_callback.json.gz DEBUG: Setup failed for 69aefa1fce46d8ec186abf59. Transitting job node result: incomplete -> fail 2026-03-09 17:51:02,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:02,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefa1fce46d8ec186abf59 HTTP/1.1" 200 None 2026-03-09 17:51:02,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:02,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 17:51:02,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:02,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefa1fce46d8ec186abf59 HTTP/1.1" 200 None 2026-03-09 17:51:02,429 - lava_callback - INFO - Completed processing callback for node 69aefa1fce46d8ec186abf59 INFO: 172.21.0.1:34006 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:51:14,798 - lava_callback - INFO - Received callback for node 69af0827ce46d8ec186ac95c INFO: 172.21.0.1:49858 - "POST /node/69af0827ce46d8ec186ac95c HTTP/1.0" 202 Accepted 2026-03-09 17:51:14,804 - lava_callback - INFO - Starting async job submit for node 69af0827ce46d8ec186ac95c 2026-03-09 17:51:14,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:14,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0827ce46d8ec186ac95c HTTP/1.1" 200 None 2026-03-09 17:51:14,871 - lava_callback - DEBUG - Processing node 69af0827ce46d8ec186ac95c: baseline-arm64 2026-03-09 17:51:15,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:49866 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:51:15,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 17:51:15,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:15,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:15,463 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0827ce46d8ec186ac95c/log.txt.gz 2026-03-09 17:51:15,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:15,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:15,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0827ce46d8ec186ac95c/lava-logs.yaml 2026-03-09 17:51:15,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:51:16,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:51:16,134 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0827ce46d8ec186ac95c/lava_callback.json.gz DEBUG: Setup failed for 69af0827ce46d8ec186ac95c. Transitting job node result: incomplete -> fail 2026-03-09 17:51:16,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:16,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0827ce46d8ec186ac95c HTTP/1.1" 200 None 2026-03-09 17:51:16,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:16,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:51:16,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:16,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0827ce46d8ec186ac95c HTTP/1.1" 200 None 2026-03-09 17:51:16,335 - lava_callback - INFO - Completed processing callback for node 69af0827ce46d8ec186ac95c INFO: 172.21.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54864 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:51:45,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:51:45,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54056 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:52:30,929 - lava_callback - INFO - Received callback for node 69af0874ce46d8ec186ac967 INFO: 172.21.0.1:51992 - "POST /node/69af0874ce46d8ec186ac967 HTTP/1.0" 202 Accepted 2026-03-09 17:52:30,936 - lava_callback - INFO - Starting async job submit for node 69af0874ce46d8ec186ac967 2026-03-09 17:52:30,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:52:30,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0874ce46d8ec186ac967 HTTP/1.1" 200 None 2026-03-09 17:52:30,992 - lava_callback - DEBUG - Processing node 69af0874ce46d8ec186ac967: baseline-arm64 2026-03-09 17:52:31,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:52:31,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:52:31,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0874ce46d8ec186ac967/log.txt.gz 2026-03-09 17:52:32,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:52:32,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:52:32,977 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0874ce46d8ec186ac967/lava-logs.yaml 2026-03-09 17:52:32,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:52:33,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:52:33,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0874ce46d8ec186ac967/lava_callback.json.gz DEBUG: Setup failed for 69af0874ce46d8ec186ac967. Transitting job node result: incomplete -> fail 2026-03-09 17:52:33,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:52:33,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0874ce46d8ec186ac967 HTTP/1.1" 200 None 2026-03-09 17:52:33,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:52:33,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:52:33,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:52:33,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0874ce46d8ec186ac967 HTTP/1.1" 200 None 2026-03-09 17:52:33,402 - lava_callback - INFO - Completed processing callback for node 69af0874ce46d8ec186ac967 INFO: 172.21.0.1:52004 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:52:45,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:52:45,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:53:47,849 - lava_callback - INFO - Received callback for node 69ac661f46f7ae8ff3365d72 INFO: 172.21.0.1:46876 - "POST /node/69ac661f46f7ae8ff3365d72 HTTP/1.0" 202 Accepted 2026-03-09 17:53:47,854 - lava_callback - INFO - Starting async job submit for node 69ac661f46f7ae8ff3365d72 2026-03-09 17:53:47,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:53:47,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac661f46f7ae8ff3365d72 HTTP/1.1" 200 None 2026-03-09 17:53:47,870 - lava_callback - DEBUG - Processing node 69ac661f46f7ae8ff3365d72: baseline-arm64-baylibre 2026-03-09 17:53:47,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac661f46f7ae8ff3365d72 2026-03-09 17:53:47,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:53:47,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac661f46f7ae8ff3365d72 HTTP/1.1" 200 None 2026-03-09 17:53:47,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:53:47,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 17:53:47,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:53:47,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac661f46f7ae8ff3365d72 HTTP/1.1" 200 None 2026-03-09 17:53:47,971 - lava_callback - ERROR - Error processing callback for node 69ac661f46f7ae8ff3365d72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40476 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:54:14,523 - lava_callback - INFO - Received callback for node 69af08c7ce46d8ec186ac97d INFO: 172.21.0.1:41634 - "POST /node/69af08c7ce46d8ec186ac97d HTTP/1.0" 202 Accepted 2026-03-09 17:54:14,529 - lava_callback - INFO - Starting async job submit for node 69af08c7ce46d8ec186ac97d 2026-03-09 17:54:14,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:54:14,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af08c7ce46d8ec186ac97d HTTP/1.1" 200 None 2026-03-09 17:54:14,579 - lava_callback - DEBUG - Processing node 69af08c7ce46d8ec186ac97d: baseline-arm64 2026-03-09 17:54:14,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:54:15,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:54:15,102 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af08c7ce46d8ec186ac97d/log.txt.gz INFO: 172.21.0.1:41640 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:54:15,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:54:15,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:54:15,580 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af08c7ce46d8ec186ac97d/lava-logs.yaml 2026-03-09 17:54:15,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:54:15,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:54:15,747 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af08c7ce46d8ec186ac97d/lava_callback.json.gz DEBUG: Setup failed for 69af08c7ce46d8ec186ac97d. Transitting job node result: incomplete -> fail 2026-03-09 17:54:15,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:54:15,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af08c7ce46d8ec186ac97d HTTP/1.1" 200 None 2026-03-09 17:54:15,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:54:15,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:54:15,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:54:15,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af08c7ce46d8ec186ac97d HTTP/1.1" 200 None 2026-03-09 17:54:15,943 - lava_callback - INFO - Completed processing callback for node 69af08c7ce46d8ec186ac97d INFO: 172.21.0.1:41654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56534 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:54:45,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:54:45,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56950 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:55:13,015 - lava_callback - INFO - Received callback for node 69af0913ce46d8ec186ac988 INFO: 172.21.0.1:58140 - "POST /node/69af0913ce46d8ec186ac988 HTTP/1.0" 202 Accepted 2026-03-09 17:55:13,022 - lava_callback - INFO - Starting async job submit for node 69af0913ce46d8ec186ac988 2026-03-09 17:55:13,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:13,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0913ce46d8ec186ac988 HTTP/1.1" 200 None 2026-03-09 17:55:13,079 - lava_callback - DEBUG - Processing node 69af0913ce46d8ec186ac988: baseline-arm64 2026-03-09 17:55:13,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:55:13,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:55:13,536 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0913ce46d8ec186ac988/log.txt.gz 2026-03-09 17:55:13,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:55:13,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:55:13,909 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0913ce46d8ec186ac988/lava-logs.yaml 2026-03-09 17:55:13,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:55:14,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:55:14,072 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0913ce46d8ec186ac988/lava_callback.json.gz DEBUG: Setup failed for 69af0913ce46d8ec186ac988. Transitting job node result: incomplete -> fail 2026-03-09 17:55:14,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:14,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0913ce46d8ec186ac988 HTTP/1.1" 200 None 2026-03-09 17:55:14,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:14,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:55:14,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:14,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0913ce46d8ec186ac988 HTTP/1.1" 200 None 2026-03-09 17:55:14,270 - lava_callback - INFO - Completed processing callback for node 69af0913ce46d8ec186ac988 INFO: 172.21.0.1:58154 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:55:15,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:15,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45934 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:55:27,883 - lava_callback - INFO - Received callback for node 69ac668d46f7ae8ff3365dae INFO: 172.21.0.1:45946 - "POST /node/69ac668d46f7ae8ff3365dae HTTP/1.0" 202 Accepted 2026-03-09 17:55:27,888 - lava_callback - INFO - Starting async job submit for node 69ac668d46f7ae8ff3365dae 2026-03-09 17:55:27,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:27,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac668d46f7ae8ff3365dae HTTP/1.1" 200 None 2026-03-09 17:55:27,904 - lava_callback - DEBUG - Processing node 69ac668d46f7ae8ff3365dae: baseline-arm64-baylibre 2026-03-09 17:55:27,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac668d46f7ae8ff3365dae 2026-03-09 17:55:27,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:27,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac668d46f7ae8ff3365dae HTTP/1.1" 200 None 2026-03-09 17:55:27,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:27,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 17:55:27,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:55:27,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac668d46f7ae8ff3365dae HTTP/1.1" 200 None 2026-03-09 17:55:27,963 - lava_callback - ERROR - Error processing callback for node 69ac668d46f7ae8ff3365dae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51280 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:56:11,362 - lava_callback - INFO - Received callback for node 69af095ece46d8ec186ac997 INFO: 172.21.0.1:58028 - "POST /node/69af095ece46d8ec186ac997 HTTP/1.0" 202 Accepted 2026-03-09 17:56:11,376 - lava_callback - INFO - Starting async job submit for node 69af095ece46d8ec186ac997 2026-03-09 17:56:11,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:56:11,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af095ece46d8ec186ac997 HTTP/1.1" 200 None 2026-03-09 17:56:11,464 - lava_callback - DEBUG - Processing node 69af095ece46d8ec186ac997: baseline-arm64 2026-03-09 17:56:12,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:56:12,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:56:12,312 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af095ece46d8ec186ac997/log.txt.gz 2026-03-09 17:56:12,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:56:12,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:56:12,945 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af095ece46d8ec186ac997/lava-logs.yaml 2026-03-09 17:56:12,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:56:13,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:56:13,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af095ece46d8ec186ac997/lava_callback.json.gz DEBUG: Setup failed for 69af095ece46d8ec186ac997. Transitting job node result: incomplete -> fail 2026-03-09 17:56:13,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:56:13,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af095ece46d8ec186ac997 HTTP/1.1" 200 None 2026-03-09 17:56:13,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:56:13,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:56:13,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:56:13,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af095ece46d8ec186ac997 HTTP/1.1" 200 None 2026-03-09 17:56:13,384 - lava_callback - INFO - Completed processing callback for node 69af095ece46d8ec186ac997 INFO: 172.21.0.1:58036 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:56:15,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:56:15,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37740 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:57:42,325 - lava_callback - INFO - Received callback for node 69af09b0ce46d8ec186ac9a6 INFO: 172.21.0.1:53520 - "POST /node/69af09b0ce46d8ec186ac9a6 HTTP/1.0" 202 Accepted 2026-03-09 17:57:42,335 - lava_callback - INFO - Starting async job submit for node 69af09b0ce46d8ec186ac9a6 2026-03-09 17:57:42,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:57:42,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af09b0ce46d8ec186ac9a6 HTTP/1.1" 200 None 2026-03-09 17:57:42,398 - lava_callback - DEBUG - Processing node 69af09b0ce46d8ec186ac9a6: baseline-nfs-arm64 2026-03-09 17:57:42,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:57:43,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:57:43,226 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af09b0ce46d8ec186ac9a6/log.txt.gz 2026-03-09 17:57:43,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:57:43,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:57:43,731 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af09b0ce46d8ec186ac9a6/lava-logs.yaml 2026-03-09 17:57:43,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:57:43,916 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:57:43,917 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af09b0ce46d8ec186ac9a6/lava_callback.json.gz DEBUG: Setup failed for 69af09b0ce46d8ec186ac9a6. Transitting job node result: incomplete -> fail 2026-03-09 17:57:43,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:57:44,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af09b0ce46d8ec186ac9a6 HTTP/1.1" 200 None 2026-03-09 17:57:44,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:57:44,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:57:44,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:57:44,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af09b0ce46d8ec186ac9a6 HTTP/1.1" 200 None 2026-03-09 17:57:44,234 - lava_callback - INFO - Completed processing callback for node 69af09b0ce46d8ec186ac9a6 2026-03-09 17:57:45,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:57:45,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37682 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:58:59,919 - lava_callback - INFO - Received callback for node 69af0a0fce46d8ec186ac9b8 INFO: 172.21.0.1:59310 - "POST /node/69af0a0fce46d8ec186ac9b8 HTTP/1.0" 202 Accepted 2026-03-09 17:58:59,925 - lava_callback - INFO - Starting async job submit for node 69af0a0fce46d8ec186ac9b8 2026-03-09 17:58:59,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:58:59,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a0fce46d8ec186ac9b8 HTTP/1.1" 200 None 2026-03-09 17:58:59,982 - lava_callback - DEBUG - Processing node 69af0a0fce46d8ec186ac9b8: baseline-arm64-foundriesio 2026-03-09 17:59:00,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:59:00,362 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:59:00,364 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af0a0fce46d8ec186ac9b8/log.txt.gz 2026-03-09 17:59:00,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:59:00,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:59:00,753 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af0a0fce46d8ec186ac9b8/lava-logs.yaml 2026-03-09 17:59:00,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 17:59:00,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 17:59:00,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af0a0fce46d8ec186ac9b8/lava_callback.json.gz DEBUG: Setup failed for 69af0a0fce46d8ec186ac9b8. Transitting job node result: incomplete -> fail 2026-03-09 17:59:00,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:00,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a0fce46d8ec186ac9b8 HTTP/1.1" 200 None 2026-03-09 17:59:00,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:01,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 17:59:01,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:01,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0a0fce46d8ec186ac9b8 HTTP/1.1" 200 None 2026-03-09 17:59:01,114 - lava_callback - INFO - Completed processing callback for node 69af0a0fce46d8ec186ac9b8 INFO: 172.21.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 17:59:15,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:15,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34270 - "GET / HTTP/1.0" 200 OK 2026-03-09 17:59:28,131 - lava_callback - INFO - Received callback for node 69ac677c46f7ae8ff3365df0 INFO: 172.21.0.1:34274 - "POST /node/69ac677c46f7ae8ff3365df0 HTTP/1.0" 202 Accepted 2026-03-09 17:59:28,137 - lava_callback - INFO - Starting async job submit for node 69ac677c46f7ae8ff3365df0 2026-03-09 17:59:28,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:28,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac677c46f7ae8ff3365df0 HTTP/1.1" 200 None 2026-03-09 17:59:28,153 - lava_callback - DEBUG - Processing node 69ac677c46f7ae8ff3365df0: baseline-arm64-baylibre 2026-03-09 17:59:28,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac677c46f7ae8ff3365df0 2026-03-09 17:59:28,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:28,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac677c46f7ae8ff3365df0 HTTP/1.1" 200 None 2026-03-09 17:59:28,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:28,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 17:59:28,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 17:59:28,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac677c46f7ae8ff3365df0 HTTP/1.1" 200 None 2026-03-09 17:59:28,209 - lava_callback - ERROR - Error processing callback for node 69ac677c46f7ae8ff3365df0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34604 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:00:15,608 - lava_callback - INFO - Received callback for node 69af0a4dce46d8ec186ac9c7 INFO: 172.21.0.1:34618 - "POST /node/69af0a4dce46d8ec186ac9c7 HTTP/1.0" 202 Accepted 2026-03-09 18:00:15,614 - lava_callback - INFO - Starting async job submit for node 69af0a4dce46d8ec186ac9c7 2026-03-09 18:00:15,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:00:15,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a4dce46d8ec186ac9c7 HTTP/1.1" 200 None 2026-03-09 18:00:15,661 - lava_callback - DEBUG - Processing node 69af0a4dce46d8ec186ac9c7: baseline-nfs-arm64 2026-03-09 18:00:15,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:00:16,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:00:16,141 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a4dce46d8ec186ac9c7/log.txt.gz 2026-03-09 18:00:16,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:34622 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:00:16,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:00:16,550 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a4dce46d8ec186ac9c7/lava-logs.yaml 2026-03-09 18:00:16,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:00:16,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:00:16,717 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a4dce46d8ec186ac9c7/lava_callback.json.gz DEBUG: Setup failed for 69af0a4dce46d8ec186ac9c7. Transitting job node result: incomplete -> fail 2026-03-09 18:00:16,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:00:16,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a4dce46d8ec186ac9c7 HTTP/1.1" 200 None 2026-03-09 18:00:16,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:00:16,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:00:16,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:00:16,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0a4dce46d8ec186ac9c7 HTTP/1.1" 200 None 2026-03-09 18:00:16,913 - lava_callback - INFO - Completed processing callback for node 69af0a4dce46d8ec186ac9c7 INFO: 172.21.0.1:44194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60556 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:00:45,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:00:45,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56594 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:01:36,915 - lava_callback - INFO - Received callback for node 69af0a97ce46d8ec186ac9d4 INFO: 172.21.0.1:56598 - "POST /node/69af0a97ce46d8ec186ac9d4 HTTP/1.0" 202 Accepted 2026-03-09 18:01:36,922 - lava_callback - INFO - Starting async job submit for node 69af0a97ce46d8ec186ac9d4 2026-03-09 18:01:36,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:01:36,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a97ce46d8ec186ac9d4 HTTP/1.1" 200 None 2026-03-09 18:01:36,971 - lava_callback - DEBUG - Processing node 69af0a97ce46d8ec186ac9d4: baseline-nfs-arm64 2026-03-09 18:01:37,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:01:37,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:01:37,972 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a97ce46d8ec186ac9d4/log.txt.gz 2026-03-09 18:01:38,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:01:38,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:01:38,719 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a97ce46d8ec186ac9d4/lava-logs.yaml 2026-03-09 18:01:38,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:01:38,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:01:38,951 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0a97ce46d8ec186ac9d4/lava_callback.json.gz DEBUG: Setup failed for 69af0a97ce46d8ec186ac9d4. Transitting job node result: incomplete -> fail 2026-03-09 18:01:39,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:01:39,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0a97ce46d8ec186ac9d4 HTTP/1.1" 200 None 2026-03-09 18:01:39,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:01:39,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:01:39,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:01:39,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0a97ce46d8ec186ac9d4 HTTP/1.1" 200 None 2026-03-09 18:01:39,141 - lava_callback - INFO - Completed processing callback for node 69af0a97ce46d8ec186ac9d4 2026-03-09 18:01:45,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:01:45,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45600 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:02:28,268 - lava_callback - INFO - Received callback for node 69ac683146f7ae8ff3365e21 INFO: 172.21.0.1:45602 - "POST /node/69ac683146f7ae8ff3365e21 HTTP/1.0" 202 Accepted 2026-03-09 18:02:28,274 - lava_callback - INFO - Starting async job submit for node 69ac683146f7ae8ff3365e21 2026-03-09 18:02:28,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:02:28,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac683146f7ae8ff3365e21 HTTP/1.1" 200 None 2026-03-09 18:02:28,295 - lava_callback - DEBUG - Processing node 69ac683146f7ae8ff3365e21: baseline-arm64-baylibre 2026-03-09 18:02:28,295 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac683146f7ae8ff3365e21 2026-03-09 18:02:28,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:02:28,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac683146f7ae8ff3365e21 HTTP/1.1" 200 None 2026-03-09 18:02:28,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:02:28,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 18:02:28,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:02:28,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac683146f7ae8ff3365e21 HTTP/1.1" 200 None 2026-03-09 18:02:28,376 - lava_callback - ERROR - Error processing callback for node 69ac683146f7ae8ff3365e21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47332 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:03:12,861 - lava_callback - INFO - Received callback for node 69af0ae4ce46d8ec186ac9e4 INFO: 172.21.0.1:45684 - "POST /node/69af0ae4ce46d8ec186ac9e4 HTTP/1.0" 202 Accepted 2026-03-09 18:03:12,868 - lava_callback - INFO - Starting async job submit for node 69af0ae4ce46d8ec186ac9e4 2026-03-09 18:03:12,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:03:12,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0ae4ce46d8ec186ac9e4 HTTP/1.1" 200 None 2026-03-09 18:03:12,915 - lava_callback - DEBUG - Processing node 69af0ae4ce46d8ec186ac9e4: baseline-nfs-arm64 2026-03-09 18:03:13,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:03:13,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:03:13,663 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0ae4ce46d8ec186ac9e4/log.txt.gz 2026-03-09 18:03:13,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:03:14,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:03:14,217 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0ae4ce46d8ec186ac9e4/lava-logs.yaml 2026-03-09 18:03:14,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:03:14,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:03:14,404 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0ae4ce46d8ec186ac9e4/lava_callback.json.gz DEBUG: Setup failed for 69af0ae4ce46d8ec186ac9e4. Transitting job node result: incomplete -> fail 2026-03-09 18:03:14,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:03:14,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0ae4ce46d8ec186ac9e4 HTTP/1.1" 200 None 2026-03-09 18:03:14,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:03:14,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:03:14,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:03:14,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0ae4ce46d8ec186ac9e4 HTTP/1.1" 200 None 2026-03-09 18:03:14,629 - lava_callback - INFO - Completed processing callback for node 69af0ae4ce46d8ec186ac9e4 INFO: 172.21.0.1:45696 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:03:15,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:03:15,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57054 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:04:24,781 - lava_callback - INFO - Received callback for node 69af0b32ce46d8ec186ac9ef INFO: 172.21.0.1:54618 - "POST /node/69af0b32ce46d8ec186ac9ef HTTP/1.0" 202 Accepted 2026-03-09 18:04:24,787 - lava_callback - INFO - Starting async job submit for node 69af0b32ce46d8ec186ac9ef 2026-03-09 18:04:24,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:04:24,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0b32ce46d8ec186ac9ef HTTP/1.1" 200 None 2026-03-09 18:04:24,839 - lava_callback - DEBUG - Processing node 69af0b32ce46d8ec186ac9ef: baseline-nfs-arm64 2026-03-09 18:04:25,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:04:25,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:04:25,361 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0b32ce46d8ec186ac9ef/log.txt.gz 2026-03-09 18:04:25,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:04:25,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:04:25,821 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0b32ce46d8ec186ac9ef/lava-logs.yaml 2026-03-09 18:04:25,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:04:25,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:04:25,998 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0b32ce46d8ec186ac9ef/lava_callback.json.gz DEBUG: Setup failed for 69af0b32ce46d8ec186ac9ef. Transitting job node result: incomplete -> fail 2026-03-09 18:04:26,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:04:26,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0b32ce46d8ec186ac9ef HTTP/1.1" 200 None 2026-03-09 18:04:26,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:04:26,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:04:26,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:04:26,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0b32ce46d8ec186ac9ef HTTP/1.1" 200 None 2026-03-09 18:04:26,262 - lava_callback - INFO - Completed processing callback for node 69af0b32ce46d8ec186ac9ef INFO: 172.21.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42814 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:04:45,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:04:45,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58048 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:06:48,725 - lava_callback - INFO - Received callback for node 69ac693646f7ae8ff3365e77 INFO: 172.21.0.1:58058 - "POST /node/69ac693646f7ae8ff3365e77 HTTP/1.0" 202 Accepted 2026-03-09 18:06:48,731 - lava_callback - INFO - Starting async job submit for node 69ac693646f7ae8ff3365e77 2026-03-09 18:06:48,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:06:48,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac693646f7ae8ff3365e77 HTTP/1.1" 200 None 2026-03-09 18:06:48,747 - lava_callback - DEBUG - Processing node 69ac693646f7ae8ff3365e77: baseline-arm64-baylibre 2026-03-09 18:06:48,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac693646f7ae8ff3365e77 2026-03-09 18:06:48,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:06:48,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac693646f7ae8ff3365e77 HTTP/1.1" 200 None 2026-03-09 18:06:48,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:06:48,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 18:06:48,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:06:48,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac693646f7ae8ff3365e77 HTTP/1.1" 200 None 2026-03-09 18:06:48,798 - lava_callback - ERROR - Error processing callback for node 69ac693646f7ae8ff3365e77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58592 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:09:16,633 - lava_callback - INFO - Received callback for node 69af0c63ce46d8ec186aca1e INFO: 172.21.0.1:58596 - "POST /node/69af0c63ce46d8ec186aca1e HTTP/1.0" 202 Accepted 2026-03-09 18:09:16,643 - lava_callback - INFO - Starting async job submit for node 69af0c63ce46d8ec186aca1e 2026-03-09 18:09:16,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:16,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0c63ce46d8ec186aca1e HTTP/1.1" 200 None 2026-03-09 18:09:16,713 - lava_callback - DEBUG - Processing node 69af0c63ce46d8ec186aca1e: baseline-nfs-arm64 2026-03-09 18:09:16,911 - lava_callback - INFO - Received callback for node 69af0c0ece46d8ec186aca16 INFO: 172.21.0.1:58600 - "POST /node/69af0c0ece46d8ec186aca16 HTTP/1.0" 202 Accepted 2026-03-09 18:09:16,933 - lava_callback - INFO - Starting async job submit for node 69af0c0ece46d8ec186aca16 2026-03-09 18:09:17,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:17,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0c0ece46d8ec186aca16 HTTP/1.1" 200 None 2026-03-09 18:09:17,165 - lava_callback - DEBUG - Processing node 69af0c0ece46d8ec186aca16: baseline-arm64 2026-03-09 18:09:17,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:17,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:17,649 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0c63ce46d8ec186aca1e/log.txt.gz 2026-03-09 18:09:17,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:17,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:17,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:17,983 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0c0ece46d8ec186aca16/log.txt.gz 2026-03-09 18:09:18,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:18,153 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0c63ce46d8ec186aca1e/lava-logs.yaml 2026-03-09 18:09:18,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:18,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:18,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:18,393 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0c63ce46d8ec186aca1e/lava_callback.json.gz DEBUG: Setup failed for 69af0c63ce46d8ec186aca1e. Transitting job node result: incomplete -> fail 2026-03-09 18:09:18,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:18,465 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0c0ece46d8ec186aca16/lava-logs.yaml 2026-03-09 18:09:18,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:09:18,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0c63ce46d8ec186aca1e HTTP/1.1" 200 None 2026-03-09 18:09:18,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:09:18,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0c63ce46d8ec186aca1e HTTP/1.1" 200 None 2026-03-09 18:09:18,632 - lava_callback - INFO - Completed processing callback for node 69af0c63ce46d8ec186aca1e 2026-03-09 18:09:18,663 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:09:18,665 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0c0ece46d8ec186aca16/lava_callback.json.gz DEBUG: Setup failed for 69af0c0ece46d8ec186aca16. Transitting job node result: incomplete -> fail 2026-03-09 18:09:18,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0c0ece46d8ec186aca16 HTTP/1.1" 200 None 2026-03-09 18:09:18,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:09:18,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:18,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0c0ece46d8ec186aca16 HTTP/1.1" 200 None 2026-03-09 18:09:18,891 - lava_callback - INFO - Completed processing callback for node 69af0c0ece46d8ec186aca16 INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53558 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:09:45,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:09:45,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56848 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:11:28,746 - lava_callback - INFO - Received callback for node 69ac6a4f46f7ae8ff3365ed1 INFO: 172.21.0.1:56850 - "POST /node/69ac6a4f46f7ae8ff3365ed1 HTTP/1.0" 202 Accepted 2026-03-09 18:11:28,752 - lava_callback - INFO - Starting async job submit for node 69ac6a4f46f7ae8ff3365ed1 2026-03-09 18:11:28,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:11:28,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac6a4f46f7ae8ff3365ed1 HTTP/1.1" 200 None 2026-03-09 18:11:28,768 - lava_callback - DEBUG - Processing node 69ac6a4f46f7ae8ff3365ed1: baseline-arm64-baylibre 2026-03-09 18:11:28,768 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac6a4f46f7ae8ff3365ed1 2026-03-09 18:11:28,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:11:28,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac6a4f46f7ae8ff3365ed1 HTTP/1.1" 200 None 2026-03-09 18:11:28,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:11:28,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a46670a327b9815668f9d6 HTTP/1.1" 200 None 2026-03-09 18:11:28,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:11:28,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac6a4f46f7ae8ff3365ed1 HTTP/1.1" 200 None 2026-03-09 18:11:28,820 - lava_callback - ERROR - Error processing callback for node 69ac6a4f46f7ae8ff3365ed1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43808 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:12:11,323 - lava_callback - INFO - Received callback for node 69af0cabce46d8ec186aca28 INFO: 172.21.0.1:45342 - "POST /node/69af0cabce46d8ec186aca28 HTTP/1.0" 202 Accepted 2026-03-09 18:12:11,330 - lava_callback - INFO - Starting async job submit for node 69af0cabce46d8ec186aca28 2026-03-09 18:12:11,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:11,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0cabce46d8ec186aca28 HTTP/1.1" 200 None 2026-03-09 18:12:11,387 - lava_callback - DEBUG - Processing node 69af0cabce46d8ec186aca28: baseline-nfs-arm64 2026-03-09 18:12:11,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:11,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:11,952 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0cabce46d8ec186aca28/log.txt.gz 2026-03-09 18:12:12,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:12,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:12,440 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0cabce46d8ec186aca28/lava-logs.yaml 2026-03-09 18:12:12,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:12,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:12,586 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0cabce46d8ec186aca28/lava_callback.json.gz DEBUG: Setup failed for 69af0cabce46d8ec186aca28. Transitting job node result: incomplete -> fail 2026-03-09 18:12:12,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:12,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0cabce46d8ec186aca28 HTTP/1.1" 200 None 2026-03-09 18:12:12,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:12,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:12:12,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:12,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0cabce46d8ec186aca28 HTTP/1.1" 200 None 2026-03-09 18:12:12,796 - lava_callback - INFO - Completed processing callback for node 69af0cabce46d8ec186aca28 INFO: 172.21.0.1:45352 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:12:15,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:15,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51698 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:12:36,545 - lava_callback - INFO - Received callback for node 69af0d38ce46d8ec186aca41 INFO: 172.21.0.1:51704 - "POST /node/69af0d38ce46d8ec186aca41 HTTP/1.0" 202 Accepted 2026-03-09 18:12:36,551 - lava_callback - INFO - Starting async job submit for node 69af0d38ce46d8ec186aca41 2026-03-09 18:12:36,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:36,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0d38ce46d8ec186aca41 HTTP/1.1" 200 None 2026-03-09 18:12:36,598 - lava_callback - DEBUG - Processing node 69af0d38ce46d8ec186aca41: baseline-arm64 2026-03-09 18:12:36,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:37,103 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:37,105 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0d38ce46d8ec186aca41/log.txt.gz 2026-03-09 18:12:37,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:37,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:37,551 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0d38ce46d8ec186aca41/lava-logs.yaml 2026-03-09 18:12:37,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:12:37,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:12:37,726 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0d38ce46d8ec186aca41/lava_callback.json.gz DEBUG: Setup failed for 69af0d38ce46d8ec186aca41. Transitting job node result: incomplete -> fail 2026-03-09 18:12:37,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:37,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0d38ce46d8ec186aca41 HTTP/1.1" 200 None 2026-03-09 18:12:37,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:37,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:12:37,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:37,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0d38ce46d8ec186aca41 HTTP/1.1" 200 None 2026-03-09 18:12:37,932 - lava_callback - INFO - Completed processing callback for node 69af0d38ce46d8ec186aca41 2026-03-09 18:12:45,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:12:45,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38018 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:13:32,231 - lava_callback - INFO - Received callback for node 69aefc20ce46d8ec186ac280 INFO: 172.21.0.1:43410 - "POST /node/69aefc20ce46d8ec186ac280 HTTP/1.0" 202 Accepted 2026-03-09 18:13:32,238 - lava_callback - INFO - Starting async job submit for node 69aefc20ce46d8ec186ac280 2026-03-09 18:13:32,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:13:32,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc20ce46d8ec186ac280 HTTP/1.1" 200 None 2026-03-09 18:13:32,293 - lava_callback - DEBUG - Processing node 69aefc20ce46d8ec186ac280: baseline-arm64-broonie 2026-03-09 18:13:32,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:13:32,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:13:32,963 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aefc20ce46d8ec186ac280/log.txt.gz 2026-03-09 18:13:33,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:13:33,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:13:33,482 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aefc20ce46d8ec186ac280/lava-logs.yaml 2026-03-09 18:13:33,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:13:33,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:13:33,663 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69aefc20ce46d8ec186ac280/lava_callback.json.gz DEBUG: Setup failed for 69aefc20ce46d8ec186ac280. Transitting job node result: incomplete -> fail 2026-03-09 18:13:33,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:13:33,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aefc20ce46d8ec186ac280 HTTP/1.1" 200 None 2026-03-09 18:13:33,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:13:33,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-09 18:13:33,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:13:33,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aefc20ce46d8ec186ac280 HTTP/1.1" 200 None 2026-03-09 18:13:33,868 - lava_callback - INFO - Completed processing callback for node 69aefc20ce46d8ec186ac280 INFO: 172.21.0.1:43426 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:13:45,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:13:45,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51526 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:14:55,145 - lava_callback - INFO - Received callback for node 69af0e18ce46d8ec186aca70 INFO: 172.21.0.1:50726 - "POST /node/69af0e18ce46d8ec186aca70 HTTP/1.0" 202 Accepted 2026-03-09 18:14:55,151 - lava_callback - INFO - Starting async job submit for node 69af0e18ce46d8ec186aca70 2026-03-09 18:14:55,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:14:55,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0e18ce46d8ec186aca70 HTTP/1.1" 200 None 2026-03-09 18:14:55,169 - lava_callback - DEBUG - Processing node 69af0e18ce46d8ec186aca70: baseline-arm64 2026-03-09 18:14:55,173 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:14:55,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:14:55,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:14:55,356 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0e18ce46d8ec186aca70/lava-logs.yaml 2026-03-09 18:14:55,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:14:55,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:14:55,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0e18ce46d8ec186aca70/lava_callback.json.gz 2026-03-09 18:14:55,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:14:55,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0e18ce46d8ec186aca70 HTTP/1.1" 200 None 2026-03-09 18:14:55,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:14:55,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:14:55,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:14:55,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0e18ce46d8ec186aca70 HTTP/1.1" 200 None 2026-03-09 18:14:55,617 - lava_callback - INFO - Completed processing callback for node 69af0e18ce46d8ec186aca70 INFO: 172.21.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51016 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:15:15,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:15:15,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52018 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:17:25,397 - lava_callback - INFO - Received callback for node 69af0d91ce46d8ec186aca54 INFO: 172.21.0.1:51536 - "POST /node/69af0d91ce46d8ec186aca54 HTTP/1.0" 202 Accepted 2026-03-09 18:17:25,403 - lava_callback - INFO - Starting async job submit for node 69af0d91ce46d8ec186aca54 2026-03-09 18:17:25,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:17:25,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0d91ce46d8ec186aca54 HTTP/1.1" 200 None 2026-03-09 18:17:25,453 - lava_callback - DEBUG - Processing node 69af0d91ce46d8ec186aca54: baseline-arm64-broonie 2026-03-09 18:17:25,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:17:25,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:17:25,869 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0d91ce46d8ec186aca54/log.txt.gz 2026-03-09 18:17:26,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:17:26,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:17:26,228 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0d91ce46d8ec186aca54/lava-logs.yaml 2026-03-09 18:17:26,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:17:26,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:17:26,403 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0d91ce46d8ec186aca54/lava_callback.json.gz DEBUG: Setup failed for 69af0d91ce46d8ec186aca54. Transitting job node result: incomplete -> fail 2026-03-09 18:17:26,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:17:26,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0d91ce46d8ec186aca54 HTTP/1.1" 200 None 2026-03-09 18:17:26,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:51552 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:17:26,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:17:26,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:17:26,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0d91ce46d8ec186aca54 HTTP/1.1" 200 None 2026-03-09 18:17:26,624 - lava_callback - INFO - Completed processing callback for node 69af0d91ce46d8ec186aca54 INFO: 172.21.0.1:50524 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:17:45,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:17:45,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57648 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:19:54,581 - lava_callback - INFO - Received callback for node 69af0f43ce46d8ec186aca9f INFO: 172.21.0.1:45900 - "POST /node/69af0f43ce46d8ec186aca9f HTTP/1.0" 202 Accepted 2026-03-09 18:19:54,587 - lava_callback - INFO - Starting async job submit for node 69af0f43ce46d8ec186aca9f 2026-03-09 18:19:54,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:19:54,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f43ce46d8ec186aca9f HTTP/1.1" 200 None 2026-03-09 18:19:54,610 - lava_callback - DEBUG - Processing node 69af0f43ce46d8ec186aca9f: baseline-arm64 2026-03-09 18:19:54,614 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:19:54,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:19:54,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:19:54,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0f43ce46d8ec186aca9f/lava-logs.yaml 2026-03-09 18:19:54,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:19:54,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:19:54,927 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0f43ce46d8ec186aca9f/lava_callback.json.gz 2026-03-09 18:19:54,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:19:54,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f43ce46d8ec186aca9f HTTP/1.1" 200 None 2026-03-09 18:19:54,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:19:54,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:19:54,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:19:54,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0f43ce46d8ec186aca9f HTTP/1.1" 200 None 2026-03-09 18:19:55,004 - lava_callback - INFO - Completed processing callback for node 69af0f43ce46d8ec186aca9f INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45262 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:20:11,961 - lava_callback - INFO - Received callback for node 69af01c8ce46d8ec186ac6e0 INFO: 172.21.0.1:51132 - "POST /node/69af01c8ce46d8ec186ac6e0 HTTP/1.0" 202 Accepted 2026-03-09 18:20:11,967 - lava_callback - INFO - Starting async job submit for node 69af01c8ce46d8ec186ac6e0 2026-03-09 18:20:12,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:20:12,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c8ce46d8ec186ac6e0 HTTP/1.1" 200 None 2026-03-09 18:20:12,017 - lava_callback - DEBUG - Processing node 69af01c8ce46d8ec186ac6e0: baseline-arm64-broonie 2026-03-09 18:20:12,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:20:12,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:20:12,530 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c8ce46d8ec186ac6e0/log.txt.gz 2026-03-09 18:20:12,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:20:12,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:20:12,988 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c8ce46d8ec186ac6e0/lava-logs.yaml 2026-03-09 18:20:12,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:20:13,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:20:13,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af01c8ce46d8ec186ac6e0/lava_callback.json.gz DEBUG: Setup failed for 69af01c8ce46d8ec186ac6e0. Transitting job node result: incomplete -> fail 2026-03-09 18:20:13,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:20:13,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af01c8ce46d8ec186ac6e0 HTTP/1.1" 200 None 2026-03-09 18:20:13,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:20:13,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:20:13,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:20:13,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af01c8ce46d8ec186ac6e0 HTTP/1.1" 200 None 2026-03-09 18:20:13,360 - lava_callback - INFO - Completed processing callback for node 69af01c8ce46d8ec186ac6e0 INFO: 172.21.0.1:51146 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:20:15,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:20:15,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59086 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:22:25,665 - lava_callback - INFO - Received callback for node 69af0de1ce46d8ec186aca6a INFO: 172.21.0.1:52566 - "POST /node/69af0de1ce46d8ec186aca6a HTTP/1.0" 202 Accepted 2026-03-09 18:22:25,671 - lava_callback - INFO - Starting async job submit for node 69af0de1ce46d8ec186aca6a 2026-03-09 18:22:25,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:22:25,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0de1ce46d8ec186aca6a HTTP/1.1" 200 None 2026-03-09 18:22:25,722 - lava_callback - DEBUG - Processing node 69af0de1ce46d8ec186aca6a: baseline-arm64-broonie 2026-03-09 18:22:26,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:22:26,210 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:22:26,212 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0de1ce46d8ec186aca6a/log.txt.gz 2026-03-09 18:22:26,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:52570 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:22:26,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:22:26,633 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0de1ce46d8ec186aca6a/lava-logs.yaml 2026-03-09 18:22:26,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:22:26,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:22:26,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0de1ce46d8ec186aca6a/lava_callback.json.gz DEBUG: Setup failed for 69af0de1ce46d8ec186aca6a. Transitting job node result: incomplete -> fail 2026-03-09 18:22:26,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:22:26,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0de1ce46d8ec186aca6a HTTP/1.1" 200 None 2026-03-09 18:22:26,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:22:26,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:22:26,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:22:26,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0de1ce46d8ec186aca6a HTTP/1.1" 200 None 2026-03-09 18:22:27,001 - lava_callback - INFO - Completed processing callback for node 69af0de1ce46d8ec186aca6a INFO: 172.21.0.1:42030 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:22:45,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:22:45,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38524 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:25:51,454 - lava_callback - INFO - Received callback for node 69af1080ce46d8ec186acadf INFO: 172.21.0.1:52902 - "POST /node/69af1080ce46d8ec186acadf HTTP/1.0" 202 Accepted 2026-03-09 18:25:51,459 - lava_callback - INFO - Starting async job submit for node 69af1080ce46d8ec186acadf 2026-03-09 18:25:51,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:25:51,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1080ce46d8ec186acadf HTTP/1.1" 200 None 2026-03-09 18:25:51,478 - lava_callback - DEBUG - Processing node 69af1080ce46d8ec186acadf: baseline-arm64-broonie 2026-03-09 18:25:51,483 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:25:51,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:25:51,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:25:51,630 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1080ce46d8ec186acadf/lava-logs.yaml 2026-03-09 18:25:51,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:25:51,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:25:51,772 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1080ce46d8ec186acadf/lava_callback.json.gz 2026-03-09 18:25:51,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:25:51,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1080ce46d8ec186acadf HTTP/1.1" 200 None 2026-03-09 18:25:51,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:25:51,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:25:51,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:25:51,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1080ce46d8ec186acadf HTTP/1.1" 200 None 2026-03-09 18:25:51,853 - lava_callback - INFO - Completed processing callback for node 69af1080ce46d8ec186acadf INFO: 172.21.0.1:52904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47640 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:26:15,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:26:15,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46352 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:27:33,669 - lava_callback - INFO - Received callback for node 69af0e62ce46d8ec186aca7b INFO: 172.21.0.1:36950 - "POST /node/69af0e62ce46d8ec186aca7b HTTP/1.0" 202 Accepted 2026-03-09 18:27:33,675 - lava_callback - INFO - Starting async job submit for node 69af0e62ce46d8ec186aca7b 2026-03-09 18:27:33,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:33,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0e62ce46d8ec186aca7b HTTP/1.1" 200 None 2026-03-09 18:27:33,723 - lava_callback - DEBUG - Processing node 69af0e62ce46d8ec186aca7b: baseline-arm64 2026-03-09 18:27:34,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:34,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:34,209 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0e62ce46d8ec186aca7b/log.txt.gz 2026-03-09 18:27:34,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:34,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:34,620 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0e62ce46d8ec186aca7b/lava-logs.yaml 2026-03-09 18:27:34,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:34,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:34,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0e62ce46d8ec186aca7b/lava_callback.json.gz 2026-03-09 18:27:34,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:34,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0e62ce46d8ec186aca7b HTTP/1.1" 200 None 2026-03-09 18:27:34,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:34,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:27:34,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:34,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0e62ce46d8ec186aca7b HTTP/1.1" 200 None 2026-03-09 18:27:34,972 - lava_callback - INFO - Completed processing callback for node 69af0e62ce46d8ec186aca7b INFO: 172.21.0.1:36966 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:27:40,905 - lava_callback - INFO - Received callback for node 69af0eafce46d8ec186aca83 INFO: 172.21.0.1:56684 - "POST /node/69af0eafce46d8ec186aca83 HTTP/1.0" 202 Accepted 2026-03-09 18:27:40,912 - lava_callback - INFO - Starting async job submit for node 69af0eafce46d8ec186aca83 2026-03-09 18:27:40,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:40,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0eafce46d8ec186aca83 HTTP/1.1" 200 None 2026-03-09 18:27:40,952 - lava_callback - DEBUG - Processing node 69af0eafce46d8ec186aca83: baseline-arm64 2026-03-09 18:27:41,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:41,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:41,778 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0eafce46d8ec186aca83/log.txt.gz 2026-03-09 18:27:42,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:42,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:42,244 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0eafce46d8ec186aca83/lava-logs.yaml 2026-03-09 18:27:42,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:27:42,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:27:42,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af0eafce46d8ec186aca83/lava_callback.json.gz 2026-03-09 18:27:42,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:42,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0eafce46d8ec186aca83 HTTP/1.1" 200 None 2026-03-09 18:27:42,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:42,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:27:42,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:42,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0eafce46d8ec186aca83 HTTP/1.1" 200 None 2026-03-09 18:27:42,557 - lava_callback - INFO - Completed processing callback for node 69af0eafce46d8ec186aca83 2026-03-09 18:27:45,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:27:45,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59720 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:30:12,918 - lava_callback - INFO - Received callback for node 69af1198ce46d8ec186acb14 INFO: 172.21.0.1:36238 - "POST /node/69af1198ce46d8ec186acb14 HTTP/1.0" 202 Accepted 2026-03-09 18:30:12,924 - lava_callback - INFO - Starting async job submit for node 69af1198ce46d8ec186acb14 2026-03-09 18:30:12,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:12,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1198ce46d8ec186acb14 HTTP/1.1" 200 None 2026-03-09 18:30:12,954 - lava_callback - DEBUG - Processing node 69af1198ce46d8ec186acb14: baseline-arm64 2026-03-09 18:30:12,959 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:30:12,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:30:13,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:30:13,127 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1198ce46d8ec186acb14/lava-logs.yaml 2026-03-09 18:30:13,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:30:13,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:30:13,288 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1198ce46d8ec186acb14/lava_callback.json.gz 2026-03-09 18:30:13,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:13,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1198ce46d8ec186acb14 HTTP/1.1" 200 None 2026-03-09 18:30:13,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:13,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:30:13,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:13,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1198ce46d8ec186acb14 HTTP/1.1" 200 None 2026-03-09 18:30:13,352 - lava_callback - INFO - Completed processing callback for node 69af1198ce46d8ec186acb14 INFO: 172.21.0.1:36248 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:30:15,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:15,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42848 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:30:59,206 - lava_callback - INFO - Received callback for node 69af0f00ce46d8ec186aca9a INFO: 172.21.0.1:42854 - "POST /node/69af0f00ce46d8ec186aca9a HTTP/1.0" 202 Accepted 2026-03-09 18:30:59,213 - lava_callback - INFO - Starting async job submit for node 69af0f00ce46d8ec186aca9a 2026-03-09 18:30:59,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:30:59,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f00ce46d8ec186aca9a HTTP/1.1" 200 None 2026-03-09 18:30:59,253 - lava_callback - DEBUG - Processing node 69af0f00ce46d8ec186aca9a: baseline-nfs-arm64 2026-03-09 18:30:59,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:30:59,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:30:59,982 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f00ce46d8ec186aca9a/log.txt.gz 2026-03-09 18:31:00,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:31:00,494 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:31:00,496 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f00ce46d8ec186aca9a/lava-logs.yaml 2026-03-09 18:31:00,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:31:00,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:31:00,651 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f00ce46d8ec186aca9a/lava_callback.json.gz 2026-03-09 18:31:00,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:31:00,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f00ce46d8ec186aca9a HTTP/1.1" 200 None 2026-03-09 18:31:00,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:31:00,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:31:00,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:31:00,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0f00ce46d8ec186aca9a HTTP/1.1" 200 None 2026-03-09 18:31:00,800 - lava_callback - INFO - Completed processing callback for node 69af0f00ce46d8ec186aca9a INFO: 172.21.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33936 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:31:15,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:31:15,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39382 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:32:23,785 - lava_callback - INFO - Received callback for node 69af0f94ce46d8ec186acab3 INFO: 172.21.0.1:57442 - "POST /node/69af0f94ce46d8ec186acab3 HTTP/1.0" 202 Accepted 2026-03-09 18:32:23,792 - lava_callback - INFO - Starting async job submit for node 69af0f94ce46d8ec186acab3 2026-03-09 18:32:23,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:32:23,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f94ce46d8ec186acab3 HTTP/1.1" 200 None 2026-03-09 18:32:23,846 - lava_callback - DEBUG - Processing node 69af0f94ce46d8ec186acab3: baseline-nfs-arm64 2026-03-09 18:32:24,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:32:24,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:32:24,327 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f94ce46d8ec186acab3/log.txt.gz 2026-03-09 18:32:24,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:32:24,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:32:24,798 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f94ce46d8ec186acab3/lava-logs.yaml 2026-03-09 18:32:24,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:32:24,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:32:24,975 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af0f94ce46d8ec186acab3/lava_callback.json.gz 2026-03-09 18:32:25,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:32:25,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0f94ce46d8ec186acab3 HTTP/1.1" 200 None 2026-03-09 18:32:25,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:32:25,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:32:25,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:32:25,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0f94ce46d8ec186acab3 HTTP/1.1" 200 None 2026-03-09 18:32:25,171 - lava_callback - INFO - Completed processing callback for node 69af0f94ce46d8ec186acab3 INFO: 172.21.0.1:57458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56440 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:32:45,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:32:45,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58854 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:33:28,540 - lava_callback - INFO - Received callback for node 69af1255ce46d8ec186acb43 INFO: 172.21.0.1:58870 - "POST /node/69af1255ce46d8ec186acb43 HTTP/1.0" 202 Accepted 2026-03-09 18:33:28,546 - lava_callback - INFO - Starting async job submit for node 69af1255ce46d8ec186acb43 2026-03-09 18:33:28,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:33:28,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1255ce46d8ec186acb43 HTTP/1.1" 200 None 2026-03-09 18:33:28,636 - lava_callback - DEBUG - Processing node 69af1255ce46d8ec186acb43: baseline-arm64 2026-03-09 18:33:28,686 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:33:28,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:33:28,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:33:28,926 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1255ce46d8ec186acb43/lava-logs.yaml 2026-03-09 18:33:28,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:33:29,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:33:29,076 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1255ce46d8ec186acb43/lava_callback.json.gz 2026-03-09 18:33:29,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:33:29,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1255ce46d8ec186acb43 HTTP/1.1" 200 None 2026-03-09 18:33:29,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:33:29,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:33:29,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:33:29,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1255ce46d8ec186acb43 HTTP/1.1" 200 None 2026-03-09 18:33:29,195 - lava_callback - INFO - Completed processing callback for node 69af1255ce46d8ec186acb43 INFO: 172.21.0.1:58896 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:33:45,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:33:45,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39622 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:34:12,702 - lava_callback - INFO - Received callback for node 69af115bce46d8ec186acb0f INFO: 172.21.0.1:39970 - "POST /node/69af115bce46d8ec186acb0f HTTP/1.0" 202 Accepted 2026-03-09 18:34:12,708 - lava_callback - INFO - Starting async job submit for node 69af115bce46d8ec186acb0f 2026-03-09 18:34:12,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:12,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af115bce46d8ec186acb0f HTTP/1.1" 200 None 2026-03-09 18:34:12,768 - lava_callback - DEBUG - Processing node 69af115bce46d8ec186acb0f: baseline-arm64-broonie 2026-03-09 18:34:13,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:34:13,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:34:13,350 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af115bce46d8ec186acb0f/log.txt.gz 2026-03-09 18:34:13,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:34:13,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:34:13,850 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af115bce46d8ec186acb0f/lava-logs.yaml 2026-03-09 18:34:13,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:34:14,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:34:14,050 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af115bce46d8ec186acb0f/lava_callback.json.gz DEBUG: Setup failed for 69af115bce46d8ec186acb0f. Transitting job node result: incomplete -> fail 2026-03-09 18:34:14,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:14,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af115bce46d8ec186acb0f HTTP/1.1" 200 None 2026-03-09 18:34:14,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:14,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:34:14,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:14,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af115bce46d8ec186acb0f HTTP/1.1" 200 None 2026-03-09 18:34:14,285 - lava_callback - INFO - Completed processing callback for node 69af115bce46d8ec186acb0f INFO: 172.21.0.1:39972 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:34:15,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:15,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37622 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:34:37,379 - lava_callback - INFO - Received callback for node 69af12a7ce46d8ec186acb6a INFO: 172.21.0.1:37630 - "POST /node/69af12a7ce46d8ec186acb6a HTTP/1.0" 202 Accepted 2026-03-09 18:34:37,385 - lava_callback - INFO - Starting async job submit for node 69af12a7ce46d8ec186acb6a 2026-03-09 18:34:37,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:37,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af12a7ce46d8ec186acb6a HTTP/1.1" 200 None 2026-03-09 18:34:37,402 - lava_callback - DEBUG - Processing node 69af12a7ce46d8ec186acb6a: baseline-nfs-arm64 2026-03-09 18:34:37,404 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:34:37,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:34:37,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:34:37,569 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af12a7ce46d8ec186acb6a/lava-logs.yaml 2026-03-09 18:34:37,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:34:37,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:34:37,718 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af12a7ce46d8ec186acb6a/lava_callback.json.gz 2026-03-09 18:34:37,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:37,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af12a7ce46d8ec186acb6a HTTP/1.1" 200 None 2026-03-09 18:34:37,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:37,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:34:37,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:37,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af12a7ce46d8ec186acb6a HTTP/1.1" 200 None 2026-03-09 18:34:37,782 - lava_callback - INFO - Completed processing callback for node 69af12a7ce46d8ec186acb6a 2026-03-09 18:34:45,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:34:45,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45868 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:36:11,081 - lava_callback - INFO - Received callback for node 69af12eece46d8ec186acb75 INFO: 172.21.0.1:40284 - "POST /node/69af12eece46d8ec186acb75 HTTP/1.0" 202 Accepted 2026-03-09 18:36:11,087 - lava_callback - INFO - Starting async job submit for node 69af12eece46d8ec186acb75 2026-03-09 18:36:11,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:11,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af12eece46d8ec186acb75 HTTP/1.1" 200 None 2026-03-09 18:36:11,119 - lava_callback - DEBUG - Processing node 69af12eece46d8ec186acb75: baseline-arm64 2026-03-09 18:36:11,185 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:36:11,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:36:11,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:36:11,360 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af12eece46d8ec186acb75/lava-logs.yaml 2026-03-09 18:36:11,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:36:11,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:36:11,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af12eece46d8ec186acb75/lava_callback.json.gz 2026-03-09 18:36:11,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:11,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af12eece46d8ec186acb75 HTTP/1.1" 200 None 2026-03-09 18:36:11,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:11,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:36:11,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:11,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af12eece46d8ec186acb75 HTTP/1.1" 200 None 2026-03-09 18:36:11,614 - lava_callback - INFO - Completed processing callback for node 69af12eece46d8ec186acb75 INFO: 172.21.0.1:40294 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:36:15,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:15,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40304 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:36:25,603 - lava_callback - INFO - Received callback for node 69af0bd2ce46d8ec186aca10 INFO: 172.21.0.1:35208 - "POST /node/69af0bd2ce46d8ec186aca10 HTTP/1.0" 202 Accepted 2026-03-09 18:36:25,608 - lava_callback - INFO - Starting async job submit for node 69af0bd2ce46d8ec186aca10 2026-03-09 18:36:25,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:25,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0bd2ce46d8ec186aca10 HTTP/1.1" 200 None 2026-03-09 18:36:25,642 - lava_callback - DEBUG - Processing node 69af0bd2ce46d8ec186aca10: baseline-arm64-broonie 2026-03-09 18:36:25,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:36:25,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:36:25,835 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0bd2ce46d8ec186aca10/log.txt.gz 2026-03-09 18:36:25,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:36:26,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:36:26,044 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0bd2ce46d8ec186aca10/lava-logs.yaml 2026-03-09 18:36:26,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:36:26,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:36:26,193 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af0bd2ce46d8ec186aca10/lava_callback.json.gz 2026-03-09 18:36:26,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:26,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0bd2ce46d8ec186aca10 HTTP/1.1" 200 None 2026-03-09 18:36:26,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:26,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:36:26,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:26,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0bd2ce46d8ec186aca10 HTTP/1.1" 200 None 2026-03-09 18:36:26,310 - lava_callback - INFO - Completed processing callback for node 69af0bd2ce46d8ec186aca10 INFO: 172.21.0.1:35214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35316 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:36:45,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:36:45,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39278 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:37:01,417 - lava_callback - INFO - Received callback for node 69af1334ce46d8ec186acb82 INFO: 172.21.0.1:42470 - "POST /node/69af1334ce46d8ec186acb82 HTTP/1.0" 202 Accepted 2026-03-09 18:37:01,423 - lava_callback - INFO - Starting async job submit for node 69af1334ce46d8ec186acb82 2026-03-09 18:37:01,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:37:01,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1334ce46d8ec186acb82 HTTP/1.1" 200 None 2026-03-09 18:37:01,465 - lava_callback - DEBUG - Processing node 69af1334ce46d8ec186acb82: baseline-arm64 2026-03-09 18:37:01,513 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:37:01,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:37:01,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:37:01,710 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1334ce46d8ec186acb82/lava-logs.yaml 2026-03-09 18:37:01,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:37:01,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:37:01,834 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1334ce46d8ec186acb82/lava_callback.json.gz 2026-03-09 18:37:01,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:37:01,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1334ce46d8ec186acb82 HTTP/1.1" 200 None 2026-03-09 18:37:01,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:37:01,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:37:01,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:37:01,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1334ce46d8ec186acb82 HTTP/1.1" 200 None 2026-03-09 18:37:01,949 - lava_callback - INFO - Completed processing callback for node 69af1334ce46d8ec186acb82 INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35720 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:37:15,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:37:15,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59694 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:39:26,955 - lava_callback - INFO - Received callback for node 69af0fd6ce46d8ec186acab9 INFO: 172.21.0.1:59710 - "POST /node/69af0fd6ce46d8ec186acab9 HTTP/1.0" 202 Accepted 2026-03-09 18:39:26,961 - lava_callback - INFO - Starting async job submit for node 69af0fd6ce46d8ec186acab9 2026-03-09 18:39:27,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:27,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0fd6ce46d8ec186acab9 HTTP/1.1" 200 None 2026-03-09 18:39:27,049 - lava_callback - DEBUG - Processing node 69af0fd6ce46d8ec186acab9: ltp-smoke 2026-03-09 18:39:27,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:39:27,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:39:27,732 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af0fd6ce46d8ec186acab9/log.txt.gz 2026-03-09 18:39:28,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:39:28,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:39:28,849 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af0fd6ce46d8ec186acab9/lava-logs.yaml 2026-03-09 18:39:28,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:39:29,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:39:29,055 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af0fd6ce46d8ec186acab9/lava_callback.json.gz 2026-03-09 18:39:29,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:29,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af0fd6ce46d8ec186acab9 HTTP/1.1" 200 None 2026-03-09 18:39:29,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:29,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef42ace46d8ec186ab814 HTTP/1.1" 200 None 2026-03-09 18:39:29,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:29,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af0fd6ce46d8ec186acab9 HTTP/1.1" 200 None 2026-03-09 18:39:29,352 - lava_callback - INFO - Completed processing callback for node 69af0fd6ce46d8ec186acab9 INFO: 172.21.0.1:40544 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:39:45,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:45,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36990 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:39:56,905 - lava_callback - INFO - Received callback for node 69af13dace46d8ec186acbaa INFO: 172.21.0.1:36994 - "POST /node/69af13dace46d8ec186acbaa HTTP/1.0" 202 Accepted 2026-03-09 18:39:56,911 - lava_callback - INFO - Starting async job submit for node 69af13dace46d8ec186acbaa 2026-03-09 18:39:56,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:56,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af13dace46d8ec186acbaa HTTP/1.1" 200 None 2026-03-09 18:39:56,929 - lava_callback - DEBUG - Processing node 69af13dace46d8ec186acbaa: baseline-nfs-arm64 2026-03-09 18:39:56,934 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:39:56,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:39:57,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:39:57,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af13dace46d8ec186acbaa/lava-logs.yaml 2026-03-09 18:39:57,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:39:57,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:39:57,211 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af13dace46d8ec186acbaa/lava_callback.json.gz 2026-03-09 18:39:57,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:57,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af13dace46d8ec186acbaa HTTP/1.1" 200 None 2026-03-09 18:39:57,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:57,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:39:57,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:39:57,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af13dace46d8ec186acbaa HTTP/1.1" 200 None 2026-03-09 18:39:57,277 - lava_callback - INFO - Completed processing callback for node 69af13dace46d8ec186acbaa INFO: 172.21.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40820 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:40:15,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:40:16,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47432 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:41:50,351 - lava_callback - INFO - Received callback for node 69ac716946f7ae8ff3366031 INFO: 172.21.0.1:38506 - "POST /node/69ac716946f7ae8ff3366031 HTTP/1.0" 202 Accepted 2026-03-09 18:41:50,356 - lava_callback - INFO - Starting async job submit for node 69ac716946f7ae8ff3366031 2026-03-09 18:41:50,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:41:50,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac716946f7ae8ff3366031 HTTP/1.1" 200 None 2026-03-09 18:41:50,372 - lava_callback - DEBUG - Processing node 69ac716946f7ae8ff3366031: baseline-arm64-baylibre 2026-03-09 18:41:50,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac716946f7ae8ff3366031 2026-03-09 18:41:50,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:41:50,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac716946f7ae8ff3366031 HTTP/1.1" 200 None 2026-03-09 18:41:50,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:41:50,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:41:50,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:41:50,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac716946f7ae8ff3366031 HTTP/1.1" 200 None 2026-03-09 18:41:50,427 - lava_callback - ERROR - Error processing callback for node 69ac716946f7ae8ff3366031: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47222 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:42:24,976 - lava_callback - INFO - Received callback for node 69af1485ce46d8ec186acbd4 INFO: 172.21.0.1:40040 - "POST /node/69af1485ce46d8ec186acbd4 HTTP/1.0" 202 Accepted 2026-03-09 18:42:24,982 - lava_callback - INFO - Starting async job submit for node 69af1485ce46d8ec186acbd4 2026-03-09 18:42:24,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:42:25,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1485ce46d8ec186acbd4 HTTP/1.1" 200 None 2026-03-09 18:42:25,014 - lava_callback - DEBUG - Processing node 69af1485ce46d8ec186acbd4: baseline-arm64-foundriesio 2026-03-09 18:42:25,064 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:42:25,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:42:25,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:42:25,291 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af1485ce46d8ec186acbd4/lava-logs.yaml 2026-03-09 18:42:25,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:42:25,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:42:25,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af1485ce46d8ec186acbd4/lava_callback.json.gz 2026-03-09 18:42:25,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:42:25,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1485ce46d8ec186acbd4 HTTP/1.1" 200 None 2026-03-09 18:42:25,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:42:25,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:42:25,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:42:25,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1485ce46d8ec186acbd4 HTTP/1.1" 200 None 2026-03-09 18:42:25,560 - lava_callback - INFO - Completed processing callback for node 69af1485ce46d8ec186acbd4 INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35032 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:42:46,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:42:46,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59382 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:43:10,376 - lava_callback - INFO - Received callback for node 69ac71b946f7ae8ff3366057 INFO: 172.21.0.1:35034 - "POST /node/69ac71b946f7ae8ff3366057 HTTP/1.0" 202 Accepted 2026-03-09 18:43:10,382 - lava_callback - INFO - Starting async job submit for node 69ac71b946f7ae8ff3366057 2026-03-09 18:43:10,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:43:10,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac71b946f7ae8ff3366057 HTTP/1.1" 200 None 2026-03-09 18:43:10,398 - lava_callback - DEBUG - Processing node 69ac71b946f7ae8ff3366057: baseline-arm64-baylibre 2026-03-09 18:43:10,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac71b946f7ae8ff3366057 2026-03-09 18:43:10,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:43:10,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac71b946f7ae8ff3366057 HTTP/1.1" 200 None 2026-03-09 18:43:10,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:43:10,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:43:10,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:43:10,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac71b946f7ae8ff3366057 HTTP/1.1" 200 None 2026-03-09 18:43:10,480 - lava_callback - ERROR - Error processing callback for node 69ac71b946f7ae8ff3366057: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:44:30,446 - lava_callback - INFO - Received callback for node 69ac720a46f7ae8ff3366074 INFO: 172.21.0.1:38584 - "POST /node/69ac720a46f7ae8ff3366074 HTTP/1.0" 202 Accepted 2026-03-09 18:44:30,452 - lava_callback - INFO - Starting async job submit for node 69ac720a46f7ae8ff3366074 2026-03-09 18:44:30,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:44:30,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac720a46f7ae8ff3366074 HTTP/1.1" 200 None 2026-03-09 18:44:30,468 - lava_callback - DEBUG - Processing node 69ac720a46f7ae8ff3366074: baseline-arm64-baylibre 2026-03-09 18:44:30,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac720a46f7ae8ff3366074 2026-03-09 18:44:30,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:44:30,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac720a46f7ae8ff3366074 HTTP/1.1" 200 None 2026-03-09 18:44:30,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:44:30,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:44:30,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:44:30,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac720a46f7ae8ff3366074 HTTP/1.1" 200 None 2026-03-09 18:44:30,520 - lava_callback - ERROR - Error processing callback for node 69ac720a46f7ae8ff3366074: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58284 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:46:07,312 - lava_callback - INFO - Received callback for node 69af14cace46d8ec186acbe3 INFO: 172.21.0.1:58300 - "POST /node/69af14cace46d8ec186acbe3 HTTP/1.0" 202 Accepted 2026-03-09 18:46:07,318 - lava_callback - INFO - Starting async job submit for node 69af14cace46d8ec186acbe3 2026-03-09 18:46:07,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:07,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af14cace46d8ec186acbe3 HTTP/1.1" 200 None 2026-03-09 18:46:07,374 - lava_callback - DEBUG - Processing node 69af14cace46d8ec186acbe3: baseline-arm64-broonie 2026-03-09 18:46:07,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:46:07,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:46:07,785 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af14cace46d8ec186acbe3/log.txt.gz 2026-03-09 18:46:07,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:46:08,162 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:46:08,164 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af14cace46d8ec186acbe3/lava-logs.yaml 2026-03-09 18:46:08,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:46:08,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:46:08,332 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af14cace46d8ec186acbe3/lava_callback.json.gz DEBUG: Setup failed for 69af14cace46d8ec186acbe3. Transitting job node result: incomplete -> fail 2026-03-09 18:46:08,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:08,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af14cace46d8ec186acbe3 HTTP/1.1" 200 None 2026-03-09 18:46:08,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:08,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:46:08,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:08,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af14cace46d8ec186acbe3 HTTP/1.1" 200 None 2026-03-09 18:46:08,579 - lava_callback - INFO - Completed processing callback for node 69af14cace46d8ec186acbe3 2026-03-09 18:46:10,562 - lava_callback - INFO - Received callback for node 69ac726f46f7ae8ff336608b INFO: 172.21.0.1:58400 - "POST /node/69ac726f46f7ae8ff336608b HTTP/1.0" 202 Accepted 2026-03-09 18:46:10,567 - lava_callback - INFO - Starting async job submit for node 69ac726f46f7ae8ff336608b 2026-03-09 18:46:10,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:10,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac726f46f7ae8ff336608b HTTP/1.1" 200 None 2026-03-09 18:46:10,584 - lava_callback - DEBUG - Processing node 69ac726f46f7ae8ff336608b: baseline-arm64-baylibre 2026-03-09 18:46:10,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac726f46f7ae8ff336608b 2026-03-09 18:46:10,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:10,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac726f46f7ae8ff336608b HTTP/1.1" 200 None 2026-03-09 18:46:10,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:10,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:46:10,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:10,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac726f46f7ae8ff336608b HTTP/1.1" 200 None 2026-03-09 18:46:10,641 - lava_callback - ERROR - Error processing callback for node 69ac726f46f7ae8ff336608b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58416 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:46:16,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:46:16,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33414 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:47:30,625 - lava_callback - INFO - Received callback for node 69ac72c046f7ae8ff33660ac INFO: 172.21.0.1:50538 - "POST /node/69ac72c046f7ae8ff33660ac HTTP/1.0" 202 Accepted 2026-03-09 18:47:30,630 - lava_callback - INFO - Starting async job submit for node 69ac72c046f7ae8ff33660ac 2026-03-09 18:47:30,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:30,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac72c046f7ae8ff33660ac HTTP/1.1" 200 None 2026-03-09 18:47:30,647 - lava_callback - DEBUG - Processing node 69ac72c046f7ae8ff33660ac: baseline-arm64-baylibre 2026-03-09 18:47:30,647 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac72c046f7ae8ff33660ac 2026-03-09 18:47:30,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:30,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac72c046f7ae8ff33660ac HTTP/1.1" 200 None 2026-03-09 18:47:30,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:30,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:47:30,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:30,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac72c046f7ae8ff33660ac HTTP/1.1" 200 None 2026-03-09 18:47:30,731 - lava_callback - ERROR - Error processing callback for node 69ac72c046f7ae8ff33660ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50546 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:47:39,482 - lava_callback - INFO - Received callback for node 69af15b2ce46d8ec186acc1d INFO: 172.21.0.1:50562 - "POST /node/69af15b2ce46d8ec186acc1d HTTP/1.0" 202 Accepted 2026-03-09 18:47:39,487 - lava_callback - INFO - Starting async job submit for node 69af15b2ce46d8ec186acc1d 2026-03-09 18:47:39,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:39,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af15b2ce46d8ec186acc1d HTTP/1.1" 200 None 2026-03-09 18:47:39,506 - lava_callback - DEBUG - Processing node 69af15b2ce46d8ec186acc1d: baseline-arm64-clabbe 2026-03-09 18:47:39,510 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:47:39,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:47:39,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:47:39,685 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af15b2ce46d8ec186acc1d/lava-logs.yaml 2026-03-09 18:47:39,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:47:39,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:47:39,821 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af15b2ce46d8ec186acc1d/lava_callback.json.gz 2026-03-09 18:47:39,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:39,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af15b2ce46d8ec186acc1d HTTP/1.1" 200 None 2026-03-09 18:47:39,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:39,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:47:39,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:39,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af15b2ce46d8ec186acc1d HTTP/1.1" 200 None 2026-03-09 18:47:39,887 - lava_callback - INFO - Completed processing callback for node 69af15b2ce46d8ec186acc1d 2026-03-09 18:47:46,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:46,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 18:47:46,402 - lava_callback - INFO - Received callback for node 69af1562ce46d8ec186acc06 INFO: 172.21.0.1:35560 - "POST /node/69af1562ce46d8ec186acc06 HTTP/1.0" 202 Accepted 2026-03-09 18:47:46,409 - lava_callback - INFO - Starting async job submit for node 69af1562ce46d8ec186acc06 2026-03-09 18:47:46,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:46,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1562ce46d8ec186acc06 HTTP/1.1" 200 None 2026-03-09 18:47:46,469 - lava_callback - DEBUG - Processing node 69af1562ce46d8ec186acc06: baseline-arm64-broonie INFO: 172.21.0.1:35564 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:47:46,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:47:46,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:47:46,815 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1562ce46d8ec186acc06/log.txt.gz 2026-03-09 18:47:46,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:47:47,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:47:47,158 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1562ce46d8ec186acc06/lava-logs.yaml 2026-03-09 18:47:47,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:47:47,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:47:47,313 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1562ce46d8ec186acc06/lava_callback.json.gz DEBUG: Setup failed for 69af1562ce46d8ec186acc06. Transitting job node result: incomplete -> fail 2026-03-09 18:47:47,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:47,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1562ce46d8ec186acc06 HTTP/1.1" 200 None 2026-03-09 18:47:47,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:47,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:47:47,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:47:47,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1562ce46d8ec186acc06 HTTP/1.1" 200 None 2026-03-09 18:47:47,557 - lava_callback - INFO - Completed processing callback for node 69af1562ce46d8ec186acc06 INFO: 172.21.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54156 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:48:15,409 - lava_callback - INFO - Received callback for node 69af122bce46d8ec186acb2b INFO: 172.21.0.1:54166 - "POST /node/69af122bce46d8ec186acb2b HTTP/1.0" 202 Accepted 2026-03-09 18:48:15,416 - lava_callback - INFO - Starting async job submit for node 69af122bce46d8ec186acb2b 2026-03-09 18:48:15,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:15,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af122bce46d8ec186acb2b HTTP/1.1" 200 None 2026-03-09 18:48:15,490 - lava_callback - DEBUG - Processing node 69af122bce46d8ec186acb2b: baseline-arm 2026-03-09 18:48:16,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:16,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:48:16,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 18:48:16,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:48:16,217 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af122bce46d8ec186acb2b/log.txt.gz INFO: 172.21.0.1:54176 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:48:16,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:48:16,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:48:16,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af122bce46d8ec186acb2b/lava-logs.yaml 2026-03-09 18:48:16,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:48:17,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:48:17,007 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af122bce46d8ec186acb2b/lava_callback.json.gz DEBUG: Setup failed for 69af122bce46d8ec186acb2b. Transitting job node result: incomplete -> fail 2026-03-09 18:48:17,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:17,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af122bce46d8ec186acb2b HTTP/1.1" 200 None 2026-03-09 18:48:17,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:17,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 18:48:17,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:17,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af122bce46d8ec186acb2b HTTP/1.1" 200 None 2026-03-09 18:48:17,293 - lava_callback - INFO - Completed processing callback for node 69af122bce46d8ec186acb2b INFO: 172.21.0.1:39672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60186 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:48:46,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:48:46,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:49:10,766 - lava_callback - INFO - Received callback for node 69ac732446f7ae8ff33660bf INFO: 172.21.0.1:48448 - "POST /node/69ac732446f7ae8ff33660bf HTTP/1.0" 202 Accepted 2026-03-09 18:49:10,772 - lava_callback - INFO - Starting async job submit for node 69ac732446f7ae8ff33660bf 2026-03-09 18:49:10,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:10,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac732446f7ae8ff33660bf HTTP/1.1" 200 None 2026-03-09 18:49:10,790 - lava_callback - DEBUG - Processing node 69ac732446f7ae8ff33660bf: baseline-arm64-baylibre 2026-03-09 18:49:10,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac732446f7ae8ff33660bf 2026-03-09 18:49:10,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:10,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac732446f7ae8ff33660bf HTTP/1.1" 200 None 2026-03-09 18:49:10,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:10,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a70fafe31b554803095f11 HTTP/1.1" 200 None 2026-03-09 18:49:10,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:10,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac732446f7ae8ff33660bf HTTP/1.1" 200 None 2026-03-09 18:49:10,851 - lava_callback - ERROR - Error processing callback for node 69ac732446f7ae8ff33660bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 18:49:12,364 - lava_callback - INFO - Received callback for node 69af15ffce46d8ec186acc48 INFO: 172.21.0.1:48452 - "POST /node/69af15ffce46d8ec186acc48 HTTP/1.0" 202 Accepted 2026-03-09 18:49:12,370 - lava_callback - INFO - Starting async job submit for node 69af15ffce46d8ec186acc48 2026-03-09 18:49:12,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:12,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af15ffce46d8ec186acc48 HTTP/1.1" 200 None 2026-03-09 18:49:12,392 - lava_callback - DEBUG - Processing node 69af15ffce46d8ec186acc48: baseline-arm64-broonie 2026-03-09 18:49:12,397 - lava_callback - WARNING - Failed to upload log 2026-03-09 18:49:12,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:49:12,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:49:12,534 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af15ffce46d8ec186acc48/lava-logs.yaml 2026-03-09 18:49:12,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:49:12,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:49:12,677 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af15ffce46d8ec186acc48/lava_callback.json.gz 2026-03-09 18:49:12,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:12,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af15ffce46d8ec186acc48 HTTP/1.1" 200 None 2026-03-09 18:49:12,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:12,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:49:12,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:12,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af15ffce46d8ec186acc48 HTTP/1.1" 200 None 2026-03-09 18:49:12,739 - lava_callback - INFO - Completed processing callback for node 69af15ffce46d8ec186acc48 INFO: 172.21.0.1:48454 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:49:16,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:49:16,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60130 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:51:47,671 - lava_callback - INFO - Received callback for node 69af165fce46d8ec186acc7d INFO: 172.21.0.1:60146 - "POST /node/69af165fce46d8ec186acc7d HTTP/1.0" 202 Accepted 2026-03-09 18:51:47,677 - lava_callback - INFO - Starting async job submit for node 69af165fce46d8ec186acc7d 2026-03-09 18:51:47,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:51:47,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af165fce46d8ec186acc7d HTTP/1.1" 200 None 2026-03-09 18:51:47,727 - lava_callback - DEBUG - Processing node 69af165fce46d8ec186acc7d: baseline-arm64 2026-03-09 18:51:48,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:51:48,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:51:48,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af165fce46d8ec186acc7d/log.txt.gz 2026-03-09 18:51:48,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:51:48,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:51:48,805 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af165fce46d8ec186acc7d/lava-logs.yaml 2026-03-09 18:51:48,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:51:48,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:51:48,978 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af165fce46d8ec186acc7d/lava_callback.json.gz DEBUG: Setup failed for 69af165fce46d8ec186acc7d. Transitting job node result: incomplete -> fail 2026-03-09 18:51:49,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:51:49,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af165fce46d8ec186acc7d HTTP/1.1" 200 None 2026-03-09 18:51:49,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:51:49,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:51:49,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:51:49,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af165fce46d8ec186acc7d HTTP/1.1" 200 None 2026-03-09 18:51:49,195 - lava_callback - INFO - Completed processing callback for node 69af165fce46d8ec186acc7d INFO: 172.21.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:52:16,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:16,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:52:30,980 - lava_callback - INFO - Received callback for node 69ac73dc46f7ae8ff33660fd INFO: 172.21.0.1:44132 - "POST /node/69ac73dc46f7ae8ff33660fd HTTP/1.0" 202 Accepted 2026-03-09 18:52:30,986 - lava_callback - INFO - Starting async job submit for node 69ac73dc46f7ae8ff33660fd 2026-03-09 18:52:30,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:31,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac73dc46f7ae8ff33660fd HTTP/1.1" 200 None 2026-03-09 18:52:31,025 - lava_callback - DEBUG - Processing node 69ac73dc46f7ae8ff33660fd: baseline-arm-baylibre 2026-03-09 18:52:31,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac73dc46f7ae8ff33660fd 2026-03-09 18:52:31,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:31,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac73dc46f7ae8ff33660fd HTTP/1.1" 200 None 2026-03-09 18:52:31,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:31,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:52:31,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:31,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac73dc46f7ae8ff33660fd HTTP/1.1" 200 None 2026-03-09 18:52:31,080 - lava_callback - ERROR - Error processing callback for node 69ac73dc46f7ae8ff33660fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44134 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:52:45,012 - lava_callback - INFO - Received callback for node 69af16abce46d8ec186acc86 INFO: 172.21.0.1:48728 - "POST /node/69af16abce46d8ec186acc86 HTTP/1.0" 202 Accepted 2026-03-09 18:52:45,019 - lava_callback - INFO - Starting async job submit for node 69af16abce46d8ec186acc86 2026-03-09 18:52:45,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:45,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af16abce46d8ec186acc86 HTTP/1.1" 200 None 2026-03-09 18:52:45,083 - lava_callback - DEBUG - Processing node 69af16abce46d8ec186acc86: baseline-arm64 2026-03-09 18:52:45,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:52:46,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:52:46,085 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16abce46d8ec186acc86/log.txt.gz INFO: 172.21.0.1:48744 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:52:46,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:52:46,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:52:46,795 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16abce46d8ec186acc86/lava-logs.yaml 2026-03-09 18:52:46,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:52:46,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:52:46,957 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16abce46d8ec186acc86/lava_callback.json.gz DEBUG: Setup failed for 69af16abce46d8ec186acc86. Transitting job node result: incomplete -> fail 2026-03-09 18:52:47,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:47,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af16abce46d8ec186acc86 HTTP/1.1" 200 None 2026-03-09 18:52:47,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:47,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:52:47,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:52:47,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af16abce46d8ec186acc86 HTTP/1.1" 200 None 2026-03-09 18:52:47,178 - lava_callback - INFO - Completed processing callback for node 69af16abce46d8ec186acc86 INFO: 172.21.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33206 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:53:11,004 - lava_callback - INFO - Received callback for node 69ac741046f7ae8ff336610f INFO: 172.21.0.1:49108 - "POST /node/69ac741046f7ae8ff336610f HTTP/1.0" 202 Accepted 2026-03-09 18:53:11,009 - lava_callback - INFO - Starting async job submit for node 69ac741046f7ae8ff336610f 2026-03-09 18:53:11,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:53:11,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac741046f7ae8ff336610f HTTP/1.1" 200 None 2026-03-09 18:53:11,026 - lava_callback - DEBUG - Processing node 69ac741046f7ae8ff336610f: baseline-arm-baylibre 2026-03-09 18:53:11,026 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac741046f7ae8ff336610f 2026-03-09 18:53:11,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:53:11,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac741046f7ae8ff336610f HTTP/1.1" 200 None 2026-03-09 18:53:11,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:53:11,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:53:11,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:53:11,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac741046f7ae8ff336610f HTTP/1.1" 200 None 2026-03-09 18:53:11,078 - lava_callback - ERROR - Error processing callback for node 69ac741046f7ae8ff336610f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49112 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:53:16,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:53:16,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45074 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:54:11,037 - lava_callback - INFO - Received callback for node 69ac744d46f7ae8ff3366131 INFO: 172.21.0.1:44738 - "POST /node/69ac744d46f7ae8ff3366131 HTTP/1.0" 202 Accepted 2026-03-09 18:54:11,042 - lava_callback - INFO - Starting async job submit for node 69ac744d46f7ae8ff3366131 2026-03-09 18:54:11,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:11,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac744d46f7ae8ff3366131 HTTP/1.1" 200 None 2026-03-09 18:54:11,061 - lava_callback - DEBUG - Processing node 69ac744d46f7ae8ff3366131: baseline-arm-baylibre 2026-03-09 18:54:11,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac744d46f7ae8ff3366131 2026-03-09 18:54:11,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:11,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac744d46f7ae8ff3366131 HTTP/1.1" 200 None 2026-03-09 18:54:11,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:11,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:54:11,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:11,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac744d46f7ae8ff3366131 HTTP/1.1" 200 None 2026-03-09 18:54:11,164 - lava_callback - ERROR - Error processing callback for node 69ac744d46f7ae8ff3366131: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 18:54:13,495 - lava_callback - INFO - Received callback for node 69af16f9ce46d8ec186acc97 INFO: 172.21.0.1:44742 - "POST /node/69af16f9ce46d8ec186acc97 HTTP/1.0" 202 Accepted 2026-03-09 18:54:13,501 - lava_callback - INFO - Starting async job submit for node 69af16f9ce46d8ec186acc97 2026-03-09 18:54:13,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:13,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af16f9ce46d8ec186acc97 HTTP/1.1" 200 None 2026-03-09 18:54:13,560 - lava_callback - DEBUG - Processing node 69af16f9ce46d8ec186acc97: baseline-arm64 2026-03-09 18:54:13,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:54:14,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:54:14,047 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16f9ce46d8ec186acc97/log.txt.gz 2026-03-09 18:54:14,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:54:14,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:54:14,479 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16f9ce46d8ec186acc97/lava-logs.yaml 2026-03-09 18:54:14,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:54:14,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:54:14,618 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af16f9ce46d8ec186acc97/lava_callback.json.gz DEBUG: Setup failed for 69af16f9ce46d8ec186acc97. Transitting job node result: incomplete -> fail 2026-03-09 18:54:14,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:14,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af16f9ce46d8ec186acc97 HTTP/1.1" 200 None 2026-03-09 18:54:14,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:14,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:54:14,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:14,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af16f9ce46d8ec186acc97 HTTP/1.1" 200 None 2026-03-09 18:54:14,809 - lava_callback - INFO - Completed processing callback for node 69af16f9ce46d8ec186acc97 INFO: 172.21.0.1:44744 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:54:16,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:16,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52874 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:54:51,030 - lava_callback - INFO - Received callback for node 69ac747146f7ae8ff3366145 INFO: 172.21.0.1:43118 - "POST /node/69ac747146f7ae8ff3366145 HTTP/1.0" 202 Accepted 2026-03-09 18:54:51,035 - lava_callback - INFO - Starting async job submit for node 69ac747146f7ae8ff3366145 2026-03-09 18:54:51,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:51,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac747146f7ae8ff3366145 HTTP/1.1" 200 None 2026-03-09 18:54:51,051 - lava_callback - DEBUG - Processing node 69ac747146f7ae8ff3366145: baseline-arm-baylibre 2026-03-09 18:54:51,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac747146f7ae8ff3366145 2026-03-09 18:54:51,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:51,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac747146f7ae8ff3366145 HTTP/1.1" 200 None 2026-03-09 18:54:51,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:51,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:54:51,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:54:51,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac747146f7ae8ff3366145 HTTP/1.1" 200 None 2026-03-09 18:54:51,159 - lava_callback - ERROR - Error processing callback for node 69ac747146f7ae8ff3366145: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38390 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:55:31,079 - lava_callback - INFO - Received callback for node 69ac749546f7ae8ff3366159 INFO: 172.21.0.1:36890 - "POST /node/69ac749546f7ae8ff3366159 HTTP/1.0" 202 Accepted 2026-03-09 18:55:31,084 - lava_callback - INFO - Starting async job submit for node 69ac749546f7ae8ff3366159 2026-03-09 18:55:31,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:31,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac749546f7ae8ff3366159 HTTP/1.1" 200 None 2026-03-09 18:55:31,101 - lava_callback - DEBUG - Processing node 69ac749546f7ae8ff3366159: baseline-arm-baylibre 2026-03-09 18:55:31,101 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac749546f7ae8ff3366159 2026-03-09 18:55:31,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:31,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac749546f7ae8ff3366159 HTTP/1.1" 200 None 2026-03-09 18:55:31,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:31,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:55:31,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:31,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac749546f7ae8ff3366159 HTTP/1.1" 200 None 2026-03-09 18:55:31,170 - lava_callback - ERROR - Error processing callback for node 69ac749546f7ae8ff3366159: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 18:55:36,368 - lava_callback - INFO - Received callback for node 69af1742ce46d8ec186acca4 INFO: 172.21.0.1:36902 - "POST /node/69af1742ce46d8ec186acca4 HTTP/1.0" 202 Accepted 2026-03-09 18:55:36,374 - lava_callback - INFO - Starting async job submit for node 69af1742ce46d8ec186acca4 2026-03-09 18:55:36,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:36,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1742ce46d8ec186acca4 HTTP/1.1" 200 None 2026-03-09 18:55:36,420 - lava_callback - DEBUG - Processing node 69af1742ce46d8ec186acca4: baseline-arm64 INFO: 172.21.0.1:36908 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:55:36,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:55:36,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:55:36,911 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1742ce46d8ec186acca4/log.txt.gz 2026-03-09 18:55:37,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:55:37,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:55:37,357 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1742ce46d8ec186acca4/lava-logs.yaml 2026-03-09 18:55:37,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:55:37,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:55:37,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1742ce46d8ec186acca4/lava_callback.json.gz DEBUG: Setup failed for 69af1742ce46d8ec186acca4. Transitting job node result: incomplete -> fail 2026-03-09 18:55:37,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:37,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1742ce46d8ec186acca4 HTTP/1.1" 200 None 2026-03-09 18:55:37,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:37,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:55:37,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:37,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1742ce46d8ec186acca4 HTTP/1.1" 200 None 2026-03-09 18:55:37,722 - lava_callback - INFO - Completed processing callback for node 69af1742ce46d8ec186acca4 2026-03-09 18:55:46,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:55:46,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45374 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:57:07,449 - lava_callback - INFO - Received callback for node 69af178dce46d8ec186accb6 INFO: 172.21.0.1:45384 - "POST /node/69af178dce46d8ec186accb6 HTTP/1.0" 202 Accepted 2026-03-09 18:57:07,455 - lava_callback - INFO - Starting async job submit for node 69af178dce46d8ec186accb6 2026-03-09 18:57:07,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:07,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af178dce46d8ec186accb6 HTTP/1.1" 200 None 2026-03-09 18:57:07,502 - lava_callback - DEBUG - Processing node 69af178dce46d8ec186accb6: baseline-arm64 2026-03-09 18:57:08,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:57:08,189 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:57:08,191 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af178dce46d8ec186accb6/log.txt.gz 2026-03-09 18:57:08,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:57:08,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:57:08,726 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af178dce46d8ec186accb6/lava-logs.yaml 2026-03-09 18:57:08,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:57:08,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:57:08,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af178dce46d8ec186accb6/lava_callback.json.gz DEBUG: Setup failed for 69af178dce46d8ec186accb6. Transitting job node result: incomplete -> fail 2026-03-09 18:57:08,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:08,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af178dce46d8ec186accb6 HTTP/1.1" 200 None 2026-03-09 18:57:08,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:08,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:57:08,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:09,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af178dce46d8ec186accb6 HTTP/1.1" 200 None 2026-03-09 18:57:09,084 - lava_callback - INFO - Completed processing callback for node 69af178dce46d8ec186accb6 INFO: 172.21.0.1:58524 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 18:57:16,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:16,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58538 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:57:23,208 - lava_callback - INFO - Received callback for node 69ac750546f7ae8ff3366190 INFO: 172.21.0.1:49804 - "POST /node/69ac750546f7ae8ff3366190 HTTP/1.0" 202 Accepted 2026-03-09 18:57:23,213 - lava_callback - INFO - Starting async job submit for node 69ac750546f7ae8ff3366190 2026-03-09 18:57:23,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:23,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac750546f7ae8ff3366190 HTTP/1.1" 200 None 2026-03-09 18:57:23,255 - lava_callback - DEBUG - Processing node 69ac750546f7ae8ff3366190: baseline-arm-clabbe 2026-03-09 18:57:23,255 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac750546f7ae8ff3366190 2026-03-09 18:57:23,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:23,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac750546f7ae8ff3366190 HTTP/1.1" 200 None 2026-03-09 18:57:23,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:23,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:57:23,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:57:23,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac750546f7ae8ff3366190 HTTP/1.1" 200 None 2026-03-09 18:57:23,339 - lava_callback - ERROR - Error processing callback for node 69ac750546f7ae8ff3366190: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:58:03,185 - lava_callback - INFO - Received callback for node 69ac752946f7ae8ff336619d INFO: 172.21.0.1:53708 - "POST /node/69ac752946f7ae8ff336619d HTTP/1.0" 202 Accepted 2026-03-09 18:58:03,191 - lava_callback - INFO - Starting async job submit for node 69ac752946f7ae8ff336619d 2026-03-09 18:58:03,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:03,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac752946f7ae8ff336619d HTTP/1.1" 200 None 2026-03-09 18:58:03,211 - lava_callback - DEBUG - Processing node 69ac752946f7ae8ff336619d: baseline-arm-clabbe 2026-03-09 18:58:03,212 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac752946f7ae8ff336619d 2026-03-09 18:58:03,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:03,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac752946f7ae8ff336619d HTTP/1.1" 200 None 2026-03-09 18:58:03,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:03,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:58:03,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:03,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac752946f7ae8ff336619d HTTP/1.1" 200 None 2026-03-09 18:58:03,269 - lava_callback - ERROR - Error processing callback for node 69ac752946f7ae8ff336619d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34954 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:58:23,967 - lava_callback - INFO - Received callback for node 69af17ddce46d8ec186accc6 INFO: 172.21.0.1:58822 - "POST /node/69af17ddce46d8ec186accc6 HTTP/1.0" 202 Accepted 2026-03-09 18:58:23,974 - lava_callback - INFO - Starting async job submit for node 69af17ddce46d8ec186accc6 2026-03-09 18:58:24,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:24,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af17ddce46d8ec186accc6 HTTP/1.1" 200 None 2026-03-09 18:58:24,021 - lava_callback - DEBUG - Processing node 69af17ddce46d8ec186accc6: baseline-nfs-arm64 2026-03-09 18:58:24,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:58:24,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:58:24,583 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af17ddce46d8ec186accc6/log.txt.gz 2026-03-09 18:58:24,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:58:25,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:58:25,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af17ddce46d8ec186accc6/lava-logs.yaml 2026-03-09 18:58:25,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:58:25,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:58:25,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af17ddce46d8ec186accc6/lava_callback.json.gz DEBUG: Setup failed for 69af17ddce46d8ec186accc6. Transitting job node result: incomplete -> fail 2026-03-09 18:58:25,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:25,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af17ddce46d8ec186accc6 HTTP/1.1" 200 None 2026-03-09 18:58:25,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:25,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:58:25,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:25,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af17ddce46d8ec186accc6 HTTP/1.1" 200 None 2026-03-09 18:58:25,473 - lava_callback - INFO - Completed processing callback for node 69af17ddce46d8ec186accc6 INFO: 172.21.0.1:58828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36688 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:58:43,260 - lava_callback - INFO - Received callback for node 69ac754e46f7ae8ff33661b2 INFO: 172.21.0.1:34544 - "POST /node/69ac754e46f7ae8ff33661b2 HTTP/1.0" 202 Accepted 2026-03-09 18:58:43,266 - lava_callback - INFO - Starting async job submit for node 69ac754e46f7ae8ff33661b2 2026-03-09 18:58:43,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:43,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac754e46f7ae8ff33661b2 HTTP/1.1" 200 None 2026-03-09 18:58:43,282 - lava_callback - DEBUG - Processing node 69ac754e46f7ae8ff33661b2: baseline-arm-clabbe 2026-03-09 18:58:43,282 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac754e46f7ae8ff33661b2 2026-03-09 18:58:43,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:43,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac754e46f7ae8ff33661b2 HTTP/1.1" 200 None 2026-03-09 18:58:43,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:43,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e34efa461f55e59cdb7 HTTP/1.1" 200 None 2026-03-09 18:58:43,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:43,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac754e46f7ae8ff33661b2 HTTP/1.1" 200 None 2026-03-09 18:58:43,348 - lava_callback - ERROR - Error processing callback for node 69ac754e46f7ae8ff33661b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 18:58:46,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:58:46,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45006 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:59:25,157 - lava_callback - INFO - Received callback for node 69af1839ce46d8ec186accde INFO: 172.21.0.1:56436 - "POST /node/69af1839ce46d8ec186accde HTTP/1.0" 202 Accepted 2026-03-09 18:59:25,163 - lava_callback - INFO - Starting async job submit for node 69af1839ce46d8ec186accde 2026-03-09 18:59:25,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:59:25,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1839ce46d8ec186accde HTTP/1.1" 200 None 2026-03-09 18:59:25,210 - lava_callback - DEBUG - Processing node 69af1839ce46d8ec186accde: baseline-arm64-foundriesio 2026-03-09 18:59:25,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:59:25,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:59:25,575 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af1839ce46d8ec186accde/log.txt.gz 2026-03-09 18:59:25,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:59:25,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:59:25,923 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af1839ce46d8ec186accde/lava-logs.yaml 2026-03-09 18:59:25,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 18:59:26,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 18:59:26,124 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af1839ce46d8ec186accde/lava_callback.json.gz DEBUG: Setup failed for 69af1839ce46d8ec186accde. Transitting job node result: incomplete -> fail 2026-03-09 18:59:26,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:59:26,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1839ce46d8ec186accde HTTP/1.1" 200 None 2026-03-09 18:59:26,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:59:26,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 18:59:26,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:59:26,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1839ce46d8ec186accde HTTP/1.1" 200 None 2026-03-09 18:59:26,321 - lava_callback - INFO - Completed processing callback for node 69af1839ce46d8ec186accde INFO: 172.21.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56072 - "GET / HTTP/1.0" 200 OK 2026-03-09 18:59:46,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 18:59:46,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53828 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:01:01,526 - lava_callback - INFO - Received callback for node 69af1877ce46d8ec186accef INFO: 172.21.0.1:55830 - "POST /node/69af1877ce46d8ec186accef HTTP/1.0" 202 Accepted 2026-03-09 19:01:01,532 - lava_callback - INFO - Starting async job submit for node 69af1877ce46d8ec186accef 2026-03-09 19:01:01,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:01,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1877ce46d8ec186accef HTTP/1.1" 200 None 2026-03-09 19:01:01,587 - lava_callback - DEBUG - Processing node 69af1877ce46d8ec186accef: baseline-nfs-arm64 2026-03-09 19:01:01,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:01:02,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:01:02,053 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1877ce46d8ec186accef/log.txt.gz 2026-03-09 19:01:02,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:01:02,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:01:02,545 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1877ce46d8ec186accef/lava-logs.yaml 2026-03-09 19:01:02,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:01:02,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:01:02,699 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1877ce46d8ec186accef/lava_callback.json.gz DEBUG: Setup failed for 69af1877ce46d8ec186accef. Transitting job node result: incomplete -> fail 2026-03-09 19:01:02,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:02,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1877ce46d8ec186accef HTTP/1.1" 200 None 2026-03-09 19:01:02,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:02,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:01:02,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:02,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1877ce46d8ec186accef HTTP/1.1" 200 None 2026-03-09 19:01:02,904 - lava_callback - INFO - Completed processing callback for node 69af1877ce46d8ec186accef INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52242 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:01:16,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:16,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50372 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:01:59,024 - lava_callback - INFO - Received callback for node 69af18c3ce46d8ec186acd01 INFO: 172.21.0.1:50380 - "POST /node/69af18c3ce46d8ec186acd01 HTTP/1.0" 202 Accepted 2026-03-09 19:01:59,031 - lava_callback - INFO - Starting async job submit for node 69af18c3ce46d8ec186acd01 2026-03-09 19:01:59,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:01:59,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af18c3ce46d8ec186acd01 HTTP/1.1" 200 None 2026-03-09 19:01:59,079 - lava_callback - DEBUG - Processing node 69af18c3ce46d8ec186acd01: baseline-nfs-arm64 2026-03-09 19:01:59,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:02:00,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:02:00,085 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af18c3ce46d8ec186acd01/log.txt.gz 2026-03-09 19:02:00,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:02:00,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:02:00,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af18c3ce46d8ec186acd01/lava-logs.yaml 2026-03-09 19:02:00,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:02:00,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:02:00,951 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af18c3ce46d8ec186acd01/lava_callback.json.gz DEBUG: Setup failed for 69af18c3ce46d8ec186acd01. Transitting job node result: incomplete -> fail 2026-03-09 19:02:01,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:02:01,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af18c3ce46d8ec186acd01 HTTP/1.1" 200 None 2026-03-09 19:02:01,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:02:01,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:02:01,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:02:01,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af18c3ce46d8ec186acd01 HTTP/1.1" 200 None 2026-03-09 19:02:01,155 - lava_callback - INFO - Completed processing callback for node 69af18c3ce46d8ec186acd01 INFO: 172.21.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33776 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:02:16,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:02:16,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40554 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:03:33,479 - lava_callback - INFO - Received callback for node 69af1912ce46d8ec186acd14 INFO: 172.21.0.1:52594 - "POST /node/69af1912ce46d8ec186acd14 HTTP/1.0" 202 Accepted 2026-03-09 19:03:33,486 - lava_callback - INFO - Starting async job submit for node 69af1912ce46d8ec186acd14 2026-03-09 19:03:33,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:03:33,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1912ce46d8ec186acd14 HTTP/1.1" 200 None 2026-03-09 19:03:33,533 - lava_callback - DEBUG - Processing node 69af1912ce46d8ec186acd14: baseline-nfs-arm64 2026-03-09 19:03:34,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:03:34,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:03:34,238 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1912ce46d8ec186acd14/log.txt.gz 2026-03-09 19:03:34,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:03:34,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:03:34,750 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1912ce46d8ec186acd14/lava-logs.yaml 2026-03-09 19:03:34,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:03:34,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:03:34,934 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1912ce46d8ec186acd14/lava_callback.json.gz DEBUG: Setup failed for 69af1912ce46d8ec186acd14. Transitting job node result: incomplete -> fail 2026-03-09 19:03:35,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:03:35,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1912ce46d8ec186acd14 HTTP/1.1" 200 None 2026-03-09 19:03:35,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:03:35,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:03:35,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:03:35,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1912ce46d8ec186acd14 HTTP/1.1" 200 None 2026-03-09 19:03:35,124 - lava_callback - INFO - Completed processing callback for node 69af1912ce46d8ec186acd14 INFO: 172.21.0.1:52600 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:03:46,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:03:46,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46280 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:05:02,096 - lava_callback - INFO - Received callback for node 69af1964ce46d8ec186acd23 INFO: 172.21.0.1:48848 - "POST /node/69af1964ce46d8ec186acd23 HTTP/1.0" 202 Accepted 2026-03-09 19:05:02,102 - lava_callback - INFO - Starting async job submit for node 69af1964ce46d8ec186acd23 2026-03-09 19:05:02,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:02,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1964ce46d8ec186acd23 HTTP/1.1" 200 None 2026-03-09 19:05:02,169 - lava_callback - DEBUG - Processing node 69af1964ce46d8ec186acd23: baseline-nfs-arm64 2026-03-09 19:05:02,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:02,710 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:02,712 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1964ce46d8ec186acd23/log.txt.gz 2026-03-09 19:05:02,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:03,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:03,196 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1964ce46d8ec186acd23/lava-logs.yaml 2026-03-09 19:05:03,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:03,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:03,358 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1964ce46d8ec186acd23/lava_callback.json.gz DEBUG: Setup failed for 69af1964ce46d8ec186acd23. Transitting job node result: incomplete -> fail 2026-03-09 19:05:03,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:03,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1964ce46d8ec186acd23 HTTP/1.1" 200 None 2026-03-09 19:05:03,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:03,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:05:03,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:03,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1964ce46d8ec186acd23 HTTP/1.1" 200 None 2026-03-09 19:05:03,558 - lava_callback - INFO - Completed processing callback for node 69af1964ce46d8ec186acd23 INFO: 172.21.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42898 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:05:16,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:16,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43068 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:05:57,117 - lava_callback - INFO - Received callback for node 69af19b3ce46d8ec186acd35 INFO: 172.21.0.1:43080 - "POST /node/69af19b3ce46d8ec186acd35 HTTP/1.0" 202 Accepted 2026-03-09 19:05:57,124 - lava_callback - INFO - Starting async job submit for node 69af19b3ce46d8ec186acd35 2026-03-09 19:05:57,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:57,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af19b3ce46d8ec186acd35 HTTP/1.1" 200 None 2026-03-09 19:05:57,171 - lava_callback - DEBUG - Processing node 69af19b3ce46d8ec186acd35: baseline-nfs-arm64 2026-03-09 19:05:57,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:57,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:57,725 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af19b3ce46d8ec186acd35/log.txt.gz INFO: 172.21.0.1:43096 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:05:57,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:58,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:58,214 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af19b3ce46d8ec186acd35/lava-logs.yaml 2026-03-09 19:05:58,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:05:58,469 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:05:58,471 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af19b3ce46d8ec186acd35/lava_callback.json.gz DEBUG: Setup failed for 69af19b3ce46d8ec186acd35. Transitting job node result: incomplete -> fail 2026-03-09 19:05:58,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:58,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af19b3ce46d8ec186acd35 HTTP/1.1" 200 None 2026-03-09 19:05:58,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:58,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:05:58,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:05:58,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af19b3ce46d8ec186acd35 HTTP/1.1" 200 None 2026-03-09 19:05:58,674 - lava_callback - INFO - Completed processing callback for node 69af19b3ce46d8ec186acd35 INFO: 172.21.0.1:52036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34840 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:06:16,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:06:16,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60664 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:08:35,381 - lava_callback - INFO - Received callback for node 69af19f7ce46d8ec186acd45 INFO: 172.21.0.1:48554 - "POST /node/69af19f7ce46d8ec186acd45 HTTP/1.0" 202 Accepted 2026-03-09 19:08:35,387 - lava_callback - INFO - Starting async job submit for node 69af19f7ce46d8ec186acd45 2026-03-09 19:08:35,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:08:35,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af19f7ce46d8ec186acd45 HTTP/1.1" 200 None 2026-03-09 19:08:35,433 - lava_callback - DEBUG - Processing node 69af19f7ce46d8ec186acd45: baseline-arm64 2026-03-09 19:08:35,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:08:35,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:08:35,950 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af19f7ce46d8ec186acd45/log.txt.gz 2026-03-09 19:08:36,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:08:36,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:08:36,412 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af19f7ce46d8ec186acd45/lava-logs.yaml 2026-03-09 19:08:36,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:48570 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:08:36,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:08:36,588 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af19f7ce46d8ec186acd45/lava_callback.json.gz DEBUG: Setup failed for 69af19f7ce46d8ec186acd45. Transitting job node result: incomplete -> fail 2026-03-09 19:08:36,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:08:36,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af19f7ce46d8ec186acd45 HTTP/1.1" 200 None 2026-03-09 19:08:36,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:08:36,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:08:36,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:08:36,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af19f7ce46d8ec186acd45 HTTP/1.1" 200 None 2026-03-09 19:08:36,787 - lava_callback - INFO - Completed processing callback for node 69af19f7ce46d8ec186acd45 2026-03-09 19:08:46,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:08:46,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41416 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:09:05,144 - lava_callback - INFO - Received callback for node 69af1640ce46d8ec186acc72 INFO: 172.21.0.1:59796 - "POST /node/69af1640ce46d8ec186acc72 HTTP/1.0" 202 Accepted 2026-03-09 19:09:05,193 - lava_callback - INFO - Starting async job submit for node 69af1640ce46d8ec186acc72 2026-03-09 19:09:05,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:09:05,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1640ce46d8ec186acc72 HTTP/1.1" 200 None 2026-03-09 19:09:05,232 - lava_callback - DEBUG - Processing node 69af1640ce46d8ec186acc72: baseline-arm-pengutronix 2026-03-09 19:09:05,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:09:05,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:09:05,607 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af1640ce46d8ec186acc72/log.txt.gz 2026-03-09 19:09:05,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:09:05,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:09:05,852 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af1640ce46d8ec186acc72/lava-logs.yaml 2026-03-09 19:09:05,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:09:05,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:09:05,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af1640ce46d8ec186acc72/lava_callback.json.gz DEBUG: Setup failed for 69af1640ce46d8ec186acc72. Transitting job node result: incomplete -> fail 2026-03-09 19:09:06,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:09:06,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1640ce46d8ec186acc72 HTTP/1.1" 200 None 2026-03-09 19:09:06,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:09:06,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 19:09:06,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:09:06,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1640ce46d8ec186acc72 HTTP/1.1" 200 None 2026-03-09 19:09:06,151 - lava_callback - INFO - Completed processing callback for node 69af1640ce46d8ec186acc72 INFO: 172.21.0.1:59802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60808 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:09:16,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:09:16,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58074 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:10:38,839 - lava_callback - INFO - Received callback for node 69af1ad7ce46d8ec186acd73 INFO: 172.21.0.1:58084 - "POST /node/69af1ad7ce46d8ec186acd73 HTTP/1.0" 202 Accepted 2026-03-09 19:10:38,845 - lava_callback - INFO - Starting async job submit for node 69af1ad7ce46d8ec186acd73 2026-03-09 19:10:38,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:38,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1ad7ce46d8ec186acd73 HTTP/1.1" 200 None 2026-03-09 19:10:38,891 - lava_callback - DEBUG - Processing node 69af1ad7ce46d8ec186acd73: baseline-arm64 2026-03-09 19:10:39,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:10:39,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:10:39,431 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1ad7ce46d8ec186acd73/log.txt.gz 2026-03-09 19:10:39,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:10:39,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:10:39,913 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1ad7ce46d8ec186acd73/lava-logs.yaml 2026-03-09 19:10:39,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:10:40,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:10:40,049 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1ad7ce46d8ec186acd73/lava_callback.json.gz DEBUG: Setup failed for 69af1ad7ce46d8ec186acd73. Transitting job node result: incomplete -> fail 2026-03-09 19:10:40,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:40,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1ad7ce46d8ec186acd73 HTTP/1.1" 200 None 2026-03-09 19:10:40,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:40,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:10:40,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:40,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1ad7ce46d8ec186acd73 HTTP/1.1" 200 None 2026-03-09 19:10:40,258 - lava_callback - INFO - Completed processing callback for node 69af1ad7ce46d8ec186acd73 2026-03-09 19:10:46,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:46,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52144 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:10:51,961 - lava_callback - INFO - Received callback for node 69ac783846f7ae8ff336624e INFO: 172.21.0.1:55078 - "POST /node/69ac783846f7ae8ff336624e HTTP/1.0" 202 Accepted 2026-03-09 19:10:51,967 - lava_callback - INFO - Starting async job submit for node 69ac783846f7ae8ff336624e 2026-03-09 19:10:51,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:51,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac783846f7ae8ff336624e HTTP/1.1" 200 None 2026-03-09 19:10:51,983 - lava_callback - DEBUG - Processing node 69ac783846f7ae8ff336624e: baseline-arm64-baylibre 2026-03-09 19:10:51,983 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac783846f7ae8ff336624e 2026-03-09 19:10:51,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:51,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac783846f7ae8ff336624e HTTP/1.1" 200 None 2026-03-09 19:10:51,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:52,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:10:52,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:10:52,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac783846f7ae8ff336624e HTTP/1.1" 200 None 2026-03-09 19:10:52,041 - lava_callback - ERROR - Error processing callback for node 69ac783846f7ae8ff336624e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42678 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:11:29,062 - lava_callback - INFO - Received callback for node 69af1a89ce46d8ec186acd5a INFO: 172.21.0.1:42692 - "POST /node/69af1a89ce46d8ec186acd5a HTTP/1.0" 202 Accepted 2026-03-09 19:11:29,068 - lava_callback - INFO - Starting async job submit for node 69af1a89ce46d8ec186acd5a 2026-03-09 19:11:29,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:29,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1a89ce46d8ec186acd5a HTTP/1.1" 200 None 2026-03-09 19:11:29,119 - lava_callback - DEBUG - Processing node 69af1a89ce46d8ec186acd5a: baseline-nfs-arm64 2026-03-09 19:11:29,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:11:29,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:11:29,680 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1a89ce46d8ec186acd5a/log.txt.gz 2026-03-09 19:11:29,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:11:30,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:11:30,102 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1a89ce46d8ec186acd5a/lava-logs.yaml 2026-03-09 19:11:30,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:11:30,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:11:30,248 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af1a89ce46d8ec186acd5a/lava_callback.json.gz DEBUG: Setup failed for 69af1a89ce46d8ec186acd5a. Transitting job node result: incomplete -> fail 2026-03-09 19:11:30,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:30,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1a89ce46d8ec186acd5a HTTP/1.1" 200 None 2026-03-09 19:11:30,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:30,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:11:30,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:30,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1a89ce46d8ec186acd5a HTTP/1.1" 200 None 2026-03-09 19:11:30,504 - lava_callback - INFO - Completed processing callback for node 69af1a89ce46d8ec186acd5a INFO: 172.21.0.1:41160 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:11:46,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:46,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35420 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:11:57,474 - lava_callback - INFO - Received callback for node 69af1b6dce46d8ec186acd91 INFO: 172.21.0.1:35424 - "POST /node/69af1b6dce46d8ec186acd91 HTTP/1.0" 202 Accepted 2026-03-09 19:11:57,480 - lava_callback - INFO - Starting async job submit for node 69af1b6dce46d8ec186acd91 2026-03-09 19:11:57,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:57,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1b6dce46d8ec186acd91 HTTP/1.1" 200 None 2026-03-09 19:11:57,497 - lava_callback - DEBUG - Processing node 69af1b6dce46d8ec186acd91: baseline-arm64 2026-03-09 19:11:57,501 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:11:57,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:11:57,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:11:57,632 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1b6dce46d8ec186acd91/lava-logs.yaml 2026-03-09 19:11:57,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:11:57,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:11:57,765 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1b6dce46d8ec186acd91/lava_callback.json.gz 2026-03-09 19:11:57,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:57,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1b6dce46d8ec186acd91 HTTP/1.1" 200 None 2026-03-09 19:11:57,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:57,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:11:57,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:11:57,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1b6dce46d8ec186acd91 HTTP/1.1" 200 None 2026-03-09 19:11:57,844 - lava_callback - INFO - Completed processing callback for node 69af1b6dce46d8ec186acd91 INFO: 172.21.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53068 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:12:11,981 - lava_callback - INFO - Received callback for node 69ac788746f7ae8ff3366271 INFO: 172.21.0.1:56678 - "POST /node/69ac788746f7ae8ff3366271 HTTP/1.0" 202 Accepted 2026-03-09 19:12:11,987 - lava_callback - INFO - Starting async job submit for node 69ac788746f7ae8ff3366271 2026-03-09 19:12:11,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:12:12,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac788746f7ae8ff3366271 HTTP/1.1" 200 None 2026-03-09 19:12:12,009 - lava_callback - DEBUG - Processing node 69ac788746f7ae8ff3366271: baseline-arm64-baylibre 2026-03-09 19:12:12,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac788746f7ae8ff3366271 2026-03-09 19:12:12,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:12:12,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac788746f7ae8ff3366271 HTTP/1.1" 200 None 2026-03-09 19:12:12,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:12:12,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:12:12,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:12:12,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac788746f7ae8ff3366271 HTTP/1.1" 200 None 2026-03-09 19:12:12,062 - lava_callback - ERROR - Error processing callback for node 69ac788746f7ae8ff3366271: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56690 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:12:16,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:12:16,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57088 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:13:32,090 - lava_callback - INFO - Received callback for node 69ac78d246f7ae8ff336628a INFO: 172.21.0.1:37932 - "POST /node/69ac78d246f7ae8ff336628a HTTP/1.0" 202 Accepted 2026-03-09 19:13:32,096 - lava_callback - INFO - Starting async job submit for node 69ac78d246f7ae8ff336628a 2026-03-09 19:13:32,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:13:32,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac78d246f7ae8ff336628a HTTP/1.1" 200 None 2026-03-09 19:13:32,112 - lava_callback - DEBUG - Processing node 69ac78d246f7ae8ff336628a: baseline-arm64-baylibre 2026-03-09 19:13:32,113 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac78d246f7ae8ff336628a 2026-03-09 19:13:32,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:13:32,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac78d246f7ae8ff336628a HTTP/1.1" 200 None 2026-03-09 19:13:32,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:13:32,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:13:32,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:13:32,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac78d246f7ae8ff336628a HTTP/1.1" 200 None 2026-03-09 19:13:32,171 - lava_callback - ERROR - Error processing callback for node 69ac78d246f7ae8ff336628a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:14:37,902 - lava_callback - INFO - Received callback for node 69af1c07ce46d8ec186acdab INFO: 172.21.0.1:40514 - "POST /node/69af1c07ce46d8ec186acdab HTTP/1.0" 202 Accepted 2026-03-09 19:14:37,908 - lava_callback - INFO - Starting async job submit for node 69af1c07ce46d8ec186acdab 2026-03-09 19:14:37,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:37,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1c07ce46d8ec186acdab HTTP/1.1" 200 None 2026-03-09 19:14:37,926 - lava_callback - DEBUG - Processing node 69af1c07ce46d8ec186acdab: baseline-arm64 2026-03-09 19:14:37,931 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:14:37,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:14:38,083 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:14:38,085 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1c07ce46d8ec186acdab/lava-logs.yaml 2026-03-09 19:14:38,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:14:38,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:14:38,228 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1c07ce46d8ec186acdab/lava_callback.json.gz 2026-03-09 19:14:38,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:38,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1c07ce46d8ec186acdab HTTP/1.1" 200 None 2026-03-09 19:14:38,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:38,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:14:38,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:38,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1c07ce46d8ec186acdab HTTP/1.1" 200 None 2026-03-09 19:14:38,299 - lava_callback - INFO - Completed processing callback for node 69af1c07ce46d8ec186acdab 2026-03-09 19:14:46,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:46,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38144 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:14:52,145 - lava_callback - INFO - Received callback for node 69ac791c46f7ae8ff33662b2 INFO: 172.21.0.1:45296 - "POST /node/69ac791c46f7ae8ff33662b2 HTTP/1.0" 202 Accepted 2026-03-09 19:14:52,151 - lava_callback - INFO - Starting async job submit for node 69ac791c46f7ae8ff33662b2 2026-03-09 19:14:52,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:52,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac791c46f7ae8ff33662b2 HTTP/1.1" 200 None 2026-03-09 19:14:52,167 - lava_callback - DEBUG - Processing node 69ac791c46f7ae8ff33662b2: baseline-arm64-baylibre 2026-03-09 19:14:52,167 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac791c46f7ae8ff33662b2 2026-03-09 19:14:52,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:52,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac791c46f7ae8ff33662b2 HTTP/1.1" 200 None 2026-03-09 19:14:52,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:52,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:14:52,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:14:52,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac791c46f7ae8ff33662b2 HTTP/1.1" 200 None 2026-03-09 19:14:52,238 - lava_callback - ERROR - Error processing callback for node 69ac791c46f7ae8ff33662b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39242 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:15:36,114 - lava_callback - INFO - Received callback for node 69af1b34ce46d8ec186acd85 INFO: 172.21.0.1:47192 - "POST /node/69af1b34ce46d8ec186acd85 HTTP/1.0" 202 Accepted 2026-03-09 19:15:36,121 - lava_callback - INFO - Starting async job submit for node 69af1b34ce46d8ec186acd85 2026-03-09 19:15:36,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:36,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1b34ce46d8ec186acd85 HTTP/1.1" 200 None 2026-03-09 19:15:36,175 - lava_callback - DEBUG - Processing node 69af1b34ce46d8ec186acd85: baseline-arm64-broonie INFO: 172.21.0.1:47194 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:15:36,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:36,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:36,804 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1b34ce46d8ec186acd85/log.txt.gz 2026-03-09 19:15:37,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:37,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:37,247 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1b34ce46d8ec186acd85/lava-logs.yaml 2026-03-09 19:15:37,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:37,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:37,450 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1b34ce46d8ec186acd85/lava_callback.json.gz DEBUG: Setup failed for 69af1b34ce46d8ec186acd85. Transitting job node result: incomplete -> fail 2026-03-09 19:15:37,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:37,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1b34ce46d8ec186acd85 HTTP/1.1" 200 None 2026-03-09 19:15:37,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:37,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-09 19:15:37,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:37,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1b34ce46d8ec186acd85 HTTP/1.1" 200 None 2026-03-09 19:15:37,679 - lava_callback - INFO - Completed processing callback for node 69af1b34ce46d8ec186acd85 2026-03-09 19:15:45,104 - lava_callback - INFO - Received callback for node 69ae6fb08ef4424e487b5ebc INFO: 172.21.0.1:38936 - "POST /node/69ae6fb08ef4424e487b5ebc HTTP/1.0" 202 Accepted 2026-03-09 19:15:45,109 - lava_callback - INFO - Starting async job submit for node 69ae6fb08ef4424e487b5ebc 2026-03-09 19:15:45,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:45,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6fb08ef4424e487b5ebc HTTP/1.1" 200 None 2026-03-09 19:15:45,142 - lava_callback - DEBUG - Processing node 69ae6fb08ef4424e487b5ebc: baseline-x86 2026-03-09 19:15:45,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:45,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:45,629 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ae6fb08ef4424e487b5ebc/log.txt.gz 2026-03-09 19:15:45,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:46,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:46,039 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ae6fb08ef4424e487b5ebc/lava-logs.yaml 2026-03-09 19:15:46,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:46,194 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:46,196 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ae6fb08ef4424e487b5ebc/lava_callback.json.gz 2026-03-09 19:15:46,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:46,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae6fb08ef4424e487b5ebc HTTP/1.1" 200 None 2026-03-09 19:15:46,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:46,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 19:15:46,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:46,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae6fb08ef4424e487b5ebc HTTP/1.1" 200 None 2026-03-09 19:15:46,292 - lava_callback - INFO - Completed processing callback for node 69ae6fb08ef4424e487b5ebc 2026-03-09 19:15:46,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:38950 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:15:46,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:15:46,885 - lava_callback - INFO - Received callback for node 69ae1a5b8ef4424e487b3966 INFO: 172.21.0.1:38952 - "POST /node/69ae1a5b8ef4424e487b3966 HTTP/1.0" 202 Accepted 2026-03-09 19:15:46,891 - lava_callback - INFO - Starting async job submit for node 69ae1a5b8ef4424e487b3966 2026-03-09 19:15:46,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:46,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a5b8ef4424e487b3966 HTTP/1.1" 200 None 2026-03-09 19:15:46,926 - lava_callback - DEBUG - Processing node 69ae1a5b8ef4424e487b3966: baseline-x86 2026-03-09 19:15:47,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:47,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:47,379 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a5b8ef4424e487b3966/log.txt.gz 2026-03-09 19:15:47,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:47,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:47,755 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a5b8ef4424e487b3966/lava-logs.yaml 2026-03-09 19:15:47,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:15:47,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:15:47,920 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ae1a5b8ef4424e487b3966/lava_callback.json.gz 2026-03-09 19:15:47,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:47,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae1a5b8ef4424e487b3966 HTTP/1.1" 200 None 2026-03-09 19:15:47,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:47,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 19:15:47,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:47,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae1a5b8ef4424e487b3966 HTTP/1.1" 200 None 2026-03-09 19:15:47,994 - lava_callback - INFO - Completed processing callback for node 69ae1a5b8ef4424e487b3966 2026-03-09 19:15:52,157 - lava_callback - INFO - Received callback for node 69ac796546f7ae8ff33662bf INFO: 172.21.0.1:55440 - "POST /node/69ac796546f7ae8ff33662bf HTTP/1.0" 202 Accepted 2026-03-09 19:15:52,163 - lava_callback - INFO - Starting async job submit for node 69ac796546f7ae8ff33662bf 2026-03-09 19:15:52,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:52,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac796546f7ae8ff33662bf HTTP/1.1" 200 None 2026-03-09 19:15:52,180 - lava_callback - DEBUG - Processing node 69ac796546f7ae8ff33662bf: baseline-arm64-baylibre 2026-03-09 19:15:52,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac796546f7ae8ff33662bf 2026-03-09 19:15:52,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:52,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac796546f7ae8ff33662bf HTTP/1.1" 200 None 2026-03-09 19:15:52,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:52,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:15:52,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:15:52,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac796546f7ae8ff33662bf HTTP/1.1" 200 None 2026-03-09 19:15:52,244 - lava_callback - ERROR - Error processing callback for node 69ac796546f7ae8ff33662bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47976 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 19:16:16,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:47982 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:16:16,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34218 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:17:12,287 - lava_callback - INFO - Received callback for node 69ac79b046f7ae8ff33662e0 INFO: 172.21.0.1:59504 - "POST /node/69ac79b046f7ae8ff33662e0 HTTP/1.0" 202 Accepted 2026-03-09 19:17:12,292 - lava_callback - INFO - Starting async job submit for node 69ac79b046f7ae8ff33662e0 2026-03-09 19:17:12,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:12,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac79b046f7ae8ff33662e0 HTTP/1.1" 200 None 2026-03-09 19:17:12,309 - lava_callback - DEBUG - Processing node 69ac79b046f7ae8ff33662e0: baseline-arm64-baylibre 2026-03-09 19:17:12,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac79b046f7ae8ff33662e0 2026-03-09 19:17:12,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:12,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac79b046f7ae8ff33662e0 HTTP/1.1" 200 None 2026-03-09 19:17:12,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:12,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:17:12,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:12,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac79b046f7ae8ff33662e0 HTTP/1.1" 200 None 2026-03-09 19:17:12,365 - lava_callback - ERROR - Error processing callback for node 69ac79b046f7ae8ff33662e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56976 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:17:26,664 - lava_callback - INFO - Received callback for node 69ae875fc660ead8a389adf6 INFO: 172.21.0.1:56978 - "POST /node/69ae875fc660ead8a389adf6 HTTP/1.0" 202 Accepted 2026-03-09 19:17:26,670 - lava_callback - INFO - Starting async job submit for node 69ae875fc660ead8a389adf6 2026-03-09 19:17:26,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:26,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae875fc660ead8a389adf6 HTTP/1.1" 200 None 2026-03-09 19:17:26,704 - lava_callback - DEBUG - Processing node 69ae875fc660ead8a389adf6: baseline-x86 2026-03-09 19:17:27,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:17:27,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:17:27,159 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69ae875fc660ead8a389adf6/log.txt.gz 2026-03-09 19:17:27,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:17:27,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:17:27,559 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69ae875fc660ead8a389adf6/lava-logs.yaml 2026-03-09 19:17:27,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:17:27,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:17:27,714 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69ae875fc660ead8a389adf6/lava_callback.json.gz 2026-03-09 19:17:27,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:27,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae875fc660ead8a389adf6 HTTP/1.1" 200 None 2026-03-09 19:17:27,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:27,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8240c660ead8a389ab2d HTTP/1.1" 200 None 2026-03-09 19:17:27,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:27,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae875fc660ead8a389adf6 HTTP/1.1" 200 None 2026-03-09 19:17:27,800 - lava_callback - INFO - Completed processing callback for node 69ae875fc660ead8a389adf6 INFO: 172.21.0.1:60408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60908 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:17:46,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:17:46,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:18:08,923 - lava_callback - INFO - Received callback for node 69af1bcace46d8ec186acda1 INFO: 172.21.0.1:52314 - "POST /node/69af1bcace46d8ec186acda1 HTTP/1.0" 202 Accepted 2026-03-09 19:18:08,929 - lava_callback - INFO - Starting async job submit for node 69af1bcace46d8ec186acda1 2026-03-09 19:18:08,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:08,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1bcace46d8ec186acda1 HTTP/1.1" 200 None 2026-03-09 19:18:09,000 - lava_callback - DEBUG - Processing node 69af1bcace46d8ec186acda1: baseline-arm64-broonie 2026-03-09 19:18:09,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:18:09,391 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:18:09,393 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1bcace46d8ec186acda1/log.txt.gz 2026-03-09 19:18:09,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:18:09,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:18:09,713 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1bcace46d8ec186acda1/lava-logs.yaml 2026-03-09 19:18:09,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:18:09,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:18:09,845 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1bcace46d8ec186acda1/lava_callback.json.gz DEBUG: Setup failed for 69af1bcace46d8ec186acda1. Transitting job node result: incomplete -> fail 2026-03-09 19:18:09,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:09,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1bcace46d8ec186acda1 HTTP/1.1" 200 None 2026-03-09 19:18:09,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:09,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:18:09,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:09,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1bcace46d8ec186acda1 HTTP/1.1" 200 None 2026-03-09 19:18:10,058 - lava_callback - INFO - Completed processing callback for node 69af1bcace46d8ec186acda1 INFO: 172.21.0.1:35704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35712 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:18:16,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:16,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44244 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:18:32,355 - lava_callback - INFO - Received callback for node 69ac79fa46f7ae8ff33662f7 INFO: 172.21.0.1:55188 - "POST /node/69ac79fa46f7ae8ff33662f7 HTTP/1.0" 202 Accepted 2026-03-09 19:18:32,360 - lava_callback - INFO - Starting async job submit for node 69ac79fa46f7ae8ff33662f7 2026-03-09 19:18:32,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:32,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac79fa46f7ae8ff33662f7 HTTP/1.1" 200 None 2026-03-09 19:18:32,376 - lava_callback - DEBUG - Processing node 69ac79fa46f7ae8ff33662f7: baseline-arm64-baylibre 2026-03-09 19:18:32,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac79fa46f7ae8ff33662f7 2026-03-09 19:18:32,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:32,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac79fa46f7ae8ff33662f7 HTTP/1.1" 200 None 2026-03-09 19:18:32,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:32,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:18:32,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:18:32,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac79fa46f7ae8ff33662f7 HTTP/1.1" 200 None 2026-03-09 19:18:32,439 - lava_callback - ERROR - Error processing callback for node 69ac79fa46f7ae8ff33662f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50784 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:19:52,491 - lava_callback - INFO - Received callback for node 69ac7a4346f7ae8ff3366318 INFO: 172.21.0.1:54300 - "POST /node/69ac7a4346f7ae8ff3366318 HTTP/1.0" 202 Accepted 2026-03-09 19:19:52,496 - lava_callback - INFO - Starting async job submit for node 69ac7a4346f7ae8ff3366318 2026-03-09 19:19:52,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:19:52,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7a4346f7ae8ff3366318 HTTP/1.1" 200 None 2026-03-09 19:19:52,512 - lava_callback - DEBUG - Processing node 69ac7a4346f7ae8ff3366318: baseline-arm64-baylibre 2026-03-09 19:19:52,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7a4346f7ae8ff3366318 2026-03-09 19:19:52,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:19:52,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7a4346f7ae8ff3366318 HTTP/1.1" 200 None 2026-03-09 19:19:52,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:19:52,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:19:52,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:19:52,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7a4346f7ae8ff3366318 HTTP/1.1" 200 None 2026-03-09 19:19:52,569 - lava_callback - ERROR - Error processing callback for node 69ac7a4346f7ae8ff3366318: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46276 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:20:37,501 - lava_callback - INFO - Received callback for node 69aec064feb96b28aa34ab95 INFO: 172.21.0.1:46288 - "POST /node/69aec064feb96b28aa34ab95 HTTP/1.0" 202 Accepted 2026-03-09 19:20:37,507 - lava_callback - INFO - Starting async job submit for node 69aec064feb96b28aa34ab95 2026-03-09 19:20:37,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:37,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec064feb96b28aa34ab95 HTTP/1.1" 200 None 2026-03-09 19:20:37,550 - lava_callback - DEBUG - Processing node 69aec064feb96b28aa34ab95: baseline-x86 2026-03-09 19:20:37,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:20:38,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:38,039 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aec064feb96b28aa34ab95/log.txt.gz 2026-03-09 19:20:38,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:20:38,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:38,437 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aec064feb96b28aa34ab95/lava-logs.yaml 2026-03-09 19:20:38,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:20:38,602 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:38,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aec064feb96b28aa34ab95/lava_callback.json.gz 2026-03-09 19:20:38,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:38,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec064feb96b28aa34ab95 HTTP/1.1" 200 None 2026-03-09 19:20:38,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:38,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba54feb96b28aa34a79d HTTP/1.1" 200 None 2026-03-09 19:20:38,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:38,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aec064feb96b28aa34ab95 HTTP/1.1" 200 None 2026-03-09 19:20:38,677 - lava_callback - INFO - Completed processing callback for node 69aec064feb96b28aa34ab95 INFO: 172.21.0.1:40230 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:20:46,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:46,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:20:52,462 - lava_callback - INFO - Received callback for node 69ac7a8e46f7ae8ff3366326 INFO: 172.21.0.1:45514 - "POST /node/69ac7a8e46f7ae8ff3366326 HTTP/1.0" 202 Accepted 2026-03-09 19:20:52,467 - lava_callback - INFO - Starting async job submit for node 69ac7a8e46f7ae8ff3366326 2026-03-09 19:20:52,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:52,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7a8e46f7ae8ff3366326 HTTP/1.1" 200 None 2026-03-09 19:20:52,488 - lava_callback - DEBUG - Processing node 69ac7a8e46f7ae8ff3366326: baseline-arm64-baylibre 2026-03-09 19:20:52,488 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7a8e46f7ae8ff3366326 2026-03-09 19:20:52,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:52,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7a8e46f7ae8ff3366326 HTTP/1.1" 200 None 2026-03-09 19:20:52,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:52,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:20:52,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:52,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7a8e46f7ae8ff3366326 HTTP/1.1" 200 None 2026-03-09 19:20:52,545 - lava_callback - ERROR - Error processing callback for node 69ac7a8e46f7ae8ff3366326: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 19:20:55,704 - lava_callback - INFO - Received callback for node 69aec870feb96b28aa34b61d INFO: 172.21.0.1:45526 - "POST /node/69aec870feb96b28aa34b61d HTTP/1.0" 202 Accepted 2026-03-09 19:20:55,710 - lava_callback - INFO - Starting async job submit for node 69aec870feb96b28aa34b61d 2026-03-09 19:20:55,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:55,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec870feb96b28aa34b61d HTTP/1.1" 200 None 2026-03-09 19:20:55,744 - lava_callback - DEBUG - Processing node 69aec870feb96b28aa34b61d: baseline-x86 2026-03-09 19:20:56,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:20:56,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:56,214 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aec870feb96b28aa34b61d/log.txt.gz 2026-03-09 19:20:56,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:45540 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:20:56,585 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:56,587 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aec870feb96b28aa34b61d/lava-logs.yaml 2026-03-09 19:20:56,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:20:56,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:20:56,758 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aec870feb96b28aa34b61d/lava_callback.json.gz 2026-03-09 19:20:56,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:56,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aec870feb96b28aa34b61d HTTP/1.1" 200 None 2026-03-09 19:20:56,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:56,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 19:20:56,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:20:56,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aec870feb96b28aa34b61d HTTP/1.1" 200 None 2026-03-09 19:20:56,868 - lava_callback - INFO - Completed processing callback for node 69aec870feb96b28aa34b61d INFO: 172.21.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45940 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:21:16,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:16,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41932 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:21:37,802 - lava_callback - INFO - Received callback for node 69aef10efeb96b28aa34c4b5 INFO: 172.21.0.1:41944 - "POST /node/69aef10efeb96b28aa34c4b5 HTTP/1.0" 202 Accepted 2026-03-09 19:21:37,808 - lava_callback - INFO - Starting async job submit for node 69aef10efeb96b28aa34c4b5 2026-03-09 19:21:37,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:37,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef10efeb96b28aa34c4b5 HTTP/1.1" 200 None 2026-03-09 19:21:37,844 - lava_callback - DEBUG - Processing node 69aef10efeb96b28aa34c4b5: baseline-x86 2026-03-09 19:21:38,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:38,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:38,251 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aef10efeb96b28aa34c4b5/log.txt.gz 2026-03-09 19:21:38,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:38,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:38,656 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aef10efeb96b28aa34c4b5/lava-logs.yaml 2026-03-09 19:21:38,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:38,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:38,808 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aef10efeb96b28aa34c4b5/lava_callback.json.gz 2026-03-09 19:21:38,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:38,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef10efeb96b28aa34c4b5 HTTP/1.1" 200 None 2026-03-09 19:21:38,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:38,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8240c660ead8a389ab2d HTTP/1.1" 200 None 2026-03-09 19:21:38,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:38,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aef10efeb96b28aa34c4b5 HTTP/1.1" 200 None 2026-03-09 19:21:38,896 - lava_callback - INFO - Completed processing callback for node 69aef10efeb96b28aa34c4b5 INFO: 172.21.0.1:43876 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:21:46,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:46,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:21:54,304 - lava_callback - INFO - Received callback for node 69af1c67ce46d8ec186acdcd INFO: 172.21.0.1:57300 - "POST /node/69af1c67ce46d8ec186acdcd HTTP/1.0" 202 Accepted 2026-03-09 19:21:54,311 - lava_callback - INFO - Starting async job submit for node 69af1c67ce46d8ec186acdcd 2026-03-09 19:21:54,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:54,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1c67ce46d8ec186acdcd HTTP/1.1" 200 None 2026-03-09 19:21:54,361 - lava_callback - DEBUG - Processing node 69af1c67ce46d8ec186acdcd: baseline-arm64-broonie 2026-03-09 19:21:54,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:54,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:54,930 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1c67ce46d8ec186acdcd/log.txt.gz 2026-03-09 19:21:55,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:55,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:55,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1c67ce46d8ec186acdcd/lava-logs.yaml 2026-03-09 19:21:55,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:21:55,572 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:21:55,574 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1c67ce46d8ec186acdcd/lava_callback.json.gz DEBUG: Setup failed for 69af1c67ce46d8ec186acdcd. Transitting job node result: incomplete -> fail 2026-03-09 19:21:55,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:55,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1c67ce46d8ec186acdcd HTTP/1.1" 200 None 2026-03-09 19:21:55,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:55,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:21:55,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:21:55,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1c67ce46d8ec186acdcd HTTP/1.1" 200 None 2026-03-09 19:21:55,805 - lava_callback - INFO - Completed processing callback for node 69af1c67ce46d8ec186acdcd INFO: 172.21.0.1:57310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42810 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:22:07,765 - lava_callback - INFO - Received callback for node 69aeffbace46d8ec186ac444 INFO: 172.21.0.1:42824 - "POST /node/69aeffbace46d8ec186ac444 HTTP/1.0" 202 Accepted 2026-03-09 19:22:07,770 - lava_callback - INFO - Starting async job submit for node 69aeffbace46d8ec186ac444 2026-03-09 19:22:07,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:07,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffbace46d8ec186ac444 HTTP/1.1" 200 None 2026-03-09 19:22:07,812 - lava_callback - DEBUG - Processing node 69aeffbace46d8ec186ac444: baseline-x86 2026-03-09 19:22:08,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:22:08,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:22:08,282 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffbace46d8ec186ac444/log.txt.gz 2026-03-09 19:22:08,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:22:08,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:22:08,666 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffbace46d8ec186ac444/lava-logs.yaml 2026-03-09 19:22:08,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:22:08,830 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:22:08,831 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69aeffbace46d8ec186ac444/lava_callback.json.gz 2026-03-09 19:22:08,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:08,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeffbace46d8ec186ac444 HTTP/1.1" 200 None 2026-03-09 19:22:08,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:08,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef449ce46d8ec186ab83b HTTP/1.1" 200 None 2026-03-09 19:22:08,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:08,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aeffbace46d8ec186ac444 HTTP/1.1" 200 None 2026-03-09 19:22:08,920 - lava_callback - INFO - Completed processing callback for node 69aeffbace46d8ec186ac444 2026-03-09 19:22:12,599 - lava_callback - INFO - Received callback for node 69ac7ad746f7ae8ff3366347 INFO: 172.21.0.1:54046 - "POST /node/69ac7ad746f7ae8ff3366347 HTTP/1.0" 202 Accepted 2026-03-09 19:22:12,604 - lava_callback - INFO - Starting async job submit for node 69ac7ad746f7ae8ff3366347 2026-03-09 19:22:12,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:12,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7ad746f7ae8ff3366347 HTTP/1.1" 200 None 2026-03-09 19:22:12,628 - lava_callback - DEBUG - Processing node 69ac7ad746f7ae8ff3366347: baseline-arm64-baylibre 2026-03-09 19:22:12,628 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7ad746f7ae8ff3366347 2026-03-09 19:22:12,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:12,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7ad746f7ae8ff3366347 HTTP/1.1" 200 None 2026-03-09 19:22:12,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:12,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:22:12,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:12,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7ad746f7ae8ff3366347 HTTP/1.1" 200 None 2026-03-09 19:22:12,686 - lava_callback - ERROR - Error processing callback for node 69ac7ad746f7ae8ff3366347: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54058 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:22:16,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:22:16,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34926 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:24:32,733 - lava_callback - INFO - Received callback for node 69ac7b6c46f7ae8ff336637e INFO: 172.21.0.1:38114 - "POST /node/69ac7b6c46f7ae8ff336637e HTTP/1.0" 202 Accepted 2026-03-09 19:24:32,742 - lava_callback - INFO - Starting async job submit for node 69ac7b6c46f7ae8ff336637e 2026-03-09 19:24:32,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:24:32,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7b6c46f7ae8ff336637e HTTP/1.1" 200 None 2026-03-09 19:24:32,758 - lava_callback - DEBUG - Processing node 69ac7b6c46f7ae8ff336637e: baseline-arm64-baylibre 2026-03-09 19:24:32,758 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7b6c46f7ae8ff336637e 2026-03-09 19:24:32,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:24:32,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7b6c46f7ae8ff336637e HTTP/1.1" 200 None 2026-03-09 19:24:32,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:24:32,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d3423d518458fb340278 HTTP/1.1" 200 None 2026-03-09 19:24:32,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:24:32,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7b6c46f7ae8ff336637e HTTP/1.1" 200 None 2026-03-09 19:24:32,819 - lava_callback - ERROR - Error processing callback for node 69ac7b6c46f7ae8ff336637e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33836 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:25:29,368 - lava_callback - INFO - Received callback for node 69af1cb5ce46d8ec186acddd INFO: 172.21.0.1:33852 - "POST /node/69af1cb5ce46d8ec186acddd HTTP/1.0" 202 Accepted 2026-03-09 19:25:29,374 - lava_callback - INFO - Starting async job submit for node 69af1cb5ce46d8ec186acddd 2026-03-09 19:25:29,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:25:29,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1cb5ce46d8ec186acddd HTTP/1.1" 200 None 2026-03-09 19:25:29,428 - lava_callback - DEBUG - Processing node 69af1cb5ce46d8ec186acddd: baseline-arm64-broonie 2026-03-09 19:25:29,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:25:29,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:25:29,928 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1cb5ce46d8ec186acddd/log.txt.gz 2026-03-09 19:25:30,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:25:30,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:25:30,380 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1cb5ce46d8ec186acddd/lava-logs.yaml 2026-03-09 19:25:30,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:25:30,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:25:30,553 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1cb5ce46d8ec186acddd/lava_callback.json.gz DEBUG: Setup failed for 69af1cb5ce46d8ec186acddd. Transitting job node result: incomplete -> fail 2026-03-09 19:25:30,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:25:30,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1cb5ce46d8ec186acddd HTTP/1.1" 200 None 2026-03-09 19:25:30,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:25:30,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:25:30,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:25:30,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1cb5ce46d8ec186acddd HTTP/1.1" 200 None 2026-03-09 19:25:30,792 - lava_callback - INFO - Completed processing callback for node 69af1cb5ce46d8ec186acddd INFO: 172.21.0.1:43790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51966 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:25:46,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:25:46,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41646 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:31:14,059 - lava_callback - INFO - Received callback for node 69af1e36ce46d8ec186ace7b INFO: 172.21.0.1:41682 - "POST /node/69af1e36ce46d8ec186ace7b HTTP/1.0" 202 Accepted 2026-03-09 19:31:14,064 - lava_callback - INFO - Starting async job submit for node 69af1e36ce46d8ec186ace7b 2026-03-09 19:31:14,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:31:14,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1e36ce46d8ec186ace7b HTTP/1.1" 200 None 2026-03-09 19:31:14,082 - lava_callback - DEBUG - Processing node 69af1e36ce46d8ec186ace7b: baseline-arm64-broonie 2026-03-09 19:31:14,087 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:31:14,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:31:14,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:31:14,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1e36ce46d8ec186ace7b/lava-logs.yaml 2026-03-09 19:31:14,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:31:14,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:31:14,429 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af1e36ce46d8ec186ace7b/lava_callback.json.gz 2026-03-09 19:31:14,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:31:14,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1e36ce46d8ec186ace7b HTTP/1.1" 200 None 2026-03-09 19:31:14,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:31:14,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:31:14,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:31:14,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1e36ce46d8ec186ace7b HTTP/1.1" 200 None 2026-03-09 19:31:14,498 - lava_callback - INFO - Completed processing callback for node 69af1e36ce46d8ec186ace7b INFO: 172.21.0.1:41692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41702 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:31:16,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:31:16,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39988 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:32:35,156 - lava_callback - INFO - Received callback for node 69af203dce46d8ec186aced5 INFO: 172.21.0.1:37334 - "POST /node/69af203dce46d8ec186aced5 HTTP/1.0" 202 Accepted 2026-03-09 19:32:35,162 - lava_callback - INFO - Starting async job submit for node 69af203dce46d8ec186aced5 2026-03-09 19:32:35,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:32:35,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af203dce46d8ec186aced5 HTTP/1.1" 200 None 2026-03-09 19:32:35,180 - lava_callback - DEBUG - Processing node 69af203dce46d8ec186aced5: baseline-arm64 2026-03-09 19:32:35,186 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:32:35,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:32:35,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:32:35,348 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af203dce46d8ec186aced5/lava-logs.yaml 2026-03-09 19:32:35,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:32:35,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:32:35,497 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af203dce46d8ec186aced5/lava_callback.json.gz 2026-03-09 19:32:35,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:32:35,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af203dce46d8ec186aced5 HTTP/1.1" 200 None 2026-03-09 19:32:35,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:32:35,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:32:35,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:32:35,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af203dce46d8ec186aced5 HTTP/1.1" 200 None 2026-03-09 19:32:35,563 - lava_callback - INFO - Completed processing callback for node 69af203dce46d8ec186aced5 INFO: 172.21.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:32:46,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:32:46,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60890 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:33:33,291 - lava_callback - INFO - Received callback for node 69ac7d7d46f7ae8ff3366448 INFO: 172.21.0.1:42734 - "POST /node/69ac7d7d46f7ae8ff3366448 HTTP/1.0" 202 Accepted 2026-03-09 19:33:33,296 - lava_callback - INFO - Starting async job submit for node 69ac7d7d46f7ae8ff3366448 2026-03-09 19:33:33,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:33:33,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7d7d46f7ae8ff3366448 HTTP/1.1" 200 None 2026-03-09 19:33:33,313 - lava_callback - DEBUG - Processing node 69ac7d7d46f7ae8ff3366448: baseline-arm64-baylibre 2026-03-09 19:33:33,313 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7d7d46f7ae8ff3366448 2026-03-09 19:33:33,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:33:33,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7d7d46f7ae8ff3366448 HTTP/1.1" 200 None 2026-03-09 19:33:33,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:33:33,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:33:33,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:33:33,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7d7d46f7ae8ff3366448 HTTP/1.1" 200 None 2026-03-09 19:33:33,372 - lava_callback - ERROR - Error processing callback for node 69ac7d7d46f7ae8ff3366448: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41998 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:34:33,244 - lava_callback - INFO - Received callback for node 69ac7dbe46f7ae8ff336645c INFO: 172.21.0.1:46136 - "POST /node/69ac7dbe46f7ae8ff336645c HTTP/1.0" 202 Accepted 2026-03-09 19:34:33,249 - lava_callback - INFO - Starting async job submit for node 69ac7dbe46f7ae8ff336645c 2026-03-09 19:34:33,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:34:33,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7dbe46f7ae8ff336645c HTTP/1.1" 200 None 2026-03-09 19:34:33,266 - lava_callback - DEBUG - Processing node 69ac7dbe46f7ae8ff336645c: baseline-arm-baylibre 2026-03-09 19:34:33,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7dbe46f7ae8ff336645c 2026-03-09 19:34:33,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:34:33,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7dbe46f7ae8ff336645c HTTP/1.1" 200 None 2026-03-09 19:34:33,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:34:33,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:34:33,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:34:33,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7dbe46f7ae8ff336645c HTTP/1.1" 200 None 2026-03-09 19:34:33,331 - lava_callback - ERROR - Error processing callback for node 69ac7dbe46f7ae8ff336645c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40820 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:35:13,321 - lava_callback - INFO - Received callback for node 69ac7dec46f7ae8ff3366479 INFO: 172.21.0.1:48710 - "POST /node/69ac7dec46f7ae8ff3366479 HTTP/1.0" 202 Accepted 2026-03-09 19:35:13,326 - lava_callback - INFO - Starting async job submit for node 69ac7dec46f7ae8ff3366479 2026-03-09 19:35:13,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:35:13,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7dec46f7ae8ff3366479 HTTP/1.1" 200 None 2026-03-09 19:35:13,342 - lava_callback - DEBUG - Processing node 69ac7dec46f7ae8ff3366479: baseline-arm64-baylibre 2026-03-09 19:35:13,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7dec46f7ae8ff3366479 2026-03-09 19:35:13,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:35:13,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7dec46f7ae8ff3366479 HTTP/1.1" 200 None 2026-03-09 19:35:13,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:35:13,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:35:13,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:35:13,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7dec46f7ae8ff3366479 HTTP/1.1" 200 None 2026-03-09 19:35:13,400 - lava_callback - ERROR - Error processing callback for node 69ac7dec46f7ae8ff3366479: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32996 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:36:33,409 - lava_callback - INFO - Received callback for node 69ac7e2c46f7ae8ff3366489 INFO: 172.21.0.1:35236 - "POST /node/69ac7e2c46f7ae8ff3366489 HTTP/1.0" 202 Accepted 2026-03-09 19:36:33,415 - lava_callback - INFO - Starting async job submit for node 69ac7e2c46f7ae8ff3366489 2026-03-09 19:36:33,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:36:33,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e2c46f7ae8ff3366489 HTTP/1.1" 200 None 2026-03-09 19:36:33,430 - lava_callback - DEBUG - Processing node 69ac7e2c46f7ae8ff3366489: baseline-arm-baylibre 2026-03-09 19:36:33,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7e2c46f7ae8ff3366489 2026-03-09 19:36:33,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:36:33,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e2c46f7ae8ff3366489 HTTP/1.1" 200 None 2026-03-09 19:36:33,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:36:33,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:36:33,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:36:33,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7e2c46f7ae8ff3366489 HTTP/1.1" 200 None 2026-03-09 19:36:33,529 - lava_callback - ERROR - Error processing callback for node 69ac7e2c46f7ae8ff3366489: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42130 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:37:13,404 - lava_callback - INFO - Received callback for node 69ac7e5a46f7ae8ff336649b INFO: 172.21.0.1:54368 - "POST /node/69ac7e5a46f7ae8ff336649b HTTP/1.0" 202 Accepted 2026-03-09 19:37:13,409 - lava_callback - INFO - Starting async job submit for node 69ac7e5a46f7ae8ff336649b 2026-03-09 19:37:13,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:37:13,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e5a46f7ae8ff336649b HTTP/1.1" 200 None 2026-03-09 19:37:13,425 - lava_callback - DEBUG - Processing node 69ac7e5a46f7ae8ff336649b: baseline-arm64-baylibre 2026-03-09 19:37:13,425 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7e5a46f7ae8ff336649b 2026-03-09 19:37:13,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:37:13,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e5a46f7ae8ff336649b HTTP/1.1" 200 None 2026-03-09 19:37:13,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:37:13,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:37:13,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:37:13,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7e5a46f7ae8ff336649b HTTP/1.1" 200 None 2026-03-09 19:37:13,488 - lava_callback - ERROR - Error processing callback for node 69ac7e5a46f7ae8ff336649b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50962 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:38:07,594 - lava_callback - INFO - Received callback for node 69af1f0ace46d8ec186acea5 INFO: 172.21.0.1:50972 - "POST /node/69af1f0ace46d8ec186acea5 HTTP/1.0" 202 Accepted 2026-03-09 19:38:07,604 - lava_callback - INFO - Starting async job submit for node 69af1f0ace46d8ec186acea5 2026-03-09 19:38:07,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:07,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1f0ace46d8ec186acea5 HTTP/1.1" 200 None 2026-03-09 19:38:07,661 - lava_callback - DEBUG - Processing node 69af1f0ace46d8ec186acea5: baseline-arm64 2026-03-09 19:38:07,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:38:08,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:38:08,149 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f0ace46d8ec186acea5/log.txt.gz 2026-03-09 19:38:08,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:38:09,705 - lava_callback - INFO - Received callback for node 69af2176ce46d8ec186aceff INFO: 172.21.0.1:50832 - "POST /node/69af2176ce46d8ec186aceff HTTP/1.0" 202 Accepted 2026-03-09 19:38:09,711 - lava_callback - INFO - Starting async job submit for node 69af2176ce46d8ec186aceff 2026-03-09 19:38:09,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:09,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2176ce46d8ec186aceff HTTP/1.1" 200 None 2026-03-09 19:38:09,743 - lava_callback - DEBUG - Processing node 69af2176ce46d8ec186aceff: baseline-arm64 2026-03-09 19:38:09,793 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:38:09,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:38:09,975 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:38:09,977 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2176ce46d8ec186aceff/lava-logs.yaml 2026-03-09 19:38:09,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:38:10,117 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:38:10,119 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2176ce46d8ec186aceff/lava_callback.json.gz 2026-03-09 19:38:10,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:10,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2176ce46d8ec186aceff HTTP/1.1" 200 None 2026-03-09 19:38:10,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:10,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:38:10,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:10,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2176ce46d8ec186aceff HTTP/1.1" 200 None 2026-03-09 19:38:10,239 - lava_callback - INFO - Completed processing callback for node 69af2176ce46d8ec186aceff 2026-03-09 19:38:12,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:38:12,541 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f0ace46d8ec186acea5/lava-logs.yaml 2026-03-09 19:38:12,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:38:12,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:38:12,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f0ace46d8ec186acea5/lava_callback.json.gz 2026-03-09 19:38:12,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:12,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1f0ace46d8ec186acea5 HTTP/1.1" 200 None 2026-03-09 19:38:12,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:12,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:38:12,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:12,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1f0ace46d8ec186acea5 HTTP/1.1" 200 None 2026-03-09 19:38:12,847 - lava_callback - INFO - Completed processing callback for node 69af1f0ace46d8ec186acea5 2026-03-09 19:38:13,423 - lava_callback - INFO - Received callback for node 69ac7e9a46f7ae8ff33664b6 INFO: 172.21.0.1:50840 - "POST /node/69ac7e9a46f7ae8ff33664b6 HTTP/1.0" 202 Accepted 2026-03-09 19:38:13,428 - lava_callback - INFO - Starting async job submit for node 69ac7e9a46f7ae8ff33664b6 2026-03-09 19:38:13,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:13,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e9a46f7ae8ff33664b6 HTTP/1.1" 200 None 2026-03-09 19:38:13,445 - lava_callback - DEBUG - Processing node 69ac7e9a46f7ae8ff33664b6: baseline-arm-baylibre 2026-03-09 19:38:13,445 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7e9a46f7ae8ff33664b6 2026-03-09 19:38:13,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:13,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7e9a46f7ae8ff33664b6 HTTP/1.1" 200 None 2026-03-09 19:38:13,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:13,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:38:13,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:13,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7e9a46f7ae8ff33664b6 HTTP/1.1" 200 None 2026-03-09 19:38:13,499 - lava_callback - ERROR - Error processing callback for node 69ac7e9a46f7ae8ff33664b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50866 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:38:16,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:38:16,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56238 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:39:03,039 - lava_callback - INFO - Received callback for node 69af1f5ace46d8ec186aceaf INFO: 172.21.0.1:60836 - "POST /node/69af1f5ace46d8ec186aceaf HTTP/1.0" 202 Accepted 2026-03-09 19:39:03,052 - lava_callback - INFO - Starting async job submit for node 69af1f5ace46d8ec186aceaf 2026-03-09 19:39:03,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:03,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1f5ace46d8ec186aceaf HTTP/1.1" 200 None 2026-03-09 19:39:03,094 - lava_callback - DEBUG - Processing node 69af1f5ace46d8ec186aceaf: baseline-arm64 2026-03-09 19:39:03,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:39:03,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:39:03,764 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f5ace46d8ec186aceaf/log.txt.gz 2026-03-09 19:39:04,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:39:04,309 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:39:04,311 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f5ace46d8ec186aceaf/lava-logs.yaml 2026-03-09 19:39:04,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:39:04,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:39:04,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af1f5ace46d8ec186aceaf/lava_callback.json.gz 2026-03-09 19:39:04,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:04,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af1f5ace46d8ec186aceaf HTTP/1.1" 200 None 2026-03-09 19:39:04,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:04,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:39:04,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:04,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af1f5ace46d8ec186aceaf HTTP/1.1" 200 None 2026-03-09 19:39:04,613 - lava_callback - INFO - Completed processing callback for node 69af1f5ace46d8ec186aceaf INFO: 172.21.0.1:60838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36386 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:39:16,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:16,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54920 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:39:53,585 - lava_callback - INFO - Received callback for node 69ac7f0646f7ae8ff33664db INFO: 172.21.0.1:52116 - "POST /node/69ac7f0646f7ae8ff33664db HTTP/1.0" 202 Accepted 2026-03-09 19:39:53,590 - lava_callback - INFO - Starting async job submit for node 69ac7f0646f7ae8ff33664db 2026-03-09 19:39:53,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:53,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7f0646f7ae8ff33664db HTTP/1.1" 200 None 2026-03-09 19:39:53,607 - lava_callback - DEBUG - Processing node 69ac7f0646f7ae8ff33664db: baseline-arm-baylibre 2026-03-09 19:39:53,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7f0646f7ae8ff33664db 2026-03-09 19:39:53,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:53,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7f0646f7ae8ff33664db HTTP/1.1" 200 None 2026-03-09 19:39:53,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:53,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:39:53,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:39:53,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7f0646f7ae8ff33664db HTTP/1.1" 200 None 2026-03-09 19:39:53,670 - lava_callback - ERROR - Error processing callback for node 69ac7f0646f7ae8ff33664db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50578 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:40:44,066 - lava_callback - INFO - Received callback for node 69af221bce46d8ec186acf1c INFO: 172.21.0.1:57548 - "POST /node/69af221bce46d8ec186acf1c HTTP/1.0" 202 Accepted 2026-03-09 19:40:44,072 - lava_callback - INFO - Starting async job submit for node 69af221bce46d8ec186acf1c 2026-03-09 19:40:44,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:40:44,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af221bce46d8ec186acf1c HTTP/1.1" 200 None 2026-03-09 19:40:44,089 - lava_callback - DEBUG - Processing node 69af221bce46d8ec186acf1c: baseline-nfs-arm64 2026-03-09 19:40:44,091 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:40:44,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:40:44,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:40:44,283 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af221bce46d8ec186acf1c/lava-logs.yaml 2026-03-09 19:40:44,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:40:44,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:40:44,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af221bce46d8ec186acf1c/lava_callback.json.gz 2026-03-09 19:40:44,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:40:44,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af221bce46d8ec186acf1c HTTP/1.1" 200 None 2026-03-09 19:40:44,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:40:44,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:40:44,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:40:44,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af221bce46d8ec186acf1c HTTP/1.1" 200 None 2026-03-09 19:40:44,489 - lava_callback - INFO - Completed processing callback for node 69af221bce46d8ec186acf1c INFO: 172.21.0.1:57564 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:40:46,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:40:46,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59826 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:41:53,699 - lava_callback - INFO - Received callback for node 69ac7f7346f7ae8ff33664fa INFO: 172.21.0.1:42390 - "POST /node/69ac7f7346f7ae8ff33664fa HTTP/1.0" 202 Accepted 2026-03-09 19:41:53,704 - lava_callback - INFO - Starting async job submit for node 69ac7f7346f7ae8ff33664fa 2026-03-09 19:41:53,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:41:53,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7f7346f7ae8ff33664fa HTTP/1.1" 200 None 2026-03-09 19:41:53,720 - lava_callback - DEBUG - Processing node 69ac7f7346f7ae8ff33664fa: baseline-arm-baylibre 2026-03-09 19:41:53,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7f7346f7ae8ff33664fa 2026-03-09 19:41:53,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:41:53,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7f7346f7ae8ff33664fa HTTP/1.1" 200 None 2026-03-09 19:41:53,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:41:53,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:41:53,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:41:53,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7f7346f7ae8ff33664fa HTTP/1.1" 200 None 2026-03-09 19:41:53,772 - lava_callback - ERROR - Error processing callback for node 69ac7f7346f7ae8ff33664fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42418 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:42:05,873 - lava_callback - INFO - Received callback for node 69af2266ce46d8ec186acf23 INFO: 172.21.0.1:57534 - "POST /node/69af2266ce46d8ec186acf23 HTTP/1.0" 202 Accepted 2026-03-09 19:42:05,880 - lava_callback - INFO - Starting async job submit for node 69af2266ce46d8ec186acf23 2026-03-09 19:42:05,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:05,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2266ce46d8ec186acf23 HTTP/1.1" 200 None 2026-03-09 19:42:05,911 - lava_callback - DEBUG - Processing node 69af2266ce46d8ec186acf23: baseline-arm64 2026-03-09 19:42:05,961 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:42:05,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:42:06,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:42:06,173 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2266ce46d8ec186acf23/lava-logs.yaml 2026-03-09 19:42:06,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:42:06,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:42:06,315 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2266ce46d8ec186acf23/lava_callback.json.gz 2026-03-09 19:42:06,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:06,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2266ce46d8ec186acf23 HTTP/1.1" 200 None 2026-03-09 19:42:06,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:06,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:42:06,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:06,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2266ce46d8ec186acf23 HTTP/1.1" 200 None 2026-03-09 19:42:06,435 - lava_callback - INFO - Completed processing callback for node 69af2266ce46d8ec186acf23 INFO: 172.21.0.1:57546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58646 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:42:16,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:16,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41738 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:42:33,679 - lava_callback - INFO - Received callback for node 69ac7fa146f7ae8ff336651e INFO: 172.21.0.1:42066 - "POST /node/69ac7fa146f7ae8ff336651e HTTP/1.0" 202 Accepted 2026-03-09 19:42:33,684 - lava_callback - INFO - Starting async job submit for node 69ac7fa146f7ae8ff336651e 2026-03-09 19:42:33,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:33,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7fa146f7ae8ff336651e HTTP/1.1" 200 None 2026-03-09 19:42:33,700 - lava_callback - DEBUG - Processing node 69ac7fa146f7ae8ff336651e: baseline-arm64-baylibre 2026-03-09 19:42:33,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7fa146f7ae8ff336651e 2026-03-09 19:42:33,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:33,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7fa146f7ae8ff336651e HTTP/1.1" 200 None 2026-03-09 19:42:33,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:33,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:42:33,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:42:33,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7fa146f7ae8ff336651e HTTP/1.1" 200 None 2026-03-09 19:42:33,764 - lava_callback - ERROR - Error processing callback for node 69ac7fa146f7ae8ff336651e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39838 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:43:33,744 - lava_callback - INFO - Received callback for node 69ac7fe146f7ae8ff336652b INFO: 172.21.0.1:58332 - "POST /node/69ac7fe146f7ae8ff336652b HTTP/1.0" 202 Accepted 2026-03-09 19:43:33,750 - lava_callback - INFO - Starting async job submit for node 69ac7fe146f7ae8ff336652b 2026-03-09 19:43:33,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:33,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7fe146f7ae8ff336652b HTTP/1.1" 200 None 2026-03-09 19:43:33,777 - lava_callback - DEBUG - Processing node 69ac7fe146f7ae8ff336652b: baseline-arm-baylibre 2026-03-09 19:43:33,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac7fe146f7ae8ff336652b 2026-03-09 19:43:33,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:33,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac7fe146f7ae8ff336652b HTTP/1.1" 200 None 2026-03-09 19:43:33,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:33,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:43:33,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:33,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac7fe146f7ae8ff336652b HTTP/1.1" 200 None 2026-03-09 19:43:33,856 - lava_callback - ERROR - Error processing callback for node 69ac7fe146f7ae8ff336652b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51950 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:43:54,610 - lava_callback - INFO - Received callback for node 69af2093ce46d8ec186acee5 INFO: 172.21.0.1:42578 - "POST /node/69af2093ce46d8ec186acee5 HTTP/1.0" 202 Accepted 2026-03-09 19:43:54,617 - lava_callback - INFO - Starting async job submit for node 69af2093ce46d8ec186acee5 2026-03-09 19:43:54,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:54,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2093ce46d8ec186acee5 HTTP/1.1" 200 None 2026-03-09 19:43:54,661 - lava_callback - DEBUG - Processing node 69af2093ce46d8ec186acee5: baseline-nfs-arm64 2026-03-09 19:43:55,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:43:55,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:43:55,452 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2093ce46d8ec186acee5/log.txt.gz 2026-03-09 19:43:55,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:43:56,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:43:56,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2093ce46d8ec186acee5/lava-logs.yaml 2026-03-09 19:43:56,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:43:56,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:43:56,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2093ce46d8ec186acee5/lava_callback.json.gz 2026-03-09 19:43:56,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:56,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2093ce46d8ec186acee5 HTTP/1.1" 200 None 2026-03-09 19:43:56,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:56,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:43:56,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:43:56,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2093ce46d8ec186acee5 HTTP/1.1" 200 None 2026-03-09 19:43:56,353 - lava_callback - INFO - Completed processing callback for node 69af2093ce46d8ec186acee5 INFO: 172.21.0.1:42588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40188 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:44:16,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:16,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:44:22,876 - lava_callback - INFO - Received callback for node 69af22face46d8ec186acf38 INFO: 172.21.0.1:34598 - "POST /node/69af22face46d8ec186acf38 HTTP/1.0" 202 Accepted 2026-03-09 19:44:22,882 - lava_callback - INFO - Starting async job submit for node 69af22face46d8ec186acf38 2026-03-09 19:44:22,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:22,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af22face46d8ec186acf38 HTTP/1.1" 200 None 2026-03-09 19:44:22,914 - lava_callback - DEBUG - Processing node 69af22face46d8ec186acf38: baseline-arm64 2026-03-09 19:44:22,966 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:44:22,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:44:23,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:44:23,145 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af22face46d8ec186acf38/lava-logs.yaml 2026-03-09 19:44:23,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:44:23,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:44:23,284 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af22face46d8ec186acf38/lava_callback.json.gz 2026-03-09 19:44:23,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:23,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af22face46d8ec186acf38 HTTP/1.1" 200 None 2026-03-09 19:44:23,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:23,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:44:23,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:23,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af22face46d8ec186acf38 HTTP/1.1" 200 None 2026-03-09 19:44:23,410 - lava_callback - INFO - Completed processing callback for node 69af22face46d8ec186acf38 2026-03-09 19:44:25,315 - lava_callback - INFO - Received callback for node 69ac801446f7ae8ff336653a INFO: 172.21.0.1:34612 - "POST /node/69ac801446f7ae8ff336653a HTTP/1.0" 202 Accepted 2026-03-09 19:44:25,320 - lava_callback - INFO - Starting async job submit for node 69ac801446f7ae8ff336653a 2026-03-09 19:44:25,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:25,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac801446f7ae8ff336653a HTTP/1.1" 200 None 2026-03-09 19:44:25,336 - lava_callback - DEBUG - Processing node 69ac801446f7ae8ff336653a: baseline-arm64-clabbe 2026-03-09 19:44:25,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac801446f7ae8ff336653a 2026-03-09 19:44:25,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:25,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac801446f7ae8ff336653a HTTP/1.1" 200 None 2026-03-09 19:44:25,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:25,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:44:25,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:25,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac801446f7ae8ff336653a HTTP/1.1" 200 None 2026-03-09 19:44:25,393 - lava_callback - ERROR - Error processing callback for node 69ac801446f7ae8ff336653a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56000 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:44:46,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:44:46,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47158 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:45:33,949 - lava_callback - INFO - Received callback for node 69ac804b46f7ae8ff3366557 INFO: 172.21.0.1:57276 - "POST /node/69ac804b46f7ae8ff3366557 HTTP/1.0" 202 Accepted 2026-03-09 19:45:33,954 - lava_callback - INFO - Starting async job submit for node 69ac804b46f7ae8ff3366557 2026-03-09 19:45:33,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:33,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac804b46f7ae8ff3366557 HTTP/1.1" 200 None 2026-03-09 19:45:33,974 - lava_callback - DEBUG - Processing node 69ac804b46f7ae8ff3366557: baseline-arm-baylibre 2026-03-09 19:45:33,974 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac804b46f7ae8ff3366557 2026-03-09 19:45:33,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:33,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac804b46f7ae8ff3366557 HTTP/1.1" 200 None 2026-03-09 19:45:33,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:34,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:45:34,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:34,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac804b46f7ae8ff3366557 HTTP/1.1" 200 None 2026-03-09 19:45:34,036 - lava_callback - ERROR - Error processing callback for node 69ac804b46f7ae8ff3366557: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57284 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:45:39,428 - lava_callback - INFO - Received callback for node 69af2350ce46d8ec186acf4a INFO: 172.21.0.1:57286 - "POST /node/69af2350ce46d8ec186acf4a HTTP/1.0" 202 Accepted 2026-03-09 19:45:39,434 - lava_callback - INFO - Starting async job submit for node 69af2350ce46d8ec186acf4a 2026-03-09 19:45:39,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:39,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2350ce46d8ec186acf4a HTTP/1.1" 200 None 2026-03-09 19:45:39,453 - lava_callback - DEBUG - Processing node 69af2350ce46d8ec186acf4a: baseline-nfs-arm64 2026-03-09 19:45:39,459 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:45:39,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:45:39,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:45:39,612 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2350ce46d8ec186acf4a/lava-logs.yaml 2026-03-09 19:45:39,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:45:39,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:45:39,753 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2350ce46d8ec186acf4a/lava_callback.json.gz 2026-03-09 19:45:39,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:39,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2350ce46d8ec186acf4a HTTP/1.1" 200 None 2026-03-09 19:45:39,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:39,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:45:39,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:39,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2350ce46d8ec186acf4a HTTP/1.1" 200 None 2026-03-09 19:45:39,819 - lava_callback - INFO - Completed processing callback for node 69af2350ce46d8ec186acf4a INFO: 172.21.0.1:51692 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:45:46,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:46,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:45:53,866 - lava_callback - INFO - Received callback for node 69ac807046f7ae8ff3366563 INFO: 172.21.0.1:42114 - "POST /node/69ac807046f7ae8ff3366563 HTTP/1.0" 202 Accepted 2026-03-09 19:45:53,871 - lava_callback - INFO - Starting async job submit for node 69ac807046f7ae8ff3366563 2026-03-09 19:45:53,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:53,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac807046f7ae8ff3366563 HTTP/1.1" 200 None 2026-03-09 19:45:53,891 - lava_callback - DEBUG - Processing node 69ac807046f7ae8ff3366563: baseline-arm-baylibre 2026-03-09 19:45:53,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac807046f7ae8ff3366563 2026-03-09 19:45:53,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:53,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac807046f7ae8ff3366563 HTTP/1.1" 200 None 2026-03-09 19:45:53,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:53,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:45:53,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:45:53,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac807046f7ae8ff3366563 HTTP/1.1" 200 None 2026-03-09 19:45:53,947 - lava_callback - ERROR - Error processing callback for node 69ac807046f7ae8ff3366563: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55122 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:46:30,966 - lava_callback - INFO - Received callback for node 69af22bece46d8ec186acf31 INFO: 172.21.0.1:41626 - "POST /node/69af22bece46d8ec186acf31 HTTP/1.0" 202 Accepted 2026-03-09 19:46:30,972 - lava_callback - INFO - Starting async job submit for node 69af22bece46d8ec186acf31 2026-03-09 19:46:30,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:46:31,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af22bece46d8ec186acf31 HTTP/1.1" 200 None 2026-03-09 19:46:31,006 - lava_callback - DEBUG - Processing node 69af22bece46d8ec186acf31: baseline-arm64-broonie 2026-03-09 19:46:31,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:46:31,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:46:31,491 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af22bece46d8ec186acf31/log.txt.gz 2026-03-09 19:46:31,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:46:31,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:46:31,905 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af22bece46d8ec186acf31/lava-logs.yaml 2026-03-09 19:46:31,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:46:32,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:46:32,043 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af22bece46d8ec186acf31/lava_callback.json.gz 2026-03-09 19:46:32,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:46:32,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af22bece46d8ec186acf31 HTTP/1.1" 200 None 2026-03-09 19:46:32,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:46:32,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:46:32,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:46:32,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af22bece46d8ec186acf31 HTTP/1.1" 200 None 2026-03-09 19:46:32,206 - lava_callback - INFO - Completed processing callback for node 69af22bece46d8ec186acf31 INFO: 172.21.0.1:41642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34546 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:46:46,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:46:46,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41704 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:47:05,372 - lava_callback - INFO - Received callback for node 69ac80a646f7ae8ff3366575 INFO: 172.21.0.1:37854 - "POST /node/69ac80a646f7ae8ff3366575 HTTP/1.0" 202 Accepted 2026-03-09 19:47:05,377 - lava_callback - INFO - Starting async job submit for node 69ac80a646f7ae8ff3366575 2026-03-09 19:47:05,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:05,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac80a646f7ae8ff3366575 HTTP/1.1" 200 None 2026-03-09 19:47:05,395 - lava_callback - DEBUG - Processing node 69ac80a646f7ae8ff3366575: baseline-arm64-clabbe 2026-03-09 19:47:05,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac80a646f7ae8ff3366575 2026-03-09 19:47:05,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:05,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac80a646f7ae8ff3366575 HTTP/1.1" 200 None 2026-03-09 19:47:05,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:05,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ab6e44efa461f55e59cdc8 HTTP/1.1" 200 None 2026-03-09 19:47:05,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:05,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac80a646f7ae8ff3366575 HTTP/1.1" 200 None 2026-03-09 19:47:05,494 - lava_callback - ERROR - Error processing callback for node 69ac80a646f7ae8ff3366575: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59288 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:47:29,625 - lava_callback - INFO - Received callback for node 69af212ece46d8ec186acef7 INFO: 172.21.0.1:59290 - "POST /node/69af212ece46d8ec186acef7 HTTP/1.0" 202 Accepted 2026-03-09 19:47:29,632 - lava_callback - INFO - Starting async job submit for node 69af212ece46d8ec186acef7 2026-03-09 19:47:29,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:29,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af212ece46d8ec186acef7 HTTP/1.1" 200 None 2026-03-09 19:47:29,679 - lava_callback - DEBUG - Processing node 69af212ece46d8ec186acef7: baseline-nfs-arm64 2026-03-09 19:47:29,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:47:30,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:47:30,137 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af212ece46d8ec186acef7/log.txt.gz 2026-03-09 19:47:30,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:47:30,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:47:30,578 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af212ece46d8ec186acef7/lava-logs.yaml 2026-03-09 19:47:30,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:47:30,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:47:30,725 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af212ece46d8ec186acef7/lava_callback.json.gz 2026-03-09 19:47:30,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:30,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af212ece46d8ec186acef7 HTTP/1.1" 200 None 2026-03-09 19:47:30,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:30,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:47:30,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:30,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af212ece46d8ec186acef7 HTTP/1.1" 200 None 2026-03-09 19:47:30,905 - lava_callback - INFO - Completed processing callback for node 69af212ece46d8ec186acef7 INFO: 172.21.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49116 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:47:46,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:46,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 19:47:54,040 - lava_callback - INFO - Received callback for node 69ac80df46f7ae8ff336658d INFO: 172.21.0.1:55386 - "POST /node/69ac80df46f7ae8ff336658d HTTP/1.0" 202 Accepted 2026-03-09 19:47:54,046 - lava_callback - INFO - Starting async job submit for node 69ac80df46f7ae8ff336658d 2026-03-09 19:47:54,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:54,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac80df46f7ae8ff336658d HTTP/1.1" 200 None 2026-03-09 19:47:54,062 - lava_callback - DEBUG - Processing node 69ac80df46f7ae8ff336658d: baseline-arm-baylibre 2026-03-09 19:47:54,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac80df46f7ae8ff336658d 2026-03-09 19:47:54,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:54,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac80df46f7ae8ff336658d HTTP/1.1" 200 None 2026-03-09 19:47:54,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:54,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:47:54,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:47:54,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac80df46f7ae8ff336658d HTTP/1.1" 200 None 2026-03-09 19:47:54,124 - lava_callback - ERROR - Error processing callback for node 69ac80df46f7ae8ff336658d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55806 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:48:26,329 - lava_callback - INFO - Received callback for node 69af23fdce46d8ec186acf6f INFO: 172.21.0.1:55808 - "POST /node/69af23fdce46d8ec186acf6f HTTP/1.0" 202 Accepted 2026-03-09 19:48:26,335 - lava_callback - INFO - Starting async job submit for node 69af23fdce46d8ec186acf6f 2026-03-09 19:48:26,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:26,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af23fdce46d8ec186acf6f HTTP/1.1" 200 None 2026-03-09 19:48:26,367 - lava_callback - DEBUG - Processing node 69af23fdce46d8ec186acf6f: baseline-arm64-foundriesio 2026-03-09 19:48:26,429 - lava_callback - WARNING - Failed to upload log 2026-03-09 19:48:26,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:55814 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:48:26,589 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:48:26,591 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af23fdce46d8ec186acf6f/lava-logs.yaml 2026-03-09 19:48:26,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:48:26,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:48:26,721 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af23fdce46d8ec186acf6f/lava_callback.json.gz 2026-03-09 19:48:26,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:26,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af23fdce46d8ec186acf6f HTTP/1.1" 200 None 2026-03-09 19:48:26,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:26,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 19:48:26,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:26,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af23fdce46d8ec186acf6f HTTP/1.1" 200 None 2026-03-09 19:48:26,841 - lava_callback - INFO - Completed processing callback for node 69af23fdce46d8ec186acf6f INFO: 172.21.0.1:55826 - "GET /favicon.ico HTTP/1.0" 404 Not Found 2026-03-09 19:48:34,045 - lava_callback - INFO - Received callback for node 69ac810346f7ae8ff3366599 INFO: 172.21.0.1:38676 - "POST /node/69ac810346f7ae8ff3366599 HTTP/1.0" 202 Accepted 2026-03-09 19:48:34,051 - lava_callback - INFO - Starting async job submit for node 69ac810346f7ae8ff3366599 2026-03-09 19:48:34,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:34,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac810346f7ae8ff3366599 HTTP/1.1" 200 None 2026-03-09 19:48:34,069 - lava_callback - DEBUG - Processing node 69ac810346f7ae8ff3366599: baseline-arm-baylibre 2026-03-09 19:48:34,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac810346f7ae8ff3366599 2026-03-09 19:48:34,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:34,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac810346f7ae8ff3366599 HTTP/1.1" 200 None 2026-03-09 19:48:34,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:34,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:48:34,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:34,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac810346f7ae8ff3366599 HTTP/1.1" 200 None 2026-03-09 19:48:34,134 - lava_callback - ERROR - Error processing callback for node 69ac810346f7ae8ff3366599: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38686 - "GET /.well-known/security.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:40810 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:48:46,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:48:46,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49508 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:49:14,091 - lava_callback - INFO - Received callback for node 69ac812846f7ae8ff33665a9 INFO: 172.21.0.1:47602 - "POST /node/69ac812846f7ae8ff33665a9 HTTP/1.0" 202 Accepted 2026-03-09 19:49:14,097 - lava_callback - INFO - Starting async job submit for node 69ac812846f7ae8ff33665a9 2026-03-09 19:49:14,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:14,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac812846f7ae8ff33665a9 HTTP/1.1" 200 None 2026-03-09 19:49:14,117 - lava_callback - DEBUG - Processing node 69ac812846f7ae8ff33665a9: baseline-arm-baylibre 2026-03-09 19:49:14,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac812846f7ae8ff33665a9 2026-03-09 19:49:14,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:14,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac812846f7ae8ff33665a9 HTTP/1.1" 200 None 2026-03-09 19:49:14,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:14,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:49:14,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:14,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac812846f7ae8ff33665a9 HTTP/1.1" 200 None 2026-03-09 19:49:14,169 - lava_callback - ERROR - Error processing callback for node 69ac812846f7ae8ff33665a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35668 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:49:54,102 - lava_callback - INFO - Received callback for node 69ac814d46f7ae8ff33665b6 INFO: 172.21.0.1:33766 - "POST /node/69ac814d46f7ae8ff33665b6 HTTP/1.0" 202 Accepted 2026-03-09 19:49:54,108 - lava_callback - INFO - Starting async job submit for node 69ac814d46f7ae8ff33665b6 2026-03-09 19:49:54,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:54,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac814d46f7ae8ff33665b6 HTTP/1.1" 200 None 2026-03-09 19:49:54,125 - lava_callback - DEBUG - Processing node 69ac814d46f7ae8ff33665b6: baseline-arm-baylibre 2026-03-09 19:49:54,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac814d46f7ae8ff33665b6 2026-03-09 19:49:54,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:54,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac814d46f7ae8ff33665b6 HTTP/1.1" 200 None 2026-03-09 19:49:54,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:54,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6d31c3d518458fb340253 HTTP/1.1" 200 None 2026-03-09 19:49:54,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:49:54,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac814d46f7ae8ff33665b6 HTTP/1.1" 200 None 2026-03-09 19:49:54,179 - lava_callback - ERROR - Error processing callback for node 69ac814d46f7ae8ff33665b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:54:36,626 - lava_callback - INFO - Received callback for node 69af251dce46d8ec186acf97 INFO: 172.21.0.1:50542 - "POST /node/69af251dce46d8ec186acf97 HTTP/1.0" 202 Accepted 2026-03-09 19:54:36,632 - lava_callback - INFO - Starting async job submit for node 69af251dce46d8ec186acf97 2026-03-09 19:54:36,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:54:36,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af251dce46d8ec186acf97 HTTP/1.1" 200 None 2026-03-09 19:54:36,663 - lava_callback - DEBUG - Processing node 69af251dce46d8ec186acf97: baseline-x86 2026-03-09 19:54:36,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:54:37,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:54:37,115 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af251dce46d8ec186acf97/log.txt.gz 2026-03-09 19:54:37,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:54:37,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:54:37,472 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af251dce46d8ec186acf97/lava-logs.yaml 2026-03-09 19:54:37,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 19:54:37,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 19:54:37,633 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af251dce46d8ec186acf97/lava_callback.json.gz 2026-03-09 19:54:37,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:54:37,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af251dce46d8ec186acf97 HTTP/1.1" 200 None 2026-03-09 19:54:37,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:54:37,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba54feb96b28aa34a79d HTTP/1.1" 200 None 2026-03-09 19:54:37,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:54:37,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af251dce46d8ec186acf97 HTTP/1.1" 200 None 2026-03-09 19:54:37,721 - lava_callback - INFO - Completed processing callback for node 69af251dce46d8ec186acf97 INFO: 172.21.0.1:48864 - "GET / HTTP/1.0" 200 OK 2026-03-09 19:54:46,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 19:54:46,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43662 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33512 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:00:22,527 - lava_callback - INFO - Received callback for node 69af26bace46d8ec186acfdb INFO: 172.21.0.1:44854 - "POST /node/69af26bace46d8ec186acfdb HTTP/1.0" 202 Accepted 2026-03-09 20:00:22,533 - lava_callback - INFO - Starting async job submit for node 69af26bace46d8ec186acfdb 2026-03-09 20:00:22,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:00:22,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af26bace46d8ec186acfdb HTTP/1.1" 200 None 2026-03-09 20:00:22,551 - lava_callback - DEBUG - Processing node 69af26bace46d8ec186acfdb: baseline-arm64-clabbe 2026-03-09 20:00:22,555 - lava_callback - WARNING - Failed to upload log 2026-03-09 20:00:22,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:00:22,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:00:22,692 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af26bace46d8ec186acfdb/lava-logs.yaml 2026-03-09 20:00:22,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:00:22,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:00:22,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69af26bace46d8ec186acfdb/lava_callback.json.gz 2026-03-09 20:00:22,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:00:22,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af26bace46d8ec186acfdb HTTP/1.1" 200 None 2026-03-09 20:00:22,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:00:22,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:00:22,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:00:22,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af26bace46d8ec186acfdb HTTP/1.1" 200 None 2026-03-09 20:00:22,901 - lava_callback - INFO - Completed processing callback for node 69af26bace46d8ec186acfdb INFO: 172.21.0.1:44866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58646 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:00:46,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:00:47,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44584 - "GET /login HTTP/1.0" 404 Not Found INFO: 172.21.0.1:44596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59290 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:04:10,248 - lava_callback - INFO - Received callback for node 69af2346ce46d8ec186acf47 INFO: 172.21.0.1:55954 - "POST /node/69af2346ce46d8ec186acf47 HTTP/1.0" 202 Accepted 2026-03-09 20:04:10,255 - lava_callback - INFO - Starting async job submit for node 69af2346ce46d8ec186acf47 2026-03-09 20:04:10,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:04:10,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2346ce46d8ec186acf47 HTTP/1.1" 200 None 2026-03-09 20:04:10,328 - lava_callback - DEBUG - Processing node 69af2346ce46d8ec186acf47: ltp-smoke 2026-03-09 20:04:10,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:04:10,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:04:10,982 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69af2346ce46d8ec186acf47/log.txt.gz 2026-03-09 20:04:11,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:04:11,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:04:11,588 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69af2346ce46d8ec186acf47/lava-logs.yaml 2026-03-09 20:04:11,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:04:11,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:04:11,811 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69af2346ce46d8ec186acf47/lava_callback.json.gz 2026-03-09 20:04:11,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:04:11,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2346ce46d8ec186acf47 HTTP/1.1" 200 None 2026-03-09 20:04:11,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:04:11,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef42ace46d8ec186ab814 HTTP/1.1" 200 None 2026-03-09 20:04:11,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:04:11,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2346ce46d8ec186acf47 HTTP/1.1" 200 None 2026-03-09 20:04:12,106 - lava_callback - INFO - Completed processing callback for node 69af2346ce46d8ec186acf47 INFO: 172.21.0.1:55958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55974 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:04:17,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:04:17,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41008 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:15:11,316 - lava_callback - INFO - Received callback for node 69af2a34ce46d8ec186ad096 INFO: 172.21.0.1:39576 - "POST /node/69af2a34ce46d8ec186ad096 HTTP/1.0" 202 Accepted 2026-03-09 20:15:11,321 - lava_callback - INFO - Starting async job submit for node 69af2a34ce46d8ec186ad096 2026-03-09 20:15:11,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:15:11,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2a34ce46d8ec186ad096 HTTP/1.1" 200 None 2026-03-09 20:15:11,358 - lava_callback - DEBUG - Processing node 69af2a34ce46d8ec186ad096: baseline-arm64-broonie 2026-03-09 20:15:11,364 - lava_callback - WARNING - Failed to upload log 2026-03-09 20:15:11,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:15:11,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:15:11,498 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af2a34ce46d8ec186ad096/lava-logs.yaml 2026-03-09 20:15:11,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:15:11,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:15:11,663 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af2a34ce46d8ec186ad096/lava_callback.json.gz 2026-03-09 20:15:11,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:15:11,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2a34ce46d8ec186ad096 HTTP/1.1" 200 None 2026-03-09 20:15:11,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:15:11,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:15:11,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:15:11,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2a34ce46d8ec186ad096 HTTP/1.1" 200 None 2026-03-09 20:15:11,728 - lava_callback - INFO - Completed processing callback for node 69af2a34ce46d8ec186ad096 INFO: 172.21.0.1:39584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39598 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:15:17,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:15:17,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37134 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:17:49,685 - lava_callback - INFO - Received callback for node 69af2a95ce46d8ec186ad0a8 INFO: 172.21.0.1:37148 - "POST /node/69af2a95ce46d8ec186ad0a8 HTTP/1.0" 202 Accepted 2026-03-09 20:17:49,691 - lava_callback - INFO - Starting async job submit for node 69af2a95ce46d8ec186ad0a8 2026-03-09 20:17:49,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:17:49,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2a95ce46d8ec186ad0a8 HTTP/1.1" 200 None 2026-03-09 20:17:49,738 - lava_callback - DEBUG - Processing node 69af2a95ce46d8ec186ad0a8: baseline-arm64 2026-03-09 20:17:50,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:17:50,293 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:17:50,295 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2a95ce46d8ec186ad0a8/log.txt.gz 2026-03-09 20:17:50,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:17:50,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:17:50,793 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2a95ce46d8ec186ad0a8/lava-logs.yaml 2026-03-09 20:17:50,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:17:50,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:17:50,933 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2a95ce46d8ec186ad0a8/lava_callback.json.gz DEBUG: Setup failed for 69af2a95ce46d8ec186ad0a8. Transitting job node result: incomplete -> fail 2026-03-09 20:17:50,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:17:51,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2a95ce46d8ec186ad0a8 HTTP/1.1" 200 None 2026-03-09 20:17:51,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:17:51,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:17:51,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:17:51,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2a95ce46d8ec186ad0a8 HTTP/1.1" 200 None 2026-03-09 20:17:51,196 - lava_callback - INFO - Completed processing callback for node 69af2a95ce46d8ec186ad0a8 INFO: 172.21.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54916 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:18:17,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:18:17,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54836 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:19:14,230 - lava_callback - INFO - Received callback for node 69af2ae1ce46d8ec186ad0af INFO: 172.21.0.1:59274 - "POST /node/69af2ae1ce46d8ec186ad0af HTTP/1.0" 202 Accepted 2026-03-09 20:19:14,237 - lava_callback - INFO - Starting async job submit for node 69af2ae1ce46d8ec186ad0af 2026-03-09 20:19:14,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:19:14,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2ae1ce46d8ec186ad0af HTTP/1.1" 200 None 2026-03-09 20:19:14,286 - lava_callback - DEBUG - Processing node 69af2ae1ce46d8ec186ad0af: baseline-arm64 2026-03-09 20:19:15,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:59288 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-09 20:19:15,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:19:15,267 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2ae1ce46d8ec186ad0af/log.txt.gz 2026-03-09 20:19:15,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:19:16,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:19:16,018 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2ae1ce46d8ec186ad0af/lava-logs.yaml 2026-03-09 20:19:16,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:19:16,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:19:16,200 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2ae1ce46d8ec186ad0af/lava_callback.json.gz DEBUG: Setup failed for 69af2ae1ce46d8ec186ad0af. Transitting job node result: incomplete -> fail 2026-03-09 20:19:16,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:19:16,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2ae1ce46d8ec186ad0af HTTP/1.1" 200 None 2026-03-09 20:19:16,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:19:16,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:19:16,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:19:16,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2ae1ce46d8ec186ad0af HTTP/1.1" 200 None 2026-03-09 20:19:16,429 - lava_callback - INFO - Completed processing callback for node 69af2ae1ce46d8ec186ad0af INFO: 172.21.0.1:59294 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:19:17,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:19:17,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34112 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:20:36,205 - lava_callback - INFO - Received callback for node 69af2b32ce46d8ec186ad0bd INFO: 172.21.0.1:55984 - "POST /node/69af2b32ce46d8ec186ad0bd HTTP/1.0" 202 Accepted 2026-03-09 20:20:36,212 - lava_callback - INFO - Starting async job submit for node 69af2b32ce46d8ec186ad0bd 2026-03-09 20:20:36,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:20:36,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2b32ce46d8ec186ad0bd HTTP/1.1" 200 None 2026-03-09 20:20:36,257 - lava_callback - DEBUG - Processing node 69af2b32ce46d8ec186ad0bd: baseline-arm64 INFO: 172.21.0.1:56000 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:20:36,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:20:36,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:20:36,791 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b32ce46d8ec186ad0bd/log.txt.gz 2026-03-09 20:20:36,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:20:37,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:20:37,198 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b32ce46d8ec186ad0bd/lava-logs.yaml 2026-03-09 20:20:37,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:20:37,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:20:37,365 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b32ce46d8ec186ad0bd/lava_callback.json.gz DEBUG: Setup failed for 69af2b32ce46d8ec186ad0bd. Transitting job node result: incomplete -> fail 2026-03-09 20:20:37,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:20:37,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2b32ce46d8ec186ad0bd HTTP/1.1" 200 None 2026-03-09 20:20:37,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:20:37,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:20:37,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:20:37,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2b32ce46d8ec186ad0bd HTTP/1.1" 200 None 2026-03-09 20:20:37,580 - lava_callback - INFO - Completed processing callback for node 69af2b32ce46d8ec186ad0bd INFO: 172.21.0.1:57378 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:20:47,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:20:47,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50192 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:21:51,414 - lava_callback - INFO - Received callback for node 69af2b7fce46d8ec186ad0d0 INFO: 172.21.0.1:34600 - "POST /node/69af2b7fce46d8ec186ad0d0 HTTP/1.0" 202 Accepted 2026-03-09 20:21:51,420 - lava_callback - INFO - Starting async job submit for node 69af2b7fce46d8ec186ad0d0 2026-03-09 20:21:51,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:21:51,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2b7fce46d8ec186ad0d0 HTTP/1.1" 200 None 2026-03-09 20:21:51,466 - lava_callback - DEBUG - Processing node 69af2b7fce46d8ec186ad0d0: baseline-arm64 2026-03-09 20:21:51,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:21:51,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:21:51,935 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b7fce46d8ec186ad0d0/log.txt.gz 2026-03-09 20:21:52,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:21:52,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:21:52,354 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b7fce46d8ec186ad0d0/lava-logs.yaml 2026-03-09 20:21:52,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:21:52,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:21:52,517 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2b7fce46d8ec186ad0d0/lava_callback.json.gz DEBUG: Setup failed for 69af2b7fce46d8ec186ad0d0. Transitting job node result: incomplete -> fail 2026-03-09 20:21:52,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:21:52,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2b7fce46d8ec186ad0d0 HTTP/1.1" 200 None 2026-03-09 20:21:52,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:21:52,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:21:52,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:21:52,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2b7fce46d8ec186ad0d0 HTTP/1.1" 200 None 2026-03-09 20:21:52,722 - lava_callback - INFO - Completed processing callback for node 69af2b7fce46d8ec186ad0d0 INFO: 172.21.0.1:34602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43870 - "GET /node/699447d75ae459b9df3daaa4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:43874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43884 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:22:17,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:22:17,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56690 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:23:48,641 - lava_callback - INFO - Received callback for node 69af2851ce46d8ec186ad031 INFO: 172.21.0.1:56694 - "POST /node/69af2851ce46d8ec186ad031 HTTP/1.0" 202 Accepted 2026-03-09 20:23:48,648 - lava_callback - INFO - Starting async job submit for node 69af2851ce46d8ec186ad031 2026-03-09 20:23:48,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:23:48,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2851ce46d8ec186ad031 HTTP/1.1" 200 None 2026-03-09 20:23:48,719 - lava_callback - DEBUG - Processing node 69af2851ce46d8ec186ad031: baseline-arm 2026-03-09 20:23:49,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:23:49,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:23:49,638 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69af2851ce46d8ec186ad031/log.txt.gz 2026-03-09 20:23:49,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:23:50,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:23:50,265 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69af2851ce46d8ec186ad031/lava-logs.yaml 2026-03-09 20:23:50,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:23:50,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:23:50,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69af2851ce46d8ec186ad031/lava_callback.json.gz DEBUG: Setup failed for 69af2851ce46d8ec186ad031. Transitting job node result: incomplete -> fail 2026-03-09 20:23:50,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:23:50,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2851ce46d8ec186ad031 HTTP/1.1" 200 None 2026-03-09 20:23:50,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:23:50,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 20:23:50,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:23:50,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2851ce46d8ec186ad031 HTTP/1.1" 200 None 2026-03-09 20:23:50,728 - lava_callback - INFO - Completed processing callback for node 69af2851ce46d8ec186ad031 INFO: 172.21.0.1:50260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51966 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:24:17,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:17,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 20:24:23,693 - lava_callback - INFO - Received callback for node 69af2c15ce46d8ec186ad0f6 INFO: 172.21.0.1:43304 - "POST /node/69af2c15ce46d8ec186ad0f6 HTTP/1.0" 202 Accepted 2026-03-09 20:24:23,700 - lava_callback - INFO - Starting async job submit for node 69af2c15ce46d8ec186ad0f6 2026-03-09 20:24:23,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:23,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2c15ce46d8ec186ad0f6 HTTP/1.1" 200 None 2026-03-09 20:24:23,754 - lava_callback - DEBUG - Processing node 69af2c15ce46d8ec186ad0f6: baseline-arm64 2026-03-09 20:24:24,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:24:24,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:24:24,426 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2c15ce46d8ec186ad0f6/log.txt.gz 2026-03-09 20:24:24,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:24:24,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:24:24,934 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2c15ce46d8ec186ad0f6/lava-logs.yaml 2026-03-09 20:24:24,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:24:25,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:24:25,093 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2c15ce46d8ec186ad0f6/lava_callback.json.gz DEBUG: Setup failed for 69af2c15ce46d8ec186ad0f6. Transitting job node result: incomplete -> fail 2026-03-09 20:24:25,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:25,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2c15ce46d8ec186ad0f6 HTTP/1.1" 200 None 2026-03-09 20:24:25,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:25,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:24:25,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:25,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2c15ce46d8ec186ad0f6 HTTP/1.1" 200 None 2026-03-09 20:24:25,287 - lava_callback - INFO - Completed processing callback for node 69af2c15ce46d8ec186ad0f6 INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37716 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:24:47,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:24:47,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52730 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:26:23,933 - lava_callback - INFO - Received callback for node 69af2c8ace46d8ec186ad119 INFO: 172.21.0.1:46038 - "POST /node/69af2c8ace46d8ec186ad119 HTTP/1.0" 202 Accepted 2026-03-09 20:26:23,940 - lava_callback - INFO - Starting async job submit for node 69af2c8ace46d8ec186ad119 2026-03-09 20:26:23,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:26:23,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2c8ace46d8ec186ad119 HTTP/1.1" 200 None 2026-03-09 20:26:23,988 - lava_callback - DEBUG - Processing node 69af2c8ace46d8ec186ad119: baseline-nfs-arm64 2026-03-09 20:26:24,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:26:24,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:26:24,589 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2c8ace46d8ec186ad119/log.txt.gz 2026-03-09 20:26:24,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:26:25,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:26:25,090 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2c8ace46d8ec186ad119/lava-logs.yaml 2026-03-09 20:26:25,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:26:25,238 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:26:25,239 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2c8ace46d8ec186ad119/lava_callback.json.gz DEBUG: Setup failed for 69af2c8ace46d8ec186ad119. Transitting job node result: incomplete -> fail 2026-03-09 20:26:25,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:26:25,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2c8ace46d8ec186ad119 HTTP/1.1" 200 None 2026-03-09 20:26:25,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:26:25,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:26:25,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:26:25,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2c8ace46d8ec186ad119 HTTP/1.1" 200 None 2026-03-09 20:26:25,447 - lava_callback - INFO - Completed processing callback for node 69af2c8ace46d8ec186ad119 INFO: 172.21.0.1:46050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49206 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:26:47,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:26:47,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50990 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:28:17,034 - lava_callback - INFO - Received callback for node 69af2d0ece46d8ec186ad132 INFO: 172.21.0.1:50998 - "POST /node/69af2d0ece46d8ec186ad132 HTTP/1.0" 202 Accepted 2026-03-09 20:28:17,040 - lava_callback - INFO - Starting async job submit for node 69af2d0ece46d8ec186ad132 2026-03-09 20:28:17,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:28:17,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2d0ece46d8ec186ad132 HTTP/1.1" 200 None 2026-03-09 20:28:17,091 - lava_callback - DEBUG - Processing node 69af2d0ece46d8ec186ad132: baseline-arm64-foundriesio 2026-03-09 20:28:17,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:28:17,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:28:17,477 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af2d0ece46d8ec186ad132/log.txt.gz 2026-03-09 20:28:17,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:28:17,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:28:17,854 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af2d0ece46d8ec186ad132/lava-logs.yaml 2026-03-09 20:28:17,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:28:17,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:28:17,988 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69af2d0ece46d8ec186ad132/lava_callback.json.gz DEBUG: Setup failed for 69af2d0ece46d8ec186ad132. Transitting job node result: incomplete -> fail 2026-03-09 20:28:18,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:28:18,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2d0ece46d8ec186ad132 HTTP/1.1" 200 None 2026-03-09 20:28:18,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:28:18,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:28:18,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:28:18,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2d0ece46d8ec186ad132 HTTP/1.1" 200 None 2026-03-09 20:28:18,188 - lava_callback - INFO - Completed processing callback for node 69af2d0ece46d8ec186ad132 INFO: 172.21.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37194 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:28:47,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:28:47,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50428 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:31:08,815 - lava_callback - INFO - Received callback for node 69af2d99ce46d8ec186ad150 INFO: 172.21.0.1:50436 - "POST /node/69af2d99ce46d8ec186ad150 HTTP/1.0" 202 Accepted 2026-03-09 20:31:08,822 - lava_callback - INFO - Starting async job submit for node 69af2d99ce46d8ec186ad150 2026-03-09 20:31:08,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:31:08,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2d99ce46d8ec186ad150 HTTP/1.1" 200 None 2026-03-09 20:31:08,868 - lava_callback - DEBUG - Processing node 69af2d99ce46d8ec186ad150: baseline-nfs-arm64 2026-03-09 20:31:09,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:31:09,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:31:09,349 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2d99ce46d8ec186ad150/log.txt.gz 2026-03-09 20:31:09,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:31:09,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:31:09,787 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2d99ce46d8ec186ad150/lava-logs.yaml 2026-03-09 20:31:09,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:31:09,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:31:09,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2d99ce46d8ec186ad150/lava_callback.json.gz DEBUG: Setup failed for 69af2d99ce46d8ec186ad150. Transitting job node result: incomplete -> fail 2026-03-09 20:31:10,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:31:10,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2d99ce46d8ec186ad150 HTTP/1.1" 200 None 2026-03-09 20:31:10,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:31:10,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:31:10,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:31:10,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2d99ce46d8ec186ad150 HTTP/1.1" 200 None 2026-03-09 20:31:10,169 - lava_callback - INFO - Completed processing callback for node 69af2d99ce46d8ec186ad150 INFO: 172.21.0.1:51884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51888 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:31:17,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:31:17,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58488 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:32:23,426 - lava_callback - INFO - Received callback for node 69af2de3ce46d8ec186ad155 INFO: 172.21.0.1:59030 - "POST /node/69af2de3ce46d8ec186ad155 HTTP/1.0" 202 Accepted 2026-03-09 20:32:23,433 - lava_callback - INFO - Starting async job submit for node 69af2de3ce46d8ec186ad155 2026-03-09 20:32:23,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:32:23,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2de3ce46d8ec186ad155 HTTP/1.1" 200 None 2026-03-09 20:32:23,493 - lava_callback - DEBUG - Processing node 69af2de3ce46d8ec186ad155: baseline-nfs-arm64 2026-03-09 20:32:24,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:32:24,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:32:24,505 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2de3ce46d8ec186ad155/log.txt.gz 2026-03-09 20:32:24,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:32:25,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:32:25,211 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2de3ce46d8ec186ad155/lava-logs.yaml 2026-03-09 20:32:25,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:32:25,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:32:25,432 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2de3ce46d8ec186ad155/lava_callback.json.gz DEBUG: Setup failed for 69af2de3ce46d8ec186ad155. Transitting job node result: incomplete -> fail 2026-03-09 20:32:25,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:32:25,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2de3ce46d8ec186ad155 HTTP/1.1" 200 None 2026-03-09 20:32:25,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:32:25,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:32:25,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:32:25,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2de3ce46d8ec186ad155 HTTP/1.1" 200 None 2026-03-09 20:32:25,626 - lava_callback - INFO - Completed processing callback for node 69af2de3ce46d8ec186ad155 INFO: 172.21.0.1:59040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52968 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:32:47,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:32:47,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39016 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:34:10,907 - lava_callback - INFO - Received callback for node 69af2e5ace46d8ec186ad176 INFO: 172.21.0.1:56228 - "POST /node/69af2e5ace46d8ec186ad176 HTTP/1.0" 202 Accepted 2026-03-09 20:34:10,914 - lava_callback - INFO - Starting async job submit for node 69af2e5ace46d8ec186ad176 2026-03-09 20:34:10,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:34:10,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2e5ace46d8ec186ad176 HTTP/1.1" 200 None 2026-03-09 20:34:10,975 - lava_callback - DEBUG - Processing node 69af2e5ace46d8ec186ad176: baseline-nfs-arm64 2026-03-09 20:34:11,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:34:11,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:34:11,701 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2e5ace46d8ec186ad176/log.txt.gz 2026-03-09 20:34:12,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:34:12,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:34:12,274 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2e5ace46d8ec186ad176/lava-logs.yaml 2026-03-09 20:34:12,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:34:12,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:34:12,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2e5ace46d8ec186ad176/lava_callback.json.gz DEBUG: Setup failed for 69af2e5ace46d8ec186ad176. Transitting job node result: incomplete -> fail 2026-03-09 20:34:12,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:34:12,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2e5ace46d8ec186ad176 HTTP/1.1" 200 None 2026-03-09 20:34:12,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:34:12,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:34:12,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:34:12,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2e5ace46d8ec186ad176 HTTP/1.1" 200 None 2026-03-09 20:34:12,676 - lava_callback - INFO - Completed processing callback for node 69af2e5ace46d8ec186ad176 INFO: 172.21.0.1:56234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56248 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:34:17,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:34:17,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45702 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:36:10,742 - lava_callback - INFO - Received callback for node 69af2ecece46d8ec186ad193 INFO: 172.21.0.1:49606 - "POST /node/69af2ecece46d8ec186ad193 HTTP/1.0" 202 Accepted 2026-03-09 20:36:10,748 - lava_callback - INFO - Starting async job submit for node 69af2ecece46d8ec186ad193 2026-03-09 20:36:10,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:36:10,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2ecece46d8ec186ad193 HTTP/1.1" 200 None 2026-03-09 20:36:10,798 - lava_callback - DEBUG - Processing node 69af2ecece46d8ec186ad193: baseline-nfs-arm64 2026-03-09 20:36:11,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:36:11,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:36:11,304 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2ecece46d8ec186ad193/log.txt.gz 2026-03-09 20:36:11,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:36:11,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:36:11,727 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2ecece46d8ec186ad193/lava-logs.yaml 2026-03-09 20:36:11,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:36:11,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:36:11,871 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2ecece46d8ec186ad193/lava_callback.json.gz DEBUG: Setup failed for 69af2ecece46d8ec186ad193. Transitting job node result: incomplete -> fail 2026-03-09 20:36:11,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:36:11,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2ecece46d8ec186ad193 HTTP/1.1" 200 None 2026-03-09 20:36:11,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:36:11,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:36:11,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:36:11,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2ecece46d8ec186ad193 HTTP/1.1" 200 None 2026-03-09 20:36:12,068 - lava_callback - INFO - Completed processing callback for node 69af2ecece46d8ec186ad193 INFO: 172.21.0.1:49608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49620 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:36:17,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:36:17,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58650 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:37:08,751 - lava_callback - INFO - Received callback for node 69af2f1bce46d8ec186ad19a INFO: 172.21.0.1:58654 - "POST /node/69af2f1bce46d8ec186ad19a HTTP/1.0" 202 Accepted 2026-03-09 20:37:08,758 - lava_callback - INFO - Starting async job submit for node 69af2f1bce46d8ec186ad19a 2026-03-09 20:37:08,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:37:08,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2f1bce46d8ec186ad19a HTTP/1.1" 200 None 2026-03-09 20:37:08,808 - lava_callback - DEBUG - Processing node 69af2f1bce46d8ec186ad19a: baseline-nfs-arm64 2026-03-09 20:37:09,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:37:09,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:37:09,388 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2f1bce46d8ec186ad19a/log.txt.gz 2026-03-09 20:37:09,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:37:09,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:37:09,850 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2f1bce46d8ec186ad19a/lava-logs.yaml 2026-03-09 20:37:09,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:37:09,996 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:37:09,998 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af2f1bce46d8ec186ad19a/lava_callback.json.gz DEBUG: Setup failed for 69af2f1bce46d8ec186ad19a. Transitting job node result: incomplete -> fail 2026-03-09 20:37:10,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:37:10,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2f1bce46d8ec186ad19a HTTP/1.1" 200 None 2026-03-09 20:37:10,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:37:10,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:37:10,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:37:10,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2f1bce46d8ec186ad19a HTTP/1.1" 200 None 2026-03-09 20:37:10,227 - lava_callback - INFO - Completed processing callback for node 69af2f1bce46d8ec186ad19a INFO: 172.21.0.1:54114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54126 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:37:17,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:37:17,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47492 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:40:00,663 - lava_callback - INFO - Received callback for node 69af2f60ce46d8ec186ad1ab INFO: 172.21.0.1:35340 - "POST /node/69af2f60ce46d8ec186ad1ab HTTP/1.0" 202 Accepted 2026-03-09 20:40:00,669 - lava_callback - INFO - Starting async job submit for node 69af2f60ce46d8ec186ad1ab 2026-03-09 20:40:00,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:00,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2f60ce46d8ec186ad1ab HTTP/1.1" 200 None 2026-03-09 20:40:00,717 - lava_callback - DEBUG - Processing node 69af2f60ce46d8ec186ad1ab: baseline-arm64 2026-03-09 20:40:01,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:01,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:01,320 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2f60ce46d8ec186ad1ab/log.txt.gz 2026-03-09 20:40:01,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:01,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:01,772 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2f60ce46d8ec186ad1ab/lava-logs.yaml 2026-03-09 20:40:01,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:01,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:01,942 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2f60ce46d8ec186ad1ab/lava_callback.json.gz DEBUG: Setup failed for 69af2f60ce46d8ec186ad1ab. Transitting job node result: incomplete -> fail 2026-03-09 20:40:02,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:02,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2f60ce46d8ec186ad1ab HTTP/1.1" 200 None 2026-03-09 20:40:02,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:02,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:40:02,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:02,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2f60ce46d8ec186ad1ab HTTP/1.1" 200 None 2026-03-09 20:40:02,152 - lava_callback - INFO - Completed processing callback for node 69af2f60ce46d8ec186ad1ab INFO: 172.21.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48466 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:40:17,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:17,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45922 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:40:30,782 - lava_callback - INFO - Received callback for node 69af2fdace46d8ec186ad1c6 INFO: 172.21.0.1:59978 - "POST /node/69af2fdace46d8ec186ad1c6 HTTP/1.0" 202 Accepted 2026-03-09 20:40:30,788 - lava_callback - INFO - Starting async job submit for node 69af2fdace46d8ec186ad1c6 2026-03-09 20:40:30,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:30,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2fdace46d8ec186ad1c6 HTTP/1.1" 200 None 2026-03-09 20:40:30,845 - lava_callback - DEBUG - Processing node 69af2fdace46d8ec186ad1c6: baseline-arm64 2026-03-09 20:40:31,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:31,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:31,360 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2fdace46d8ec186ad1c6/log.txt.gz 2026-03-09 20:40:31,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:31,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:31,770 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2fdace46d8ec186ad1c6/lava-logs.yaml 2026-03-09 20:40:31,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:40:31,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:40:31,933 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af2fdace46d8ec186ad1c6/lava_callback.json.gz DEBUG: Setup failed for 69af2fdace46d8ec186ad1c6. Transitting job node result: incomplete -> fail 2026-03-09 20:40:32,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:32,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2fdace46d8ec186ad1c6 HTTP/1.1" 200 None 2026-03-09 20:40:32,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:32,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:40:32,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:32,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2fdace46d8ec186ad1c6 HTTP/1.1" 200 None 2026-03-09 20:40:32,146 - lava_callback - INFO - Completed processing callback for node 69af2fdace46d8ec186ad1c6 INFO: 172.21.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:40:47,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:40:47,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39284 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:43:37,198 - lava_callback - INFO - Received callback for node 69ac8de346f7ae8ff336685d INFO: 172.21.0.1:39292 - "POST /node/69ac8de346f7ae8ff336685d HTTP/1.0" 202 Accepted 2026-03-09 20:43:37,204 - lava_callback - INFO - Starting async job submit for node 69ac8de346f7ae8ff336685d 2026-03-09 20:43:37,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:37,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8de346f7ae8ff336685d HTTP/1.1" 200 None 2026-03-09 20:43:37,220 - lava_callback - DEBUG - Processing node 69ac8de346f7ae8ff336685d: baseline-arm-baylibre 2026-03-09 20:43:37,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8de346f7ae8ff336685d 2026-03-09 20:43:37,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:37,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8de346f7ae8ff336685d HTTP/1.1" 200 None 2026-03-09 20:43:37,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:37,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:43:37,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:37,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8de346f7ae8ff336685d HTTP/1.1" 200 None 2026-03-09 20:43:37,293 - lava_callback - ERROR - Error processing callback for node 69ac8de346f7ae8ff336685d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 20:43:45,989 - lava_callback - INFO - Received callback for node 69af302bce46d8ec186ad1d3 INFO: 172.21.0.1:54538 - "POST /node/69af302bce46d8ec186ad1d3 HTTP/1.0" 202 Accepted 2026-03-09 20:43:45,996 - lava_callback - INFO - Starting async job submit for node 69af302bce46d8ec186ad1d3 2026-03-09 20:43:46,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:46,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af302bce46d8ec186ad1d3 HTTP/1.1" 200 None 2026-03-09 20:43:46,048 - lava_callback - DEBUG - Processing node 69af302bce46d8ec186ad1d3: baseline-nfs-arm64 2026-03-09 20:43:46,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:54546 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:43:46,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:43:46,598 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af302bce46d8ec186ad1d3/log.txt.gz 2026-03-09 20:43:46,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:43:47,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:43:47,059 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af302bce46d8ec186ad1d3/lava-logs.yaml 2026-03-09 20:43:47,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:43:47,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:43:47,213 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af302bce46d8ec186ad1d3/lava_callback.json.gz DEBUG: Setup failed for 69af302bce46d8ec186ad1d3. Transitting job node result: incomplete -> fail 2026-03-09 20:43:47,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:47,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af302bce46d8ec186ad1d3 HTTP/1.1" 200 None 2026-03-09 20:43:47,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:47,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 20:43:47,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:43:47,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af302bce46d8ec186ad1d3 HTTP/1.1" 200 None 2026-03-09 20:43:47,410 - lava_callback - INFO - Completed processing callback for node 69af302bce46d8ec186ad1d3 INFO: 172.21.0.1:49942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59540 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:44:17,075 - lava_callback - INFO - Received callback for node 69ac8e1c46f7ae8ff3366880 INFO: 172.21.0.1:59548 - "POST /node/69ac8e1c46f7ae8ff3366880 HTTP/1.0" 202 Accepted 2026-03-09 20:44:17,081 - lava_callback - INFO - Starting async job submit for node 69ac8e1c46f7ae8ff3366880 2026-03-09 20:44:17,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:44:17,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8e1c46f7ae8ff3366880 HTTP/1.1" 200 None 2026-03-09 20:44:17,097 - lava_callback - DEBUG - Processing node 69ac8e1c46f7ae8ff3366880: baseline-arm-baylibre 2026-03-09 20:44:17,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8e1c46f7ae8ff3366880 2026-03-09 20:44:17,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:44:17,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8e1c46f7ae8ff3366880 HTTP/1.1" 200 None 2026-03-09 20:44:17,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:44:17,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:44:17,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:44:17,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8e1c46f7ae8ff3366880 HTTP/1.1" 200 None 2026-03-09 20:44:17,161 - lava_callback - ERROR - Error processing callback for node 69ac8e1c46f7ae8ff3366880: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-09 20:44:17,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:44:17,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38598 - "GET /node/69a75a595a0c0e10c481c6ff%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56782 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:45:57,249 - lava_callback - INFO - Received callback for node 69ac8e7c46f7ae8ff336689d INFO: 172.21.0.1:56794 - "POST /node/69ac8e7c46f7ae8ff336689d HTTP/1.0" 202 Accepted 2026-03-09 20:45:57,255 - lava_callback - INFO - Starting async job submit for node 69ac8e7c46f7ae8ff336689d 2026-03-09 20:45:57,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:45:57,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8e7c46f7ae8ff336689d HTTP/1.1" 200 None 2026-03-09 20:45:57,270 - lava_callback - DEBUG - Processing node 69ac8e7c46f7ae8ff336689d: baseline-arm-baylibre 2026-03-09 20:45:57,271 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8e7c46f7ae8ff336689d 2026-03-09 20:45:57,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:45:57,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8e7c46f7ae8ff336689d HTTP/1.1" 200 None 2026-03-09 20:45:57,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:45:57,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:45:57,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:45:57,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8e7c46f7ae8ff336689d HTTP/1.1" 200 None 2026-03-09 20:45:57,327 - lava_callback - ERROR - Error processing callback for node 69ac8e7c46f7ae8ff336689d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51654 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:46:10,421 - lava_callback - INFO - Received callback for node 69af313cce46d8ec186ad1fd INFO: 172.21.0.1:34022 - "POST /node/69af313cce46d8ec186ad1fd HTTP/1.0" 202 Accepted 2026-03-09 20:46:10,427 - lava_callback - INFO - Starting async job submit for node 69af313cce46d8ec186ad1fd 2026-03-09 20:46:10,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:10,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af313cce46d8ec186ad1fd HTTP/1.1" 200 None 2026-03-09 20:46:10,459 - lava_callback - DEBUG - Processing node 69af313cce46d8ec186ad1fd: baseline-x86 2026-03-09 20:46:10,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:46:10,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:46:10,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69af313cce46d8ec186ad1fd/log.txt.gz 2026-03-09 20:46:11,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:46:11,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:46:11,317 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69af313cce46d8ec186ad1fd/lava-logs.yaml 2026-03-09 20:46:11,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:46:11,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:46:11,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69af313cce46d8ec186ad1fd/lava_callback.json.gz 2026-03-09 20:46:11,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:11,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af313cce46d8ec186ad1fd HTTP/1.1" 200 None 2026-03-09 20:46:11,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:11,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae118d8ef4424e487b2ecc HTTP/1.1" 200 None 2026-03-09 20:46:11,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:11,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af313cce46d8ec186ad1fd HTTP/1.1" 200 None 2026-03-09 20:46:11,552 - lava_callback - INFO - Completed processing callback for node 69af313cce46d8ec186ad1fd INFO: 172.21.0.1:34024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34026 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:46:17,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:17,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50102 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:46:37,187 - lava_callback - INFO - Received callback for node 69ac8ea146f7ae8ff33668b0 INFO: 172.21.0.1:50110 - "POST /node/69ac8ea146f7ae8ff33668b0 HTTP/1.0" 202 Accepted 2026-03-09 20:46:37,192 - lava_callback - INFO - Starting async job submit for node 69ac8ea146f7ae8ff33668b0 2026-03-09 20:46:37,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:37,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ea146f7ae8ff33668b0 HTTP/1.1" 200 None 2026-03-09 20:46:37,209 - lava_callback - DEBUG - Processing node 69ac8ea146f7ae8ff33668b0: baseline-arm-baylibre 2026-03-09 20:46:37,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8ea146f7ae8ff33668b0 2026-03-09 20:46:37,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:37,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ea146f7ae8ff33668b0 HTTP/1.1" 200 None 2026-03-09 20:46:37,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:37,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:46:37,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:46:37,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8ea146f7ae8ff33668b0 HTTP/1.1" 200 None 2026-03-09 20:46:37,265 - lava_callback - ERROR - Error processing callback for node 69ac8ea146f7ae8ff33668b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59010 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:47:17,295 - lava_callback - INFO - Received callback for node 69ac8ec446f7ae8ff33668c8 INFO: 172.21.0.1:59012 - "POST /node/69ac8ec446f7ae8ff33668c8 HTTP/1.0" 202 Accepted 2026-03-09 20:47:17,300 - lava_callback - INFO - Starting async job submit for node 69ac8ec446f7ae8ff33668c8 2026-03-09 20:47:17,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:17,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ec446f7ae8ff33668c8 HTTP/1.1" 200 None 2026-03-09 20:47:17,316 - lava_callback - DEBUG - Processing node 69ac8ec446f7ae8ff33668c8: baseline-arm-baylibre 2026-03-09 20:47:17,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8ec446f7ae8ff33668c8 2026-03-09 20:47:17,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:17,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ec446f7ae8ff33668c8 HTTP/1.1" 200 None 2026-03-09 20:47:17,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:17,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:47:17,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:17,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8ec446f7ae8ff33668c8 HTTP/1.1" 200 None 2026-03-09 20:47:17,368 - lava_callback - ERROR - Error processing callback for node 69ac8ec446f7ae8ff33668c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52720 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:47:57,275 - lava_callback - INFO - Received callback for node 69ac8ee846f7ae8ff33668d8 INFO: 172.21.0.1:52726 - "POST /node/69ac8ee846f7ae8ff33668d8 HTTP/1.0" 202 Accepted 2026-03-09 20:47:57,280 - lava_callback - INFO - Starting async job submit for node 69ac8ee846f7ae8ff33668d8 2026-03-09 20:47:57,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:57,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ee846f7ae8ff33668d8 HTTP/1.1" 200 None 2026-03-09 20:47:57,306 - lava_callback - DEBUG - Processing node 69ac8ee846f7ae8ff33668d8: baseline-arm-baylibre 2026-03-09 20:47:57,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8ee846f7ae8ff33668d8 2026-03-09 20:47:57,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:57,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8ee846f7ae8ff33668d8 HTTP/1.1" 200 None 2026-03-09 20:47:57,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:57,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:47:57,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:47:57,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8ee846f7ae8ff33668d8 HTTP/1.1" 200 None 2026-03-09 20:47:57,373 - lava_callback - ERROR - Error processing callback for node 69ac8ee846f7ae8ff33668d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57924 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:48:17,284 - lava_callback - INFO - Received callback for node 69ac8f0c46f7ae8ff33668e7 INFO: 172.21.0.1:57926 - "POST /node/69ac8f0c46f7ae8ff33668e7 HTTP/1.0" 202 Accepted 2026-03-09 20:48:17,289 - lava_callback - INFO - Starting async job submit for node 69ac8f0c46f7ae8ff33668e7 2026-03-09 20:48:17,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:48:17,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f0c46f7ae8ff33668e7 HTTP/1.1" 200 None 2026-03-09 20:48:17,305 - lava_callback - DEBUG - Processing node 69ac8f0c46f7ae8ff33668e7: baseline-arm-baylibre 2026-03-09 20:48:17,305 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8f0c46f7ae8ff33668e7 2026-03-09 20:48:17,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:48:17,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f0c46f7ae8ff33668e7 HTTP/1.1" 200 None 2026-03-09 20:48:17,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:48:17,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:48:17,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:48:17,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8f0c46f7ae8ff33668e7 HTTP/1.1" 200 None 2026-03-09 20:48:17,363 - lava_callback - ERROR - Error processing callback for node 69ac8f0c46f7ae8ff33668e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45672 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:49:37,399 - lava_callback - INFO - Received callback for node 69ac8f4d46f7ae8ff3366906 INFO: 172.21.0.1:45682 - "POST /node/69ac8f4d46f7ae8ff3366906 HTTP/1.0" 202 Accepted 2026-03-09 20:49:37,405 - lava_callback - INFO - Starting async job submit for node 69ac8f4d46f7ae8ff3366906 2026-03-09 20:49:37,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:37,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f4d46f7ae8ff3366906 HTTP/1.1" 200 None 2026-03-09 20:49:37,430 - lava_callback - DEBUG - Processing node 69ac8f4d46f7ae8ff3366906: baseline-arm-baylibre 2026-03-09 20:49:37,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8f4d46f7ae8ff3366906 2026-03-09 20:49:37,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:37,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f4d46f7ae8ff3366906 HTTP/1.1" 200 None 2026-03-09 20:49:37,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:37,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:49:37,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:37,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8f4d46f7ae8ff3366906 HTTP/1.1" 200 None 2026-03-09 20:49:37,492 - lava_callback - ERROR - Error processing callback for node 69ac8f4d46f7ae8ff3366906: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51766 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:49:57,401 - lava_callback - INFO - Received callback for node 69ac8f7146f7ae8ff3366917 INFO: 172.21.0.1:51768 - "POST /node/69ac8f7146f7ae8ff3366917 HTTP/1.0" 202 Accepted 2026-03-09 20:49:57,407 - lava_callback - INFO - Starting async job submit for node 69ac8f7146f7ae8ff3366917 2026-03-09 20:49:57,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:57,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f7146f7ae8ff3366917 HTTP/1.1" 200 None 2026-03-09 20:49:57,424 - lava_callback - DEBUG - Processing node 69ac8f7146f7ae8ff3366917: baseline-arm-baylibre 2026-03-09 20:49:57,424 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8f7146f7ae8ff3366917 2026-03-09 20:49:57,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:57,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f7146f7ae8ff3366917 HTTP/1.1" 200 None 2026-03-09 20:49:57,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:57,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:49:57,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:49:57,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8f7146f7ae8ff3366917 HTTP/1.1" 200 None 2026-03-09 20:49:57,480 - lava_callback - ERROR - Error processing callback for node 69ac8f7146f7ae8ff3366917: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:50:37,412 - lava_callback - INFO - Received callback for node 69ac8f9646f7ae8ff336692e INFO: 172.21.0.1:47926 - "POST /node/69ac8f9646f7ae8ff336692e HTTP/1.0" 202 Accepted 2026-03-09 20:50:37,418 - lava_callback - INFO - Starting async job submit for node 69ac8f9646f7ae8ff336692e 2026-03-09 20:50:37,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:50:37,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f9646f7ae8ff336692e HTTP/1.1" 200 None 2026-03-09 20:50:37,464 - lava_callback - DEBUG - Processing node 69ac8f9646f7ae8ff336692e: baseline-arm-baylibre 2026-03-09 20:50:37,464 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8f9646f7ae8ff336692e 2026-03-09 20:50:37,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:50:37,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8f9646f7ae8ff336692e HTTP/1.1" 200 None 2026-03-09 20:50:37,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:50:37,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:50:37,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:50:37,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8f9646f7ae8ff336692e HTTP/1.1" 200 None 2026-03-09 20:50:37,516 - lava_callback - ERROR - Error processing callback for node 69ac8f9646f7ae8ff336692e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34804 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:51:17,479 - lava_callback - INFO - Received callback for node 69ac8fb946f7ae8ff3366940 INFO: 172.21.0.1:34816 - "POST /node/69ac8fb946f7ae8ff3366940 HTTP/1.0" 202 Accepted 2026-03-09 20:51:17,485 - lava_callback - INFO - Starting async job submit for node 69ac8fb946f7ae8ff3366940 2026-03-09 20:51:17,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:51:17,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8fb946f7ae8ff3366940 HTTP/1.1" 200 None 2026-03-09 20:51:17,501 - lava_callback - DEBUG - Processing node 69ac8fb946f7ae8ff3366940: baseline-arm-baylibre 2026-03-09 20:51:17,501 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac8fb946f7ae8ff3366940 2026-03-09 20:51:17,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:51:17,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac8fb946f7ae8ff3366940 HTTP/1.1" 200 None 2026-03-09 20:51:17,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:51:17,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:51:17,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:51:17,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac8fb946f7ae8ff3366940 HTTP/1.1" 200 None 2026-03-09 20:51:17,560 - lava_callback - ERROR - Error processing callback for node 69ac8fb946f7ae8ff3366940: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46748 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:52:57,558 - lava_callback - INFO - Received callback for node 69ac902146f7ae8ff3366970 INFO: 172.21.0.1:46760 - "POST /node/69ac902146f7ae8ff3366970 HTTP/1.0" 202 Accepted 2026-03-09 20:52:57,563 - lava_callback - INFO - Starting async job submit for node 69ac902146f7ae8ff3366970 2026-03-09 20:52:57,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:52:57,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac902146f7ae8ff3366970 HTTP/1.1" 200 None 2026-03-09 20:52:57,579 - lava_callback - DEBUG - Processing node 69ac902146f7ae8ff3366970: baseline-arm-baylibre 2026-03-09 20:52:57,579 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac902146f7ae8ff3366970 2026-03-09 20:52:57,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:52:57,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac902146f7ae8ff3366970 HTTP/1.1" 200 None 2026-03-09 20:52:57,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:52:57,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6ed9c2da7ccec10683e47 HTTP/1.1" 200 None 2026-03-09 20:52:57,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:52:57,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac902146f7ae8ff3366970 HTTP/1.1" 200 None 2026-03-09 20:52:57,643 - lava_callback - ERROR - Error processing callback for node 69ac902146f7ae8ff3366970: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48560 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:53:37,553 - lava_callback - INFO - Received callback for node 69ac904246f7ae8ff3366981 INFO: 172.21.0.1:48562 - "POST /node/69ac904246f7ae8ff3366981 HTTP/1.0" 202 Accepted 2026-03-09 20:53:37,559 - lava_callback - INFO - Starting async job submit for node 69ac904246f7ae8ff3366981 2026-03-09 20:53:37,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:53:37,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac904246f7ae8ff3366981 HTTP/1.1" 200 None 2026-03-09 20:53:37,574 - lava_callback - DEBUG - Processing node 69ac904246f7ae8ff3366981: baseline-arm-baylibre 2026-03-09 20:53:37,574 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac904246f7ae8ff3366981 2026-03-09 20:53:37,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:53:37,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac904246f7ae8ff3366981 HTTP/1.1" 200 None 2026-03-09 20:53:37,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:53:37,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:53:37,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:53:37,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac904246f7ae8ff3366981 HTTP/1.1" 200 None 2026-03-09 20:53:37,657 - lava_callback - ERROR - Error processing callback for node 69ac904246f7ae8ff3366981: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39286 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:54:17,626 - lava_callback - INFO - Received callback for node 69ac906646f7ae8ff3366996 INFO: 172.21.0.1:39302 - "POST /node/69ac906646f7ae8ff3366996 HTTP/1.0" 202 Accepted 2026-03-09 20:54:17,632 - lava_callback - INFO - Starting async job submit for node 69ac906646f7ae8ff3366996 2026-03-09 20:54:17,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:17,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac906646f7ae8ff3366996 HTTP/1.1" 200 None 2026-03-09 20:54:17,650 - lava_callback - DEBUG - Processing node 69ac906646f7ae8ff3366996: baseline-arm-baylibre 2026-03-09 20:54:17,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac906646f7ae8ff3366996 2026-03-09 20:54:17,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:17,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac906646f7ae8ff3366996 HTTP/1.1" 200 None 2026-03-09 20:54:17,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:17,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:54:17,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:17,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac906646f7ae8ff3366996 HTTP/1.1" 200 None 2026-03-09 20:54:17,773 - lava_callback - ERROR - Error processing callback for node 69ac906646f7ae8ff3366996: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44934 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:54:37,664 - lava_callback - INFO - Received callback for node 69ac908a46f7ae8ff33669ab INFO: 172.21.0.1:44938 - "POST /node/69ac908a46f7ae8ff33669ab HTTP/1.0" 202 Accepted 2026-03-09 20:54:37,670 - lava_callback - INFO - Starting async job submit for node 69ac908a46f7ae8ff33669ab 2026-03-09 20:54:37,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:37,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac908a46f7ae8ff33669ab HTTP/1.1" 200 None 2026-03-09 20:54:37,686 - lava_callback - DEBUG - Processing node 69ac908a46f7ae8ff33669ab: baseline-arm-baylibre 2026-03-09 20:54:37,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac908a46f7ae8ff33669ab 2026-03-09 20:54:37,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:37,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac908a46f7ae8ff33669ab HTTP/1.1" 200 None 2026-03-09 20:54:37,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:37,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:54:37,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:54:37,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac908a46f7ae8ff33669ab HTTP/1.1" 200 None 2026-03-09 20:54:37,749 - lava_callback - ERROR - Error processing callback for node 69ac908a46f7ae8ff33669ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42170 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:55:17,694 - lava_callback - INFO - Received callback for node 69ac90ad46f7ae8ff33669b9 INFO: 172.21.0.1:42180 - "POST /node/69ac90ad46f7ae8ff33669b9 HTTP/1.0" 202 Accepted 2026-03-09 20:55:17,700 - lava_callback - INFO - Starting async job submit for node 69ac90ad46f7ae8ff33669b9 2026-03-09 20:55:17,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:17,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac90ad46f7ae8ff33669b9 HTTP/1.1" 200 None 2026-03-09 20:55:17,723 - lava_callback - DEBUG - Processing node 69ac90ad46f7ae8ff33669b9: baseline-arm-baylibre 2026-03-09 20:55:17,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac90ad46f7ae8ff33669b9 2026-03-09 20:55:17,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:17,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac90ad46f7ae8ff33669b9 HTTP/1.1" 200 None 2026-03-09 20:55:17,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:17,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:55:17,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:17,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac90ad46f7ae8ff33669b9 HTTP/1.1" 200 None 2026-03-09 20:55:17,784 - lava_callback - ERROR - Error processing callback for node 69ac90ad46f7ae8ff33669b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:55:57,720 - lava_callback - INFO - Received callback for node 69ac90d246f7ae8ff33669d7 INFO: 172.21.0.1:48904 - "POST /node/69ac90d246f7ae8ff33669d7 HTTP/1.0" 202 Accepted 2026-03-09 20:55:57,726 - lava_callback - INFO - Starting async job submit for node 69ac90d246f7ae8ff33669d7 2026-03-09 20:55:57,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:57,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac90d246f7ae8ff33669d7 HTTP/1.1" 200 None 2026-03-09 20:55:57,751 - lava_callback - DEBUG - Processing node 69ac90d246f7ae8ff33669d7: baseline-arm-baylibre 2026-03-09 20:55:57,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac90d246f7ae8ff33669d7 2026-03-09 20:55:57,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:57,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac90d246f7ae8ff33669d7 HTTP/1.1" 200 None 2026-03-09 20:55:57,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:57,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:55:57,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:55:57,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac90d246f7ae8ff33669d7 HTTP/1.1" 200 None 2026-03-09 20:55:57,813 - lava_callback - ERROR - Error processing callback for node 69ac90d246f7ae8ff33669d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57944 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:57:50,406 - lava_callback - INFO - Received callback for node 69af2fb9ce46d8ec186ad1bf INFO: 172.21.0.1:45632 - "POST /node/69af2fb9ce46d8ec186ad1bf HTTP/1.0" 202 Accepted 2026-03-09 20:57:50,453 - lava_callback - INFO - Starting async job submit for node 69af2fb9ce46d8ec186ad1bf 2026-03-09 20:57:50,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:57:50,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2fb9ce46d8ec186ad1bf HTTP/1.1" 200 None 2026-03-09 20:57:50,492 - lava_callback - DEBUG - Processing node 69af2fb9ce46d8ec186ad1bf: baseline-arm-pengutronix 2026-03-09 20:57:50,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:57:50,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:57:50,747 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af2fb9ce46d8ec186ad1bf/log.txt.gz 2026-03-09 20:57:50,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:57:50,974 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:57:50,976 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af2fb9ce46d8ec186ad1bf/lava-logs.yaml 2026-03-09 20:57:50,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 20:57:51,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 20:57:51,121 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af2fb9ce46d8ec186ad1bf/lava_callback.json.gz DEBUG: Setup failed for 69af2fb9ce46d8ec186ad1bf. Transitting job node result: incomplete -> fail 2026-03-09 20:57:51,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:57:51,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2fb9ce46d8ec186ad1bf HTTP/1.1" 200 None 2026-03-09 20:57:51,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:57:51,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 20:57:51,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:57:51,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2fb9ce46d8ec186ad1bf HTTP/1.1" 200 None 2026-03-09 20:57:51,285 - lava_callback - INFO - Completed processing callback for node 69af2fb9ce46d8ec186ad1bf INFO: 172.21.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55298 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:58:17,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:17,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45786 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:58:28,813 - lava_callback - INFO - Received callback for node 69ac915f46f7ae8ff3366a29 INFO: 172.21.0.1:45792 - "POST /node/69ac915f46f7ae8ff3366a29 HTTP/1.0" 202 Accepted 2026-03-09 20:58:28,819 - lava_callback - INFO - Starting async job submit for node 69ac915f46f7ae8ff3366a29 2026-03-09 20:58:28,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:28,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac915f46f7ae8ff3366a29 HTTP/1.1" 200 None 2026-03-09 20:58:28,834 - lava_callback - DEBUG - Processing node 69ac915f46f7ae8ff3366a29: baseline-arm-clabbe 2026-03-09 20:58:28,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac915f46f7ae8ff3366a29 2026-03-09 20:58:28,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:28,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac915f46f7ae8ff3366a29 HTTP/1.1" 200 None 2026-03-09 20:58:28,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:28,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:58:28,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:28,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac915f46f7ae8ff3366a29 HTTP/1.1" 200 None 2026-03-09 20:58:28,912 - lava_callback - ERROR - Error processing callback for node 69ac915f46f7ae8ff3366a29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33666 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:58:48,592 - lava_callback - INFO - Received callback for node 69ac918446f7ae8ff3366a46 INFO: 172.21.0.1:33672 - "POST /node/69ac918446f7ae8ff3366a46 HTTP/1.0" 202 Accepted 2026-03-09 20:58:48,598 - lava_callback - INFO - Starting async job submit for node 69ac918446f7ae8ff3366a46 2026-03-09 20:58:48,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:48,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac918446f7ae8ff3366a46 HTTP/1.1" 200 None 2026-03-09 20:58:48,613 - lava_callback - DEBUG - Processing node 69ac918446f7ae8ff3366a46: baseline-arm-clabbe 2026-03-09 20:58:48,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac918446f7ae8ff3366a46 2026-03-09 20:58:48,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:48,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac918446f7ae8ff3366a46 HTTP/1.1" 200 None 2026-03-09 20:58:48,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:48,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:58:48,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:58:48,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac918446f7ae8ff3366a46 HTTP/1.1" 200 None 2026-03-09 20:58:48,670 - lava_callback - ERROR - Error processing callback for node 69ac918446f7ae8ff3366a46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45504 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:59:28,651 - lava_callback - INFO - Received callback for node 69ac91a846f7ae8ff3366a5e INFO: 172.21.0.1:45514 - "POST /node/69ac91a846f7ae8ff3366a5e HTTP/1.0" 202 Accepted 2026-03-09 20:59:28,657 - lava_callback - INFO - Starting async job submit for node 69ac91a846f7ae8ff3366a5e 2026-03-09 20:59:28,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:28,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac91a846f7ae8ff3366a5e HTTP/1.1" 200 None 2026-03-09 20:59:28,672 - lava_callback - DEBUG - Processing node 69ac91a846f7ae8ff3366a5e: baseline-arm-clabbe 2026-03-09 20:59:28,673 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac91a846f7ae8ff3366a5e 2026-03-09 20:59:28,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:28,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac91a846f7ae8ff3366a5e HTTP/1.1" 200 None 2026-03-09 20:59:28,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:28,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf2740deb775998f3ba4 HTTP/1.1" 200 None 2026-03-09 20:59:28,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:28,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac91a846f7ae8ff3366a5e HTTP/1.1" 200 None 2026-03-09 20:59:28,740 - lava_callback - ERROR - Error processing callback for node 69ac91a846f7ae8ff3366a5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49366 - "GET / HTTP/1.0" 200 OK 2026-03-09 20:59:58,698 - lava_callback - INFO - Received callback for node 69ac91c746f7ae8ff3366a72 INFO: 172.21.0.1:49370 - "POST /node/69ac91c746f7ae8ff3366a72 HTTP/1.0" 202 Accepted 2026-03-09 20:59:58,704 - lava_callback - INFO - Starting async job submit for node 69ac91c746f7ae8ff3366a72 2026-03-09 20:59:58,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:58,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac91c746f7ae8ff3366a72 HTTP/1.1" 200 None 2026-03-09 20:59:58,720 - lava_callback - DEBUG - Processing node 69ac91c746f7ae8ff3366a72: baseline-arm-baylibre 2026-03-09 20:59:58,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac91c746f7ae8ff3366a72 2026-03-09 20:59:58,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:58,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac91c746f7ae8ff3366a72 HTTP/1.1" 200 None 2026-03-09 20:59:58,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:58,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 20:59:58,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 20:59:58,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac91c746f7ae8ff3366a72 HTTP/1.1" 200 None 2026-03-09 20:59:58,786 - lava_callback - ERROR - Error processing callback for node 69ac91c746f7ae8ff3366a72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51416 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:00:11,349 - lava_callback - INFO - Received callback for node 69af31ecce46d8ec186ad220 INFO: 172.21.0.1:43222 - "POST /node/69af31ecce46d8ec186ad220 HTTP/1.0" 202 Accepted 2026-03-09 21:00:11,355 - lava_callback - INFO - Starting async job submit for node 69af31ecce46d8ec186ad220 2026-03-09 21:00:11,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:00:11,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af31ecce46d8ec186ad220 HTTP/1.1" 200 None 2026-03-09 21:00:11,404 - lava_callback - DEBUG - Processing node 69af31ecce46d8ec186ad220: baseline-arm64-broonie 2026-03-09 21:00:11,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:00:11,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:00:11,789 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af31ecce46d8ec186ad220/log.txt.gz 2026-03-09 21:00:11,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:00:12,169 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:00:12,171 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af31ecce46d8ec186ad220/lava-logs.yaml 2026-03-09 21:00:12,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:00:12,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:00:12,344 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af31ecce46d8ec186ad220/lava_callback.json.gz DEBUG: Setup failed for 69af31ecce46d8ec186ad220. Transitting job node result: incomplete -> fail 2026-03-09 21:00:12,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:00:12,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af31ecce46d8ec186ad220 HTTP/1.1" 200 None 2026-03-09 21:00:12,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:00:12,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:00:12,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:00:12,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af31ecce46d8ec186ad220 HTTP/1.1" 200 None 2026-03-09 21:00:12,577 - lava_callback - INFO - Completed processing callback for node 69af31ecce46d8ec186ad220 INFO: 172.21.0.1:43224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43232 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:00:17,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:00:17,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55180 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:05:08,821 - lava_callback - INFO - Received callback for node 69ac930146f7ae8ff3366b2c INFO: 172.21.0.1:55190 - "POST /node/69ac930146f7ae8ff3366b2c HTTP/1.0" 202 Accepted 2026-03-09 21:05:08,826 - lava_callback - INFO - Starting async job submit for node 69ac930146f7ae8ff3366b2c 2026-03-09 21:05:08,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:05:08,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac930146f7ae8ff3366b2c HTTP/1.1" 200 None 2026-03-09 21:05:08,842 - lava_callback - DEBUG - Processing node 69ac930146f7ae8ff3366b2c: baseline-arm-clabbe 2026-03-09 21:05:08,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac930146f7ae8ff3366b2c 2026-03-09 21:05:08,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:05:08,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac930146f7ae8ff3366b2c HTTP/1.1" 200 None 2026-03-09 21:05:08,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:05:08,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:05:08,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:05:08,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac930146f7ae8ff3366b2c HTTP/1.1" 200 None 2026-03-09 21:05:08,898 - lava_callback - ERROR - Error processing callback for node 69ac930146f7ae8ff3366b2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60792 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:07:08,983 - lava_callback - INFO - Received callback for node 69ac936b46f7ae8ff3366b6d INFO: 172.21.0.1:60802 - "POST /node/69ac936b46f7ae8ff3366b6d HTTP/1.0" 202 Accepted 2026-03-09 21:07:08,988 - lava_callback - INFO - Starting async job submit for node 69ac936b46f7ae8ff3366b6d 2026-03-09 21:07:08,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:09,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac936b46f7ae8ff3366b6d HTTP/1.1" 200 None 2026-03-09 21:07:09,008 - lava_callback - DEBUG - Processing node 69ac936b46f7ae8ff3366b6d: baseline-arm-clabbe 2026-03-09 21:07:09,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac936b46f7ae8ff3366b6d 2026-03-09 21:07:09,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:09,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac936b46f7ae8ff3366b6d HTTP/1.1" 200 None 2026-03-09 21:07:09,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:09,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:07:09,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:09,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac936b46f7ae8ff3366b6d HTTP/1.1" 200 None 2026-03-09 21:07:09,067 - lava_callback - ERROR - Error processing callback for node 69ac936b46f7ae8ff3366b6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42158 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:07:48,991 - lava_callback - INFO - Received callback for node 69ac938f46f7ae8ff3366b8c INFO: 172.21.0.1:42172 - "POST /node/69ac938f46f7ae8ff3366b8c HTTP/1.0" 202 Accepted 2026-03-09 21:07:48,996 - lava_callback - INFO - Starting async job submit for node 69ac938f46f7ae8ff3366b8c 2026-03-09 21:07:48,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:49,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac938f46f7ae8ff3366b8c HTTP/1.1" 200 None 2026-03-09 21:07:49,012 - lava_callback - DEBUG - Processing node 69ac938f46f7ae8ff3366b8c: baseline-arm-clabbe 2026-03-09 21:07:49,012 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac938f46f7ae8ff3366b8c 2026-03-09 21:07:49,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:49,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac938f46f7ae8ff3366b8c HTTP/1.1" 200 None 2026-03-09 21:07:49,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:49,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:07:49,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:07:49,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac938f46f7ae8ff3366b8c HTTP/1.1" 200 None 2026-03-09 21:07:49,065 - lava_callback - ERROR - Error processing callback for node 69ac938f46f7ae8ff3366b8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60518 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:08:09,040 - lava_callback - INFO - Received callback for node 69ac93b346f7ae8ff3366b97 INFO: 172.21.0.1:60526 - "POST /node/69ac93b346f7ae8ff3366b97 HTTP/1.0" 202 Accepted 2026-03-09 21:08:09,045 - lava_callback - INFO - Starting async job submit for node 69ac93b346f7ae8ff3366b97 2026-03-09 21:08:09,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:08:09,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac93b346f7ae8ff3366b97 HTTP/1.1" 200 None 2026-03-09 21:08:09,062 - lava_callback - DEBUG - Processing node 69ac93b346f7ae8ff3366b97: baseline-arm-clabbe 2026-03-09 21:08:09,062 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac93b346f7ae8ff3366b97 2026-03-09 21:08:09,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:08:09,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac93b346f7ae8ff3366b97 HTTP/1.1" 200 None 2026-03-09 21:08:09,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:08:09,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:08:09,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:08:09,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac93b346f7ae8ff3366b97 HTTP/1.1" 200 None 2026-03-09 21:08:09,140 - lava_callback - ERROR - Error processing callback for node 69ac93b346f7ae8ff3366b97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42108 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:09:29,092 - lava_callback - INFO - Received callback for node 69ac93fa46f7ae8ff3366bc0 INFO: 172.21.0.1:42116 - "POST /node/69ac93fa46f7ae8ff3366bc0 HTTP/1.0" 202 Accepted 2026-03-09 21:09:29,097 - lava_callback - INFO - Starting async job submit for node 69ac93fa46f7ae8ff3366bc0 2026-03-09 21:09:29,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:09:29,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac93fa46f7ae8ff3366bc0 HTTP/1.1" 200 None 2026-03-09 21:09:29,123 - lava_callback - DEBUG - Processing node 69ac93fa46f7ae8ff3366bc0: baseline-arm-clabbe 2026-03-09 21:09:29,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac93fa46f7ae8ff3366bc0 2026-03-09 21:09:29,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:09:29,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac93fa46f7ae8ff3366bc0 HTTP/1.1" 200 None 2026-03-09 21:09:29,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:09:29,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:09:29,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:09:29,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac93fa46f7ae8ff3366bc0 HTTP/1.1" 200 None 2026-03-09 21:09:29,180 - lava_callback - ERROR - Error processing callback for node 69ac93fa46f7ae8ff3366bc0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36914 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:10:09,030 - lava_callback - INFO - Received callback for node 69ac941e46f7ae8ff3366bdd INFO: 172.21.0.1:36920 - "POST /node/69ac941e46f7ae8ff3366bdd HTTP/1.0" 202 Accepted 2026-03-09 21:10:09,036 - lava_callback - INFO - Starting async job submit for node 69ac941e46f7ae8ff3366bdd 2026-03-09 21:10:09,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:10:09,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac941e46f7ae8ff3366bdd HTTP/1.1" 200 None 2026-03-09 21:10:09,061 - lava_callback - DEBUG - Processing node 69ac941e46f7ae8ff3366bdd: baseline-arm-clabbe 2026-03-09 21:10:09,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac941e46f7ae8ff3366bdd 2026-03-09 21:10:09,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:10:09,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac941e46f7ae8ff3366bdd HTTP/1.1" 200 None 2026-03-09 21:10:09,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:10:09,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c6a7ee78ae610caa67c HTTP/1.1" 200 None 2026-03-09 21:10:09,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:10:09,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac941e46f7ae8ff3366bdd HTTP/1.1" 200 None 2026-03-09 21:10:09,131 - lava_callback - ERROR - Error processing callback for node 69ac941e46f7ae8ff3366bdd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:14:18,844 - lava_callback - INFO - Received callback for node 69ac952046f7ae8ff3366c40 INFO: 172.21.0.1:34502 - "POST /node/69ac952046f7ae8ff3366c40 HTTP/1.0" 202 Accepted 2026-03-09 21:14:18,849 - lava_callback - INFO - Starting async job submit for node 69ac952046f7ae8ff3366c40 2026-03-09 21:14:18,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:14:18,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac952046f7ae8ff3366c40 HTTP/1.1" 200 None 2026-03-09 21:14:18,865 - lava_callback - DEBUG - Processing node 69ac952046f7ae8ff3366c40: baseline-arm64-baylibre 2026-03-09 21:14:18,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac952046f7ae8ff3366c40 2026-03-09 21:14:18,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:14:18,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac952046f7ae8ff3366c40 HTTP/1.1" 200 None 2026-03-09 21:14:18,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:14:18,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:14:18,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:14:18,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac952046f7ae8ff3366c40 HTTP/1.1" 200 None 2026-03-09 21:14:18,920 - lava_callback - ERROR - Error processing callback for node 69ac952046f7ae8ff3366c40: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32800 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:15:12,145 - lava_callback - INFO - Received callback for node 69af3678ce46d8ec186ad2d5 INFO: 172.21.0.1:41956 - "POST /node/69af3678ce46d8ec186ad2d5 HTTP/1.0" 202 Accepted 2026-03-09 21:15:12,151 - lava_callback - INFO - Starting async job submit for node 69af3678ce46d8ec186ad2d5 2026-03-09 21:15:12,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:12,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3678ce46d8ec186ad2d5 HTTP/1.1" 200 None 2026-03-09 21:15:12,210 - lava_callback - DEBUG - Processing node 69af3678ce46d8ec186ad2d5: baseline-arm64-broonie 2026-03-09 21:15:12,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:15:12,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:15:12,719 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3678ce46d8ec186ad2d5/log.txt.gz 2026-03-09 21:15:12,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:15:13,156 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:15:13,158 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3678ce46d8ec186ad2d5/lava-logs.yaml 2026-03-09 21:15:13,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:15:13,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:15:13,350 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3678ce46d8ec186ad2d5/lava_callback.json.gz DEBUG: Setup failed for 69af3678ce46d8ec186ad2d5. Transitting job node result: incomplete -> fail 2026-03-09 21:15:13,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:13,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3678ce46d8ec186ad2d5 HTTP/1.1" 200 None 2026-03-09 21:15:13,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:13,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:15:13,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:13,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3678ce46d8ec186ad2d5 HTTP/1.1" 200 None 2026-03-09 21:15:13,573 - lava_callback - INFO - Completed processing callback for node 69af3678ce46d8ec186ad2d5 INFO: 172.21.0.1:41964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41974 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:15:17,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:17,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56504 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:15:38,814 - lava_callback - INFO - Received callback for node 69ac956a46f7ae8ff3366c5b INFO: 172.21.0.1:56518 - "POST /node/69ac956a46f7ae8ff3366c5b HTTP/1.0" 202 Accepted 2026-03-09 21:15:38,820 - lava_callback - INFO - Starting async job submit for node 69ac956a46f7ae8ff3366c5b 2026-03-09 21:15:38,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:38,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac956a46f7ae8ff3366c5b HTTP/1.1" 200 None 2026-03-09 21:15:38,836 - lava_callback - DEBUG - Processing node 69ac956a46f7ae8ff3366c5b: baseline-arm64-baylibre 2026-03-09 21:15:38,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac956a46f7ae8ff3366c5b 2026-03-09 21:15:38,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:38,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac956a46f7ae8ff3366c5b HTTP/1.1" 200 None 2026-03-09 21:15:38,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:38,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:15:38,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:15:38,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac956a46f7ae8ff3366c5b HTTP/1.1" 200 None 2026-03-09 21:15:38,898 - lava_callback - ERROR - Error processing callback for node 69ac956a46f7ae8ff3366c5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:16:58,671 - lava_callback - INFO - Received callback for node 69af30c6ce46d8ec186ad1ee INFO: 172.21.0.1:56098 - "POST /node/69af30c6ce46d8ec186ad1ee HTTP/1.0" 202 Accepted 2026-03-09 21:16:58,677 - lava_callback - INFO - Starting async job submit for node 69af30c6ce46d8ec186ad1ee 2026-03-09 21:16:58,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:16:58,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af30c6ce46d8ec186ad1ee HTTP/1.1" 200 None 2026-03-09 21:16:58,728 - lava_callback - DEBUG - Processing node 69af30c6ce46d8ec186ad1ee: baseline-arm64-broonie 2026-03-09 21:16:59,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:16:59,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:16:59,367 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af30c6ce46d8ec186ad1ee/log.txt.gz 2026-03-09 21:16:59,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:16:59,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:16:59,846 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af30c6ce46d8ec186ad1ee/lava-logs.yaml 2026-03-09 21:16:59,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:38810 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:17:00,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:17:00,036 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af30c6ce46d8ec186ad1ee/lava_callback.json.gz DEBUG: Setup failed for 69af30c6ce46d8ec186ad1ee. Transitting job node result: incomplete -> fail 2026-03-09 21:17:00,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:00,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af30c6ce46d8ec186ad1ee HTTP/1.1" 200 None 2026-03-09 21:17:00,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:00,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-09 21:17:00,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:00,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af30c6ce46d8ec186ad1ee HTTP/1.1" 200 None 2026-03-09 21:17:00,234 - lava_callback - INFO - Completed processing callback for node 69af30c6ce46d8ec186ad1ee INFO: 172.21.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33652 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:17:17,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:17,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46620 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:17:38,487 - lava_callback - INFO - Received callback for node 69ae8934c660ead8a389aee2 INFO: 172.21.0.1:46622 - "POST /node/69ae8934c660ead8a389aee2 HTTP/1.0" 202 Accepted 2026-03-09 21:17:38,494 - lava_callback - INFO - Starting async job submit for node 69ae8934c660ead8a389aee2 2026-03-09 21:17:38,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:38,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8934c660ead8a389aee2 HTTP/1.1" 200 None 2026-03-09 21:17:38,533 - lava_callback - DEBUG - Processing node 69ae8934c660ead8a389aee2: baseline-arm-broonie 2026-03-09 21:17:38,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:17:38,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:17:38,909 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae8934c660ead8a389aee2/log.txt.gz 2026-03-09 21:17:39,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:17:39,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:17:39,229 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae8934c660ead8a389aee2/lava-logs.yaml 2026-03-09 21:17:39,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:17:39,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:17:39,402 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69ae8934c660ead8a389aee2/lava_callback.json.gz 2026-03-09 21:17:39,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:39,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8934c660ead8a389aee2 HTTP/1.1" 200 None 2026-03-09 21:17:39,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:39,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:17:39,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:39,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae8934c660ead8a389aee2 HTTP/1.1" 200 None 2026-03-09 21:17:39,486 - lava_callback - INFO - Completed processing callback for node 69ae8934c660ead8a389aee2 INFO: 172.21.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40712 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:40714 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:17:47,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:47,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36430 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:36444 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36460 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:36466 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:36468 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:17:58,908 - lava_callback - INFO - Received callback for node 69ac95fb46f7ae8ff3366c99 INFO: 172.21.0.1:36476 - "POST /node/69ac95fb46f7ae8ff3366c99 HTTP/1.0" 202 Accepted 2026-03-09 21:17:58,914 - lava_callback - INFO - Starting async job submit for node 69ac95fb46f7ae8ff3366c99 2026-03-09 21:17:58,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:58,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac95fb46f7ae8ff3366c99 HTTP/1.1" 200 None 2026-03-09 21:17:58,946 - lava_callback - DEBUG - Processing node 69ac95fb46f7ae8ff3366c99: baseline-arm64-baylibre 2026-03-09 21:17:58,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac95fb46f7ae8ff3366c99 2026-03-09 21:17:58,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:58,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac95fb46f7ae8ff3366c99 HTTP/1.1" 200 None 2026-03-09 21:17:58,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:58,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:17:58,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:17:58,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac95fb46f7ae8ff3366c99 HTTP/1.1" 200 None 2026-03-09 21:17:59,000 - lava_callback - ERROR - Error processing callback for node 69ac95fb46f7ae8ff3366c99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49616 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:19:18,979 - lava_callback - INFO - Received callback for node 69ac964446f7ae8ff3366cbc INFO: 172.21.0.1:49626 - "POST /node/69ac964446f7ae8ff3366cbc HTTP/1.0" 202 Accepted 2026-03-09 21:19:18,985 - lava_callback - INFO - Starting async job submit for node 69ac964446f7ae8ff3366cbc 2026-03-09 21:19:18,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:19,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac964446f7ae8ff3366cbc HTTP/1.1" 200 None 2026-03-09 21:19:19,001 - lava_callback - DEBUG - Processing node 69ac964446f7ae8ff3366cbc: baseline-arm64-baylibre 2026-03-09 21:19:19,002 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac964446f7ae8ff3366cbc 2026-03-09 21:19:19,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:19,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac964446f7ae8ff3366cbc HTTP/1.1" 200 None 2026-03-09 21:19:19,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:19,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-09 21:19:19,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:19,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac964446f7ae8ff3366cbc HTTP/1.1" 200 None 2026-03-09 21:19:19,058 - lava_callback - ERROR - Error processing callback for node 69ac964446f7ae8ff3366cbc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42358 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:19:38,985 - lava_callback - INFO - Received callback for node 69ae8963c660ead8a389af89 INFO: 172.21.0.1:42362 - "POST /node/69ae8963c660ead8a389af89 HTTP/1.0" 202 Accepted 2026-03-09 21:19:38,995 - lava_callback - INFO - Starting async job submit for node 69ae8963c660ead8a389af89 2026-03-09 21:19:39,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:39,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8963c660ead8a389af89 HTTP/1.1" 200 None 2026-03-09 21:19:39,046 - lava_callback - DEBUG - Processing node 69ae8963c660ead8a389af89: ltp-cap_bounds 2026-03-09 21:19:39,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:19:39,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:19:39,632 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69ae8963c660ead8a389af89/log.txt.gz 2026-03-09 21:19:39,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:19:40,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:19:40,103 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69ae8963c660ead8a389af89/lava-logs.yaml 2026-03-09 21:19:40,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:19:40,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:19:40,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69ae8963c660ead8a389af89/lava_callback.json.gz 2026-03-09 21:19:40,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:40,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8963c660ead8a389af89 HTTP/1.1" 200 None 2026-03-09 21:19:40,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:40,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:19:40,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:40,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae8963c660ead8a389af89 HTTP/1.1" 200 None 2026-03-09 21:19:40,365 - lava_callback - INFO - Completed processing callback for node 69ae8963c660ead8a389af89 INFO: 172.21.0.1:43170 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:19:47,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:19:47,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57824 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:20:19,005 - lava_callback - INFO - Received callback for node 69ac969046f7ae8ff3366cd2 INFO: 172.21.0.1:57832 - "POST /node/69ac969046f7ae8ff3366cd2 HTTP/1.0" 202 Accepted 2026-03-09 21:20:19,010 - lava_callback - INFO - Starting async job submit for node 69ac969046f7ae8ff3366cd2 2026-03-09 21:20:19,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:20:19,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac969046f7ae8ff3366cd2 HTTP/1.1" 200 None 2026-03-09 21:20:19,027 - lava_callback - DEBUG - Processing node 69ac969046f7ae8ff3366cd2: baseline-arm64-baylibre 2026-03-09 21:20:19,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac969046f7ae8ff3366cd2 2026-03-09 21:20:19,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:20:19,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac969046f7ae8ff3366cd2 HTTP/1.1" 200 None 2026-03-09 21:20:19,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:20:19,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:20:19,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:20:19,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac969046f7ae8ff3366cd2 HTTP/1.1" 200 None 2026-03-09 21:20:19,082 - lava_callback - ERROR - Error processing callback for node 69ac969046f7ae8ff3366cd2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50130 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:21:39,141 - lava_callback - INFO - Received callback for node 69ac96d946f7ae8ff3366cee INFO: 172.21.0.1:50138 - "POST /node/69ac96d946f7ae8ff3366cee HTTP/1.0" 202 Accepted 2026-03-09 21:21:39,147 - lava_callback - INFO - Starting async job submit for node 69ac96d946f7ae8ff3366cee 2026-03-09 21:21:39,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:21:39,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac96d946f7ae8ff3366cee HTTP/1.1" 200 None 2026-03-09 21:21:39,164 - lava_callback - DEBUG - Processing node 69ac96d946f7ae8ff3366cee: baseline-arm64-baylibre 2026-03-09 21:21:39,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac96d946f7ae8ff3366cee 2026-03-09 21:21:39,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:21:39,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac96d946f7ae8ff3366cee HTTP/1.1" 200 None 2026-03-09 21:21:39,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:21:39,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-09 21:21:39,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:21:39,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac96d946f7ae8ff3366cee HTTP/1.1" 200 None 2026-03-09 21:21:39,226 - lava_callback - ERROR - Error processing callback for node 69ac96d946f7ae8ff3366cee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54250 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:21:58,884 - lava_callback - INFO - Received callback for node 69ae8965c660ead8a389af8c INFO: 172.21.0.1:54260 - "POST /node/69ae8965c660ead8a389af8c HTTP/1.0" 202 Accepted 2026-03-09 21:21:58,890 - lava_callback - INFO - Starting async job submit for node 69ae8965c660ead8a389af8c 2026-03-09 21:21:58,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:21:58,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8965c660ead8a389af8c HTTP/1.1" 200 None 2026-03-09 21:21:58,930 - lava_callback - DEBUG - Processing node 69ae8965c660ead8a389af8c: ltp-containers 2026-03-09 21:21:59,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:21:59,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:21:59,507 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69ae8965c660ead8a389af8c/log.txt.gz 2026-03-09 21:21:59,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46244 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:21:59,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:21:59,943 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69ae8965c660ead8a389af8c/lava-logs.yaml 2026-03-09 21:21:59,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:22:00,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:22:00,117 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69ae8965c660ead8a389af8c/lava_callback.json.gz 2026-03-09 21:22:00,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:22:00,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8965c660ead8a389af8c HTTP/1.1" 200 None 2026-03-09 21:22:00,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:22:00,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:22:00,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:22:00,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae8965c660ead8a389af8c HTTP/1.1" 200 None 2026-03-09 21:22:00,206 - lava_callback - INFO - Completed processing callback for node 69ae8965c660ead8a389af8c INFO: 172.21.0.1:46254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46260 - "GET /node/69939574cebe99dd8e0743c0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57424 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:22:17,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:22:17,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41762 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:23:19,078 - lava_callback - INFO - Received callback for node 69af362bce46d8ec186ad2ce INFO: 172.21.0.1:41772 - "POST /node/69af362bce46d8ec186ad2ce HTTP/1.0" 202 Accepted 2026-03-09 21:23:19,085 - lava_callback - INFO - Starting async job submit for node 69af362bce46d8ec186ad2ce 2026-03-09 21:23:19,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:19,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af362bce46d8ec186ad2ce HTTP/1.1" 200 None 2026-03-09 21:23:19,136 - lava_callback - DEBUG - Processing node 69af362bce46d8ec186ad2ce: baseline-arm64-broonie 2026-03-09 21:23:19,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:19,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:19,702 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af362bce46d8ec186ad2ce/log.txt.gz 2026-03-09 21:23:19,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:20,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:20,155 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af362bce46d8ec186ad2ce/lava-logs.yaml 2026-03-09 21:23:20,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:20,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:20,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af362bce46d8ec186ad2ce/lava_callback.json.gz DEBUG: Setup failed for 69af362bce46d8ec186ad2ce. Transitting job node result: incomplete -> fail 2026-03-09 21:23:20,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:20,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af362bce46d8ec186ad2ce HTTP/1.1" 200 None 2026-03-09 21:23:20,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:20,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:23:20,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:20,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af362bce46d8ec186ad2ce HTTP/1.1" 200 None 2026-03-09 21:23:20,564 - lava_callback - INFO - Completed processing callback for node 69af362bce46d8ec186ad2ce INFO: 172.21.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49852 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:23:39,129 - lava_callback - INFO - Received callback for node 69ae8966c660ead8a389af8f INFO: 172.21.0.1:49860 - "POST /node/69ae8966c660ead8a389af8f HTTP/1.0" 202 Accepted 2026-03-09 21:23:39,135 - lava_callback - INFO - Starting async job submit for node 69ae8966c660ead8a389af8f 2026-03-09 21:23:39,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:39,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8966c660ead8a389af8f HTTP/1.1" 200 None 2026-03-09 21:23:39,174 - lava_callback - DEBUG - Processing node 69ae8966c660ead8a389af8f: ltp-controllers 2026-03-09 21:23:39,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:39,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:39,786 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69ae8966c660ead8a389af8f/log.txt.gz 2026-03-09 21:23:40,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:40,081 - lava_callback - INFO - Received callback for node 69af36c7ce46d8ec186ad2e3 INFO: 172.21.0.1:33440 - "POST /node/69af36c7ce46d8ec186ad2e3 HTTP/1.0" 202 Accepted 2026-03-09 21:23:40,087 - lava_callback - INFO - Starting async job submit for node 69af36c7ce46d8ec186ad2e3 2026-03-09 21:23:40,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af36c7ce46d8ec186ad2e3 HTTP/1.1" 200 None 2026-03-09 21:23:40,106 - lava_callback - DEBUG - Processing node 69af36c7ce46d8ec186ad2e3: baseline-arm64-broonie 2026-03-09 21:23:40,111 - lava_callback - WARNING - Failed to upload log 2026-03-09 21:23:40,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:40,295 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:40,298 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69ae8966c660ead8a389af8f/lava-logs.yaml 2026-03-09 21:23:40,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:40,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:40,430 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af36c7ce46d8ec186ad2e3/lava-logs.yaml 2026-03-09 21:23:40,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:23:40,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:40,479 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69ae8966c660ead8a389af8f/lava_callback.json.gz 2026-03-09 21:23:40,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8966c660ead8a389af8f HTTP/1.1" 200 None 2026-03-09 21:23:40,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:23:40,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae8966c660ead8a389af8f HTTP/1.1" 200 None 2026-03-09 21:23:40,588 - lava_callback - INFO - Completed processing callback for node 69ae8966c660ead8a389af8f 2026-03-09 21:23:40,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:23:40,600 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af36c7ce46d8ec186ad2e3/lava_callback.json.gz 2026-03-09 21:23:40,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af36c7ce46d8ec186ad2e3 HTTP/1.1" 200 None 2026-03-09 21:23:40,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:23:40,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:40,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af36c7ce46d8ec186ad2e3 HTTP/1.1" 200 None 2026-03-09 21:23:40,720 - lava_callback - INFO - Completed processing callback for node 69af36c7ce46d8ec186ad2e3 INFO: 172.21.0.1:33444 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:23:47,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:23:47,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51890 - "GET /node/6991f8e101ff1e8e0859de42%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:25:19,457 - lava_callback - INFO - Received callback for node 69ac97b646f7ae8ff3366d38 INFO: 172.21.0.1:52142 - "POST /node/69ac97b646f7ae8ff3366d38 HTTP/1.0" 202 Accepted 2026-03-09 21:25:19,463 - lava_callback - INFO - Starting async job submit for node 69ac97b646f7ae8ff3366d38 2026-03-09 21:25:19,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:25:19,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac97b646f7ae8ff3366d38 HTTP/1.1" 200 None 2026-03-09 21:25:19,479 - lava_callback - DEBUG - Processing node 69ac97b646f7ae8ff3366d38: baseline-arm64-baylibre 2026-03-09 21:25:19,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac97b646f7ae8ff3366d38 2026-03-09 21:25:19,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:25:19,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac97b646f7ae8ff3366d38 HTTP/1.1" 200 None 2026-03-09 21:25:19,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:25:19,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:25:19,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:25:19,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac97b646f7ae8ff3366d38 HTTP/1.1" 200 None 2026-03-09 21:25:19,534 - lava_callback - ERROR - Error processing callback for node 69ac97b646f7ae8ff3366d38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44248 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:26:02,456 - lava_callback - INFO - Received callback for node 69af3702ce46d8ec186ad2e8 INFO: 172.21.0.1:44258 - "POST /node/69af3702ce46d8ec186ad2e8 HTTP/1.0" 202 Accepted 2026-03-09 21:26:02,462 - lava_callback - INFO - Starting async job submit for node 69af3702ce46d8ec186ad2e8 2026-03-09 21:26:02,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:02,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3702ce46d8ec186ad2e8 HTTP/1.1" 200 None 2026-03-09 21:26:02,507 - lava_callback - DEBUG - Processing node 69af3702ce46d8ec186ad2e8: baseline-arm64 2026-03-09 21:26:02,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:02,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:02,962 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3702ce46d8ec186ad2e8/log.txt.gz 2026-03-09 21:26:03,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:03,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:03,342 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3702ce46d8ec186ad2e8/lava-logs.yaml 2026-03-09 21:26:03,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:03,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:03,482 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3702ce46d8ec186ad2e8/lava_callback.json.gz 2026-03-09 21:26:03,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:03,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3702ce46d8ec186ad2e8 HTTP/1.1" 200 None 2026-03-09 21:26:03,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:03,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:26:03,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:03,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3702ce46d8ec186ad2e8 HTTP/1.1" 200 None 2026-03-09 21:26:03,644 - lava_callback - INFO - Completed processing callback for node 69af3702ce46d8ec186ad2e8 INFO: 172.21.0.1:44272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55400 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:26:17,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:17,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-09 21:26:18,956 - lava_callback - INFO - Received callback for node 69ae8969c660ead8a389af92 INFO: 172.21.0.1:55406 - "POST /node/69ae8969c660ead8a389af92 HTTP/1.0" 202 Accepted 2026-03-09 21:26:18,963 - lava_callback - INFO - Starting async job submit for node 69ae8969c660ead8a389af92 2026-03-09 21:26:18,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:19,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8969c660ead8a389af92 HTTP/1.1" 200 None 2026-03-09 21:26:19,002 - lava_callback - DEBUG - Processing node 69ae8969c660ead8a389af92: ltp-cve 2026-03-09 21:26:19,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:19,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:19,538 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69ae8969c660ead8a389af92/log.txt.gz 2026-03-09 21:26:19,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:20,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:20,005 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69ae8969c660ead8a389af92/lava-logs.yaml 2026-03-09 21:26:20,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:26:20,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:26:20,189 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69ae8969c660ead8a389af92/lava_callback.json.gz 2026-03-09 21:26:20,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:20,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8969c660ead8a389af92 HTTP/1.1" 200 None 2026-03-09 21:26:20,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:20,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:26:20,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:20,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae8969c660ead8a389af92 HTTP/1.1" 200 None 2026-03-09 21:26:20,274 - lava_callback - INFO - Completed processing callback for node 69ae8969c660ead8a389af92 INFO: 172.21.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50464 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:26:47,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:26:47,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46548 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:27:08,348 - lava_callback - INFO - Received callback for node 69af383dce46d8ec186ad30e INFO: 172.21.0.1:46558 - "POST /node/69af383dce46d8ec186ad30e HTTP/1.0" 202 Accepted 2026-03-09 21:27:08,354 - lava_callback - INFO - Starting async job submit for node 69af383dce46d8ec186ad30e 2026-03-09 21:27:08,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:08,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af383dce46d8ec186ad30e HTTP/1.1" 200 None 2026-03-09 21:27:08,404 - lava_callback - DEBUG - Processing node 69af383dce46d8ec186ad30e: baseline-nfs-arm64 2026-03-09 21:27:08,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:08,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:08,923 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af383dce46d8ec186ad30e/log.txt.gz 2026-03-09 21:27:09,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:09,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:09,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af383dce46d8ec186ad30e/lava-logs.yaml 2026-03-09 21:27:09,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:09,484 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:09,486 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af383dce46d8ec186ad30e/lava_callback.json.gz 2026-03-09 21:27:09,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:09,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af383dce46d8ec186ad30e HTTP/1.1" 200 None 2026-03-09 21:27:09,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:09,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:27:09,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:09,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af383dce46d8ec186ad30e HTTP/1.1" 200 None 2026-03-09 21:27:09,651 - lava_callback - INFO - Completed processing callback for node 69af383dce46d8ec186ad30e INFO: 172.21.0.1:37548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37562 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:27:17,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:17,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58638 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:27:39,654 - lava_callback - INFO - Received callback for node 69af21d6ce46d8ec186acf0e INFO: 172.21.0.1:58654 - "POST /node/69af21d6ce46d8ec186acf0e HTTP/1.0" 202 Accepted 2026-03-09 21:27:39,660 - lava_callback - INFO - Starting async job submit for node 69af21d6ce46d8ec186acf0e 2026-03-09 21:27:39,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:39,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af21d6ce46d8ec186acf0e HTTP/1.1" 200 None 2026-03-09 21:27:39,712 - lava_callback - DEBUG - Processing node 69af21d6ce46d8ec186acf0e: baseline-arm64-broonie 2026-03-09 21:27:40,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:40,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:40,346 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af21d6ce46d8ec186acf0e/log.txt.gz 2026-03-09 21:27:40,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:40,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:40,785 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af21d6ce46d8ec186acf0e/lava-logs.yaml 2026-03-09 21:27:40,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:27:40,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:27:40,965 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af21d6ce46d8ec186acf0e/lava_callback.json.gz DEBUG: Setup failed for 69af21d6ce46d8ec186acf0e. Transitting job node result: incomplete -> fail 2026-03-09 21:27:41,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:41,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af21d6ce46d8ec186acf0e HTTP/1.1" 200 None 2026-03-09 21:27:41,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:41,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:27:41,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:41,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af21d6ce46d8ec186acf0e HTTP/1.1" 200 None 2026-03-09 21:27:41,165 - lava_callback - INFO - Completed processing callback for node 69af21d6ce46d8ec186acf0e INFO: 172.21.0.1:45526 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:27:47,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:27:47,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55832 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:28:59,850 - lava_callback - INFO - Received callback for node 69ac989046f7ae8ff3366d92 INFO: 172.21.0.1:48528 - "POST /node/69ac989046f7ae8ff3366d92 HTTP/1.0" 202 Accepted 2026-03-09 21:28:59,856 - lava_callback - INFO - Starting async job submit for node 69ac989046f7ae8ff3366d92 2026-03-09 21:28:59,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:28:59,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac989046f7ae8ff3366d92 HTTP/1.1" 200 None 2026-03-09 21:28:59,874 - lava_callback - DEBUG - Processing node 69ac989046f7ae8ff3366d92: baseline-arm64-baylibre 2026-03-09 21:28:59,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac989046f7ae8ff3366d92 2026-03-09 21:28:59,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:28:59,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac989046f7ae8ff3366d92 HTTP/1.1" 200 None 2026-03-09 21:28:59,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:28:59,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a744905a0c0e10c4819cd5 HTTP/1.1" 200 None 2026-03-09 21:28:59,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:28:59,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac989046f7ae8ff3366d92 HTTP/1.1" 200 None 2026-03-09 21:28:59,963 - lava_callback - ERROR - Error processing callback for node 69ac989046f7ae8ff3366d92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55154 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:30:19,626 - lava_callback - INFO - Received callback for node 69ac98d946f7ae8ff3366dae INFO: 172.21.0.1:55166 - "POST /node/69ac98d946f7ae8ff3366dae HTTP/1.0" 202 Accepted 2026-03-09 21:30:19,632 - lava_callback - INFO - Starting async job submit for node 69ac98d946f7ae8ff3366dae 2026-03-09 21:30:19,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:19,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac98d946f7ae8ff3366dae HTTP/1.1" 200 None 2026-03-09 21:30:19,657 - lava_callback - DEBUG - Processing node 69ac98d946f7ae8ff3366dae: baseline-arm64-baylibre 2026-03-09 21:30:19,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac98d946f7ae8ff3366dae 2026-03-09 21:30:19,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:19,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac98d946f7ae8ff3366dae HTTP/1.1" 200 None 2026-03-09 21:30:19,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:19,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-09 21:30:19,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:19,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac98d946f7ae8ff3366dae HTTP/1.1" 200 None 2026-03-09 21:30:19,723 - lava_callback - ERROR - Error processing callback for node 69ac98d946f7ae8ff3366dae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:30:39,642 - lava_callback - INFO - Received callback for node 69ae896cc660ead8a389af99 INFO: 172.21.0.1:52034 - "POST /node/69ae896cc660ead8a389af99 HTTP/1.0" 202 Accepted 2026-03-09 21:30:39,649 - lava_callback - INFO - Starting async job submit for node 69ae896cc660ead8a389af99 2026-03-09 21:30:39,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:39,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae896cc660ead8a389af99 HTTP/1.1" 200 None 2026-03-09 21:30:39,690 - lava_callback - DEBUG - Processing node 69ae896cc660ead8a389af99: ltp-fs_bind 2026-03-09 21:30:40,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:30:40,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:30:40,217 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69ae896cc660ead8a389af99/log.txt.gz 2026-03-09 21:30:40,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:30:40,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:30:40,677 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69ae896cc660ead8a389af99/lava-logs.yaml 2026-03-09 21:30:40,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:30:40,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:30:40,859 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69ae896cc660ead8a389af99/lava_callback.json.gz 2026-03-09 21:30:40,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:40,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae896cc660ead8a389af99 HTTP/1.1" 200 None 2026-03-09 21:30:40,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:40,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ae8223c660ead8a389aafd HTTP/1.1" 200 None 2026-03-09 21:30:40,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:40,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ae896cc660ead8a389af99 HTTP/1.1" 200 None 2026-03-09 21:30:40,952 - lava_callback - INFO - Completed processing callback for node 69ae896cc660ead8a389af99 INFO: 172.21.0.1:39198 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:30:47,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:30:47,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47766 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:31:30,102 - lava_callback - INFO - Received callback for node 69ac992846f7ae8ff3366dc7 INFO: 172.21.0.1:37974 - "POST /node/69ac992846f7ae8ff3366dc7 HTTP/1.0" 202 Accepted 2026-03-09 21:31:30,107 - lava_callback - INFO - Starting async job submit for node 69ac992846f7ae8ff3366dc7 2026-03-09 21:31:30,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:31:30,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac992846f7ae8ff3366dc7 HTTP/1.1" 200 None 2026-03-09 21:31:30,123 - lava_callback - DEBUG - Processing node 69ac992846f7ae8ff3366dc7: baseline-arm64-clabbe 2026-03-09 21:31:30,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac992846f7ae8ff3366dc7 2026-03-09 21:31:30,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:31:30,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac992846f7ae8ff3366dc7 HTTP/1.1" 200 None 2026-03-09 21:31:30,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:31:30,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-09 21:31:30,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:31:30,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac992846f7ae8ff3366dc7 HTTP/1.1" 200 None 2026-03-09 21:31:30,197 - lava_callback - ERROR - Error processing callback for node 69ac992846f7ae8ff3366dc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33496 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:33:50,162 - lava_callback - INFO - Received callback for node 69ac99bb46f7ae8ff3366e0f INFO: 172.21.0.1:39860 - "POST /node/69ac99bb46f7ae8ff3366e0f HTTP/1.0" 202 Accepted 2026-03-09 21:33:50,167 - lava_callback - INFO - Starting async job submit for node 69ac99bb46f7ae8ff3366e0f 2026-03-09 21:33:50,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:33:50,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac99bb46f7ae8ff3366e0f HTTP/1.1" 200 None 2026-03-09 21:33:50,188 - lava_callback - DEBUG - Processing node 69ac99bb46f7ae8ff3366e0f: baseline-arm64-clabbe 2026-03-09 21:33:50,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac99bb46f7ae8ff3366e0f 2026-03-09 21:33:50,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:33:50,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac99bb46f7ae8ff3366e0f HTTP/1.1" 200 None 2026-03-09 21:33:50,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:33:50,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69abdf3340deb775998f3bb5 HTTP/1.1" 200 None 2026-03-09 21:33:50,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:33:50,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac99bb46f7ae8ff3366e0f HTTP/1.1" 200 None 2026-03-09 21:33:50,250 - lava_callback - ERROR - Error processing callback for node 69ac99bb46f7ae8ff3366e0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58834 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:34:59,848 - lava_callback - INFO - Received callback for node 69ac99f346f7ae8ff3366e2c INFO: 172.21.0.1:48742 - "POST /node/69ac99f346f7ae8ff3366e2c HTTP/1.0" 202 Accepted 2026-03-09 21:34:59,854 - lava_callback - INFO - Starting async job submit for node 69ac99f346f7ae8ff3366e2c 2026-03-09 21:34:59,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:34:59,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac99f346f7ae8ff3366e2c HTTP/1.1" 200 None 2026-03-09 21:34:59,871 - lava_callback - DEBUG - Processing node 69ac99f346f7ae8ff3366e2c: baseline-arm-baylibre 2026-03-09 21:34:59,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac99f346f7ae8ff3366e2c 2026-03-09 21:34:59,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:34:59,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac99f346f7ae8ff3366e2c HTTP/1.1" 200 None 2026-03-09 21:34:59,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:34:59,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:34:59,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:34:59,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac99f346f7ae8ff3366e2c HTTP/1.1" 200 None 2026-03-09 21:34:59,945 - lava_callback - ERROR - Error processing callback for node 69ac99f346f7ae8ff3366e2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57108 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:35:19,861 - lava_callback - INFO - Received callback for node 69ac9a1646f7ae8ff3366e3f INFO: 172.21.0.1:55948 - "POST /node/69ac9a1646f7ae8ff3366e3f HTTP/1.0" 202 Accepted 2026-03-09 21:35:19,867 - lava_callback - INFO - Starting async job submit for node 69ac9a1646f7ae8ff3366e3f 2026-03-09 21:35:19,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:19,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a1646f7ae8ff3366e3f HTTP/1.1" 200 None 2026-03-09 21:35:19,883 - lava_callback - DEBUG - Processing node 69ac9a1646f7ae8ff3366e3f: baseline-arm-baylibre 2026-03-09 21:35:19,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9a1646f7ae8ff3366e3f 2026-03-09 21:35:19,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:19,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a1646f7ae8ff3366e3f HTTP/1.1" 200 None 2026-03-09 21:35:19,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:19,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:35:19,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:19,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9a1646f7ae8ff3366e3f HTTP/1.1" 200 None 2026-03-09 21:35:19,937 - lava_callback - ERROR - Error processing callback for node 69ac9a1646f7ae8ff3366e3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39914 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:35:59,896 - lava_callback - INFO - Received callback for node 69ac9a3946f7ae8ff3366e4f INFO: 172.21.0.1:40918 - "POST /node/69ac9a3946f7ae8ff3366e4f HTTP/1.0" 202 Accepted 2026-03-09 21:35:59,902 - lava_callback - INFO - Starting async job submit for node 69ac9a3946f7ae8ff3366e4f 2026-03-09 21:35:59,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:59,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a3946f7ae8ff3366e4f HTTP/1.1" 200 None 2026-03-09 21:35:59,919 - lava_callback - DEBUG - Processing node 69ac9a3946f7ae8ff3366e4f: baseline-arm-baylibre 2026-03-09 21:35:59,919 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9a3946f7ae8ff3366e4f 2026-03-09 21:35:59,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:59,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a3946f7ae8ff3366e4f HTTP/1.1" 200 None 2026-03-09 21:35:59,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:59,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:35:59,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:35:59,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9a3946f7ae8ff3366e4f HTTP/1.1" 200 None 2026-03-09 21:35:59,971 - lava_callback - ERROR - Error processing callback for node 69ac9a3946f7ae8ff3366e4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35458 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:36:39,955 - lava_callback - INFO - Received callback for node 69ac9a5e46f7ae8ff3366e60 INFO: 172.21.0.1:44474 - "POST /node/69ac9a5e46f7ae8ff3366e60 HTTP/1.0" 202 Accepted 2026-03-09 21:36:39,960 - lava_callback - INFO - Starting async job submit for node 69ac9a5e46f7ae8ff3366e60 2026-03-09 21:36:39,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:36:39,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a5e46f7ae8ff3366e60 HTTP/1.1" 200 None 2026-03-09 21:36:39,977 - lava_callback - DEBUG - Processing node 69ac9a5e46f7ae8ff3366e60: baseline-arm-baylibre 2026-03-09 21:36:39,977 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9a5e46f7ae8ff3366e60 2026-03-09 21:36:39,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:36:40,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a5e46f7ae8ff3366e60 HTTP/1.1" 200 None 2026-03-09 21:36:40,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:36:40,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:36:40,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:36:40,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9a5e46f7ae8ff3366e60 HTTP/1.1" 200 None 2026-03-09 21:36:40,054 - lava_callback - ERROR - Error processing callback for node 69ac9a5e46f7ae8ff3366e60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52460 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:37:19,976 - lava_callback - INFO - Received callback for node 69ac9a8546f7ae8ff3366e74 INFO: 172.21.0.1:52836 - "POST /node/69ac9a8546f7ae8ff3366e74 HTTP/1.0" 202 Accepted 2026-03-09 21:37:19,981 - lava_callback - INFO - Starting async job submit for node 69ac9a8546f7ae8ff3366e74 2026-03-09 21:37:19,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:37:19,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a8546f7ae8ff3366e74 HTTP/1.1" 200 None 2026-03-09 21:37:19,998 - lava_callback - DEBUG - Processing node 69ac9a8546f7ae8ff3366e74: baseline-arm-baylibre 2026-03-09 21:37:19,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9a8546f7ae8ff3366e74 2026-03-09 21:37:19,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:37:20,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9a8546f7ae8ff3366e74 HTTP/1.1" 200 None 2026-03-09 21:37:20,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:37:20,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:37:20,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:37:20,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9a8546f7ae8ff3366e74 HTTP/1.1" 200 None 2026-03-09 21:37:20,062 - lava_callback - ERROR - Error processing callback for node 69ac9a8546f7ae8ff3366e74: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41472 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:38:00,019 - lava_callback - INFO - Received callback for node 69ac9aaf46f7ae8ff3366e8d INFO: 172.21.0.1:42934 - "POST /node/69ac9aaf46f7ae8ff3366e8d HTTP/1.0" 202 Accepted 2026-03-09 21:38:00,025 - lava_callback - INFO - Starting async job submit for node 69ac9aaf46f7ae8ff3366e8d 2026-03-09 21:38:00,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:38:00,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9aaf46f7ae8ff3366e8d HTTP/1.1" 200 None 2026-03-09 21:38:00,042 - lava_callback - DEBUG - Processing node 69ac9aaf46f7ae8ff3366e8d: baseline-arm-baylibre 2026-03-09 21:38:00,042 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9aaf46f7ae8ff3366e8d 2026-03-09 21:38:00,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:38:00,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9aaf46f7ae8ff3366e8d HTTP/1.1" 200 None 2026-03-09 21:38:00,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:38:00,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7447b5a0c0e10c4819cc3 HTTP/1.1" 200 None 2026-03-09 21:38:00,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:38:00,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9aaf46f7ae8ff3366e8d HTTP/1.1" 200 None 2026-03-09 21:38:00,105 - lava_callback - ERROR - Error processing callback for node 69ac9aaf46f7ae8ff3366e8d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39218 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:39:00,055 - lava_callback - INFO - Received callback for node 69ac9adf46f7ae8ff3366eaa INFO: 172.21.0.1:38578 - "POST /node/69ac9adf46f7ae8ff3366eaa HTTP/1.0" 202 Accepted 2026-03-09 21:39:00,061 - lava_callback - INFO - Starting async job submit for node 69ac9adf46f7ae8ff3366eaa 2026-03-09 21:39:00,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:39:00,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9adf46f7ae8ff3366eaa HTTP/1.1" 200 None 2026-03-09 21:39:00,085 - lava_callback - DEBUG - Processing node 69ac9adf46f7ae8ff3366eaa: baseline-arm64-baylibre 2026-03-09 21:39:00,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9adf46f7ae8ff3366eaa 2026-03-09 21:39:00,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:39:00,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9adf46f7ae8ff3366eaa HTTP/1.1" 200 None 2026-03-09 21:39:00,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:39:00,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6edb32da7ccec10683eaa HTTP/1.1" 200 None 2026-03-09 21:39:00,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:39:00,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9adf46f7ae8ff3366eaa HTTP/1.1" 200 None 2026-03-09 21:39:00,144 - lava_callback - ERROR - Error processing callback for node 69ac9adf46f7ae8ff3366eaa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45550 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:41:20,244 - lava_callback - INFO - Received callback for node 69ac9b7646f7ae8ff3366ee3 INFO: 172.21.0.1:49166 - "POST /node/69ac9b7646f7ae8ff3366ee3 HTTP/1.0" 202 Accepted 2026-03-09 21:41:20,250 - lava_callback - INFO - Starting async job submit for node 69ac9b7646f7ae8ff3366ee3 2026-03-09 21:41:20,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:41:20,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9b7646f7ae8ff3366ee3 HTTP/1.1" 200 None 2026-03-09 21:41:20,265 - lava_callback - DEBUG - Processing node 69ac9b7646f7ae8ff3366ee3: baseline-arm64-baylibre 2026-03-09 21:41:20,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9b7646f7ae8ff3366ee3 2026-03-09 21:41:20,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:41:20,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9b7646f7ae8ff3366ee3 HTTP/1.1" 200 None 2026-03-09 21:41:20,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:41:20,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:41:20,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:41:20,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9b7646f7ae8ff3366ee3 HTTP/1.1" 200 None 2026-03-09 21:41:20,322 - lava_callback - ERROR - Error processing callback for node 69ac9b7646f7ae8ff3366ee3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35244 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:45:40,507 - lava_callback - INFO - Received callback for node 69ac9c7946f7ae8ff3366f3d INFO: 172.21.0.1:54016 - "POST /node/69ac9c7946f7ae8ff3366f3d HTTP/1.0" 202 Accepted 2026-03-09 21:45:40,513 - lava_callback - INFO - Starting async job submit for node 69ac9c7946f7ae8ff3366f3d 2026-03-09 21:45:40,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:45:40,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9c7946f7ae8ff3366f3d HTTP/1.1" 200 None 2026-03-09 21:45:40,529 - lava_callback - DEBUG - Processing node 69ac9c7946f7ae8ff3366f3d: baseline-arm64-baylibre 2026-03-09 21:45:40,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9c7946f7ae8ff3366f3d 2026-03-09 21:45:40,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:45:40,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9c7946f7ae8ff3366f3d HTTP/1.1" 200 None 2026-03-09 21:45:40,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:45:40,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:45:40,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:45:40,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9c7946f7ae8ff3366f3d HTTP/1.1" 200 None 2026-03-09 21:45:40,611 - lava_callback - ERROR - Error processing callback for node 69ac9c7946f7ae8ff3366f3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39696 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:48:00,703 - lava_callback - INFO - Received callback for node 69ac9d0d46f7ae8ff3366f6a INFO: 172.21.0.1:39712 - "POST /node/69ac9d0d46f7ae8ff3366f6a HTTP/1.0" 202 Accepted 2026-03-09 21:48:00,709 - lava_callback - INFO - Starting async job submit for node 69ac9d0d46f7ae8ff3366f6a 2026-03-09 21:48:00,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:48:00,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9d0d46f7ae8ff3366f6a HTTP/1.1" 200 None 2026-03-09 21:48:00,725 - lava_callback - DEBUG - Processing node 69ac9d0d46f7ae8ff3366f6a: baseline-arm64-baylibre 2026-03-09 21:48:00,726 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9d0d46f7ae8ff3366f6a 2026-03-09 21:48:00,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:48:00,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9d0d46f7ae8ff3366f6a HTTP/1.1" 200 None 2026-03-09 21:48:00,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:48:00,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6edb32da7ccec10683eaa HTTP/1.1" 200 None 2026-03-09 21:48:00,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:48:00,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9d0d46f7ae8ff3366f6a HTTP/1.1" 200 None 2026-03-09 21:48:00,795 - lava_callback - ERROR - Error processing callback for node 69ac9d0d46f7ae8ff3366f6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49472 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:49:33,803 - lava_callback - INFO - Received callback for node 69af2814ce46d8ec186ad02c INFO: 172.21.0.1:42572 - "POST /node/69af2814ce46d8ec186ad02c HTTP/1.0" 202 Accepted 2026-03-09 21:49:33,809 - lava_callback - INFO - Starting async job submit for node 69af2814ce46d8ec186ad02c 2026-03-09 21:49:33,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:49:33,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2814ce46d8ec186ad02c HTTP/1.1" 200 None 2026-03-09 21:49:33,860 - lava_callback - DEBUG - Processing node 69af2814ce46d8ec186ad02c: baseline-arm64-broonie 2026-03-09 21:49:34,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:49:34,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:49:34,244 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af2814ce46d8ec186ad02c/log.txt.gz 2026-03-09 21:49:34,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:49:34,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:49:34,609 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af2814ce46d8ec186ad02c/lava-logs.yaml 2026-03-09 21:49:34,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:49:34,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:49:34,750 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af2814ce46d8ec186ad02c/lava_callback.json.gz DEBUG: Setup failed for 69af2814ce46d8ec186ad02c. Transitting job node result: incomplete -> fail 2026-03-09 21:49:34,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:49:34,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af2814ce46d8ec186ad02c HTTP/1.1" 200 None 2026-03-09 21:49:34,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:49:34,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:49:34,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:49:34,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af2814ce46d8ec186ad02c HTTP/1.1" 200 None 2026-03-09 21:49:34,979 - lava_callback - INFO - Completed processing callback for node 69af2814ce46d8ec186ad02c INFO: 172.21.0.1:42578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55074 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:49:47,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:49:47,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38824 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:50:50,941 - lava_callback - INFO - Received callback for node 69ac9daa46f7ae8ff3366faf INFO: 172.21.0.1:58246 - "POST /node/69ac9daa46f7ae8ff3366faf HTTP/1.0" 202 Accepted 2026-03-09 21:50:50,947 - lava_callback - INFO - Starting async job submit for node 69ac9daa46f7ae8ff3366faf 2026-03-09 21:50:50,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:50:50,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9daa46f7ae8ff3366faf HTTP/1.1" 200 None 2026-03-09 21:50:50,963 - lava_callback - DEBUG - Processing node 69ac9daa46f7ae8ff3366faf: baseline-arm64-clabbe 2026-03-09 21:50:50,964 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9daa46f7ae8ff3366faf 2026-03-09 21:50:50,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:50:50,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9daa46f7ae8ff3366faf HTTP/1.1" 200 None 2026-03-09 21:50:50,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:50:50,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:50:50,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:50:51,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9daa46f7ae8ff3366faf HTTP/1.1" 200 None 2026-03-09 21:50:51,023 - lava_callback - ERROR - Error processing callback for node 69ac9daa46f7ae8ff3366faf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:50:59,904 - lava_callback - INFO - Received callback for node 69af3fadce46d8ec186ad486 INFO: 172.21.0.1:33766 - "POST /node/69af3fadce46d8ec186ad486 HTTP/1.0" 202 Accepted 2026-03-09 21:50:59,910 - lava_callback - INFO - Starting async job submit for node 69af3fadce46d8ec186ad486 2026-03-09 21:50:59,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:50:59,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3fadce46d8ec186ad486 HTTP/1.1" 200 None 2026-03-09 21:50:59,966 - lava_callback - DEBUG - Processing node 69af3fadce46d8ec186ad486: baseline-arm64-broonie INFO: 172.21.0.1:33768 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:51:00,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:51:00,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:51:00,585 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3fadce46d8ec186ad486/log.txt.gz 2026-03-09 21:51:00,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:51:01,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:51:01,076 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3fadce46d8ec186ad486/lava-logs.yaml 2026-03-09 21:51:01,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:51:01,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:51:01,250 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af3fadce46d8ec186ad486/lava_callback.json.gz DEBUG: Setup failed for 69af3fadce46d8ec186ad486. Transitting job node result: incomplete -> fail 2026-03-09 21:51:01,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:51:01,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3fadce46d8ec186ad486 HTTP/1.1" 200 None 2026-03-09 21:51:01,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:51:01,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:51:01,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:51:01,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3fadce46d8ec186ad486 HTTP/1.1" 200 None 2026-03-09 21:51:01,461 - lava_callback - INFO - Completed processing callback for node 69af3fadce46d8ec186ad486 INFO: 172.21.0.1:33774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52964 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:51:17,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:51:17,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37382 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:53:21,125 - lava_callback - INFO - Received callback for node 69ac9e4246f7ae8ff3366fdc INFO: 172.21.0.1:55568 - "POST /node/69ac9e4246f7ae8ff3366fdc HTTP/1.0" 202 Accepted 2026-03-09 21:53:21,131 - lava_callback - INFO - Starting async job submit for node 69ac9e4246f7ae8ff3366fdc 2026-03-09 21:53:21,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:53:21,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9e4246f7ae8ff3366fdc HTTP/1.1" 200 None 2026-03-09 21:53:21,147 - lava_callback - DEBUG - Processing node 69ac9e4246f7ae8ff3366fdc: baseline-arm64-baylibre 2026-03-09 21:53:21,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9e4246f7ae8ff3366fdc 2026-03-09 21:53:21,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:53:21,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9e4246f7ae8ff3366fdc HTTP/1.1" 200 None 2026-03-09 21:53:21,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:53:21,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:53:21,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:53:21,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9e4246f7ae8ff3366fdc HTTP/1.1" 200 None 2026-03-09 21:53:21,212 - lava_callback - ERROR - Error processing callback for node 69ac9e4246f7ae8ff3366fdc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:54:00,066 - lava_callback - INFO - Received callback for node 69af4048ce46d8ec186ad496 INFO: 172.21.0.1:38990 - "POST /node/69af4048ce46d8ec186ad496 HTTP/1.0" 202 Accepted 2026-03-09 21:54:00,073 - lava_callback - INFO - Starting async job submit for node 69af4048ce46d8ec186ad496 2026-03-09 21:54:00,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:00,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af4048ce46d8ec186ad496 HTTP/1.1" 200 None 2026-03-09 21:54:00,125 - lava_callback - DEBUG - Processing node 69af4048ce46d8ec186ad496: baseline-arm64-broonie INFO: 172.21.0.1:39004 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:54:00,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:00,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:00,692 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af4048ce46d8ec186ad496/log.txt.gz 2026-03-09 21:54:00,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:01,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:01,166 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af4048ce46d8ec186ad496/lava-logs.yaml 2026-03-09 21:54:01,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:01,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:01,340 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af4048ce46d8ec186ad496/lava_callback.json.gz DEBUG: Setup failed for 69af4048ce46d8ec186ad496. Transitting job node result: incomplete -> fail 2026-03-09 21:54:01,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:01,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af4048ce46d8ec186ad496 HTTP/1.1" 200 None 2026-03-09 21:54:01,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:01,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:54:01,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:01,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af4048ce46d8ec186ad496 HTTP/1.1" 200 None 2026-03-09 21:54:01,548 - lava_callback - INFO - Completed processing callback for node 69af4048ce46d8ec186ad496 INFO: 172.21.0.1:39010 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:54:13,730 - lava_callback - INFO - Received callback for node 69af3cf2ce46d8ec186ad412 INFO: 172.21.0.1:47542 - "POST /node/69af3cf2ce46d8ec186ad412 HTTP/1.0" 202 Accepted 2026-03-09 21:54:13,782 - lava_callback - INFO - Starting async job submit for node 69af3cf2ce46d8ec186ad412 2026-03-09 21:54:13,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:13,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3cf2ce46d8ec186ad412 HTTP/1.1" 200 None 2026-03-09 21:54:13,823 - lava_callback - DEBUG - Processing node 69af3cf2ce46d8ec186ad412: baseline-arm-pengutronix 2026-03-09 21:54:13,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:14,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:14,066 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af3cf2ce46d8ec186ad412/log.txt.gz 2026-03-09 21:54:14,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:14,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:14,284 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af3cf2ce46d8ec186ad412/lava-logs.yaml 2026-03-09 21:54:14,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:54:14,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:54:14,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69af3cf2ce46d8ec186ad412/lava_callback.json.gz DEBUG: Setup failed for 69af3cf2ce46d8ec186ad412. Transitting job node result: incomplete -> fail 2026-03-09 21:54:14,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:14,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3cf2ce46d8ec186ad412 HTTP/1.1" 200 None 2026-03-09 21:54:14,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:14,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-09 21:54:14,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:14,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3cf2ce46d8ec186ad412 HTTP/1.1" 200 None 2026-03-09 21:54:14,597 - lava_callback - INFO - Completed processing callback for node 69af3cf2ce46d8ec186ad412 INFO: 172.21.0.1:47556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47560 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:54:17,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:17,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43798 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:54:40,981 - lava_callback - INFO - Received callback for node 69ac9e9046f7ae8ff3366ffe INFO: 172.21.0.1:55660 - "POST /node/69ac9e9046f7ae8ff3366ffe HTTP/1.0" 202 Accepted 2026-03-09 21:54:40,987 - lava_callback - INFO - Starting async job submit for node 69ac9e9046f7ae8ff3366ffe 2026-03-09 21:54:40,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:41,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9e9046f7ae8ff3366ffe HTTP/1.1" 200 None 2026-03-09 21:54:41,003 - lava_callback - DEBUG - Processing node 69ac9e9046f7ae8ff3366ffe: baseline-arm64-baylibre 2026-03-09 21:54:41,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9e9046f7ae8ff3366ffe 2026-03-09 21:54:41,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:41,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9e9046f7ae8ff3366ffe HTTP/1.1" 200 None 2026-03-09 21:54:41,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:41,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6edb32da7ccec10683eaa HTTP/1.1" 200 None 2026-03-09 21:54:41,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:54:41,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9e9046f7ae8ff3366ffe HTTP/1.1" 200 None 2026-03-09 21:54:41,071 - lava_callback - ERROR - Error processing callback for node 69ac9e9046f7ae8ff3366ffe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35286 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:56:01,068 - lava_callback - INFO - Received callback for node 69ac9ee246f7ae8ff3367022 INFO: 172.21.0.1:35290 - "POST /node/69ac9ee246f7ae8ff3367022 HTTP/1.0" 202 Accepted 2026-03-09 21:56:01,074 - lava_callback - INFO - Starting async job submit for node 69ac9ee246f7ae8ff3367022 2026-03-09 21:56:01,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:56:01,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9ee246f7ae8ff3367022 HTTP/1.1" 200 None 2026-03-09 21:56:01,092 - lava_callback - DEBUG - Processing node 69ac9ee246f7ae8ff3367022: baseline-arm64-baylibre 2026-03-09 21:56:01,093 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9ee246f7ae8ff3367022 2026-03-09 21:56:01,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:56:01,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9ee246f7ae8ff3367022 HTTP/1.1" 200 None 2026-03-09 21:56:01,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:56:01,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:56:01,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:56:01,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9ee246f7ae8ff3367022 HTTP/1.1" 200 None 2026-03-09 21:56:01,157 - lava_callback - ERROR - Error processing callback for node 69ac9ee246f7ae8ff3367022: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45162 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:57:31,297 - lava_callback - INFO - Received callback for node 69ac9f3a46f7ae8ff3367044 INFO: 172.21.0.1:56290 - "POST /node/69ac9f3a46f7ae8ff3367044 HTTP/1.0" 202 Accepted 2026-03-09 21:57:31,302 - lava_callback - INFO - Starting async job submit for node 69ac9f3a46f7ae8ff3367044 2026-03-09 21:57:31,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:57:31,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9f3a46f7ae8ff3367044 HTTP/1.1" 200 None 2026-03-09 21:57:31,318 - lava_callback - DEBUG - Processing node 69ac9f3a46f7ae8ff3367044: baseline-arm64-clabbe 2026-03-09 21:57:31,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9f3a46f7ae8ff3367044 2026-03-09 21:57:31,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:57:31,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9f3a46f7ae8ff3367044 HTTP/1.1" 200 None 2026-03-09 21:57:31,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:57:31,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:57:31,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:57:31,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9f3a46f7ae8ff3367044 HTTP/1.1" 200 None 2026-03-09 21:57:31,378 - lava_callback - ERROR - Error processing callback for node 69ac9f3a46f7ae8ff3367044: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40666 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:58:51,305 - lava_callback - INFO - Received callback for node 69ac9f8746f7ae8ff3367067 INFO: 172.21.0.1:59462 - "POST /node/69ac9f8746f7ae8ff3367067 HTTP/1.0" 202 Accepted 2026-03-09 21:58:51,311 - lava_callback - INFO - Starting async job submit for node 69ac9f8746f7ae8ff3367067 2026-03-09 21:58:51,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:58:51,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9f8746f7ae8ff3367067 HTTP/1.1" 200 None 2026-03-09 21:58:51,345 - lava_callback - DEBUG - Processing node 69ac9f8746f7ae8ff3367067: baseline-arm64-clabbe 2026-03-09 21:58:51,345 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9f8746f7ae8ff3367067 2026-03-09 21:58:51,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:58:51,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9f8746f7ae8ff3367067 HTTP/1.1" 200 None 2026-03-09 21:58:51,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:58:51,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a93c777ee78ae610caa696 HTTP/1.1" 200 None 2026-03-09 21:58:51,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:58:51,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9f8746f7ae8ff3367067 HTTP/1.1" 200 None 2026-03-09 21:58:51,403 - lava_callback - ERROR - Error processing callback for node 69ac9f8746f7ae8ff3367067: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36256 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:59:41,254 - lava_callback - INFO - Received callback for node 69ac9fca46f7ae8ff336707e INFO: 172.21.0.1:33464 - "POST /node/69ac9fca46f7ae8ff336707e HTTP/1.0" 202 Accepted 2026-03-09 21:59:41,259 - lava_callback - INFO - Starting async job submit for node 69ac9fca46f7ae8ff336707e 2026-03-09 21:59:41,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:41,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9fca46f7ae8ff336707e HTTP/1.1" 200 None 2026-03-09 21:59:41,276 - lava_callback - DEBUG - Processing node 69ac9fca46f7ae8ff336707e: baseline-arm64-baylibre 2026-03-09 21:59:41,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ac9fca46f7ae8ff336707e 2026-03-09 21:59:41,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:41,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac9fca46f7ae8ff336707e HTTP/1.1" 200 None 2026-03-09 21:59:41,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:41,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a6edb32da7ccec10683eaa HTTP/1.1" 200 None 2026-03-09 21:59:41,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:41,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ac9fca46f7ae8ff336707e HTTP/1.1" 200 None 2026-03-09 21:59:41,360 - lava_callback - ERROR - Error processing callback for node 69ac9fca46f7ae8ff336707e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33480 - "GET / HTTP/1.0" 200 OK 2026-03-09 21:59:47,063 - lava_callback - INFO - Received callback for node 69af3751ce46d8ec186ad2f8 INFO: 172.21.0.1:33494 - "POST /node/69af3751ce46d8ec186ad2f8 HTTP/1.0" 202 Accepted 2026-03-09 21:59:47,070 - lava_callback - INFO - Starting async job submit for node 69af3751ce46d8ec186ad2f8 2026-03-09 21:59:47,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:47,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3751ce46d8ec186ad2f8 HTTP/1.1" 200 None 2026-03-09 21:59:47,110 - lava_callback - DEBUG - Processing node 69af3751ce46d8ec186ad2f8: baseline-arm64 2026-03-09 21:59:47,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:59:47,793 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:59:47,795 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3751ce46d8ec186ad2f8/log.txt.gz 2026-03-09 21:59:48,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:59:48,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:59:48,275 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3751ce46d8ec186ad2f8/lava-logs.yaml 2026-03-09 21:59:48,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 21:59:48,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 21:59:48,473 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69af3751ce46d8ec186ad2f8/lava_callback.json.gz 2026-03-09 21:59:48,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:48,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af3751ce46d8ec186ad2f8 HTTP/1.1" 200 None 2026-03-09 21:59:48,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:48,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 21:59:48,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 21:59:48,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af3751ce46d8ec186ad2f8 HTTP/1.1" 200 None 2026-03-09 21:59:48,594 - lava_callback - INFO - Completed processing callback for node 69af3751ce46d8ec186ad2f8 INFO: 172.21.0.1:36558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42606 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:00:17,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:00:17,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57922 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:00:55,407 - lava_callback - INFO - Received callback for node 69af42a5ce46d8ec186ad50d INFO: 172.21.0.1:54636 - "POST /node/69af42a5ce46d8ec186ad50d HTTP/1.0" 202 Accepted 2026-03-09 22:00:55,413 - lava_callback - INFO - Starting async job submit for node 69af42a5ce46d8ec186ad50d 2026-03-09 22:00:55,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:00:55,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af42a5ce46d8ec186ad50d HTTP/1.1" 200 None 2026-03-09 22:00:55,468 - lava_callback - DEBUG - Processing node 69af42a5ce46d8ec186ad50d: baseline-arm64-broonie 2026-03-09 22:00:55,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:00:55,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:00:55,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af42a5ce46d8ec186ad50d/log.txt.gz 2026-03-09 22:00:55,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:00:56,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:00:56,217 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af42a5ce46d8ec186ad50d/lava-logs.yaml 2026-03-09 22:00:56,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:00:56,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:00:56,378 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69af42a5ce46d8ec186ad50d/lava_callback.json.gz DEBUG: Setup failed for 69af42a5ce46d8ec186ad50d. Transitting job node result: incomplete -> fail 2026-03-09 22:00:56,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:00:56,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af42a5ce46d8ec186ad50d HTTP/1.1" 200 None 2026-03-09 22:00:56,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:00:56,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 22:00:56,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:00:56,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af42a5ce46d8ec186ad50d HTTP/1.1" 200 None INFO: 172.21.0.1:54638 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:00:56,593 - lava_callback - INFO - Completed processing callback for node 69af42a5ce46d8ec186ad50d INFO: 172.21.0.1:45560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53002 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:01:17,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:01:17,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44140 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:01:56,889 - lava_callback - INFO - Received callback for node 69af37a5ce46d8ec186ad2ff INFO: 172.21.0.1:44154 - "POST /node/69af37a5ce46d8ec186ad2ff HTTP/1.0" 202 Accepted 2026-03-09 22:01:56,896 - lava_callback - INFO - Starting async job submit for node 69af37a5ce46d8ec186ad2ff 2026-03-09 22:01:56,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:01:56,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af37a5ce46d8ec186ad2ff HTTP/1.1" 200 None 2026-03-09 22:01:56,962 - lava_callback - DEBUG - Processing node 69af37a5ce46d8ec186ad2ff: baseline-nfs-arm64 2026-03-09 22:01:57,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:01:57,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:01:57,629 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af37a5ce46d8ec186ad2ff/log.txt.gz 2026-03-09 22:01:57,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:01:58,133 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:01:58,135 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af37a5ce46d8ec186ad2ff/lava-logs.yaml 2026-03-09 22:01:58,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:01:58,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:01:58,337 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af37a5ce46d8ec186ad2ff/lava_callback.json.gz 2026-03-09 22:01:58,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:01:58,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af37a5ce46d8ec186ad2ff HTTP/1.1" 200 None 2026-03-09 22:01:58,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:01:58,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-09 22:01:58,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:01:58,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af37a5ce46d8ec186ad2ff HTTP/1.1" 200 None 2026-03-09 22:01:58,492 - lava_callback - INFO - Completed processing callback for node 69af37a5ce46d8ec186ad2ff INFO: 172.21.0.1:40492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56828 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:02:17,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:02:17,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:06:01,641 - lava_callback - INFO - Received callback for node 69aca14646f7ae8ff3367104 INFO: 172.21.0.1:40924 - "POST /node/69aca14646f7ae8ff3367104 HTTP/1.0" 202 Accepted 2026-03-09 22:06:01,646 - lava_callback - INFO - Starting async job submit for node 69aca14646f7ae8ff3367104 2026-03-09 22:06:01,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:01,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca14646f7ae8ff3367104 HTTP/1.1" 200 None 2026-03-09 22:06:01,692 - lava_callback - DEBUG - Processing node 69aca14646f7ae8ff3367104: baseline-arm-baylibre 2026-03-09 22:06:01,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca14646f7ae8ff3367104 2026-03-09 22:06:01,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:01,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca14646f7ae8ff3367104 HTTP/1.1" 200 None 2026-03-09 22:06:01,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:01,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:06:01,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:01,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca14646f7ae8ff3367104 HTTP/1.1" 200 None 2026-03-09 22:06:01,791 - lava_callback - ERROR - Error processing callback for node 69aca14646f7ae8ff3367104: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34300 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:06:41,616 - lava_callback - INFO - Received callback for node 69aca16a46f7ae8ff3367111 INFO: 172.21.0.1:57732 - "POST /node/69aca16a46f7ae8ff3367111 HTTP/1.0" 202 Accepted 2026-03-09 22:06:41,621 - lava_callback - INFO - Starting async job submit for node 69aca16a46f7ae8ff3367111 2026-03-09 22:06:41,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:41,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca16a46f7ae8ff3367111 HTTP/1.1" 200 None 2026-03-09 22:06:41,638 - lava_callback - DEBUG - Processing node 69aca16a46f7ae8ff3367111: baseline-arm-baylibre 2026-03-09 22:06:41,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca16a46f7ae8ff3367111 2026-03-09 22:06:41,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:41,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca16a46f7ae8ff3367111 HTTP/1.1" 200 None 2026-03-09 22:06:41,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:41,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:06:41,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:06:41,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca16a46f7ae8ff3367111 HTTP/1.1" 200 None 2026-03-09 22:06:41,696 - lava_callback - ERROR - Error processing callback for node 69aca16a46f7ae8ff3367111: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48340 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:07:21,720 - lava_callback - INFO - Received callback for node 69aca18e46f7ae8ff3367121 INFO: 172.21.0.1:41334 - "POST /node/69aca18e46f7ae8ff3367121 HTTP/1.0" 202 Accepted 2026-03-09 22:07:21,725 - lava_callback - INFO - Starting async job submit for node 69aca18e46f7ae8ff3367121 2026-03-09 22:07:21,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:07:21,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca18e46f7ae8ff3367121 HTTP/1.1" 200 None 2026-03-09 22:07:21,742 - lava_callback - DEBUG - Processing node 69aca18e46f7ae8ff3367121: baseline-arm-baylibre 2026-03-09 22:07:21,742 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca18e46f7ae8ff3367121 2026-03-09 22:07:21,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:07:21,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca18e46f7ae8ff3367121 HTTP/1.1" 200 None 2026-03-09 22:07:21,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:07:21,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:07:21,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:07:21,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca18e46f7ae8ff3367121 HTTP/1.1" 200 None 2026-03-09 22:07:21,792 - lava_callback - ERROR - Error processing callback for node 69aca18e46f7ae8ff3367121: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34436 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:08:01,682 - lava_callback - INFO - Received callback for node 69aca1b146f7ae8ff3367134 INFO: 172.21.0.1:34446 - "POST /node/69aca1b146f7ae8ff3367134 HTTP/1.0" 202 Accepted 2026-03-09 22:08:01,688 - lava_callback - INFO - Starting async job submit for node 69aca1b146f7ae8ff3367134 2026-03-09 22:08:01,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:01,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1b146f7ae8ff3367134 HTTP/1.1" 200 None 2026-03-09 22:08:01,704 - lava_callback - DEBUG - Processing node 69aca1b146f7ae8ff3367134: baseline-arm-baylibre 2026-03-09 22:08:01,704 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca1b146f7ae8ff3367134 2026-03-09 22:08:01,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:01,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1b146f7ae8ff3367134 HTTP/1.1" 200 None 2026-03-09 22:08:01,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:01,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:08:01,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:01,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca1b146f7ae8ff3367134 HTTP/1.1" 200 None 2026-03-09 22:08:01,761 - lava_callback - ERROR - Error processing callback for node 69aca1b146f7ae8ff3367134: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36784 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:08:41,764 - lava_callback - INFO - Received callback for node 69aca1d646f7ae8ff3367143 INFO: 172.21.0.1:57438 - "POST /node/69aca1d646f7ae8ff3367143 HTTP/1.0" 202 Accepted 2026-03-09 22:08:41,771 - lava_callback - INFO - Starting async job submit for node 69aca1d646f7ae8ff3367143 2026-03-09 22:08:41,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:41,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1d646f7ae8ff3367143 HTTP/1.1" 200 None 2026-03-09 22:08:41,840 - lava_callback - DEBUG - Processing node 69aca1d646f7ae8ff3367143: baseline-arm-baylibre 2026-03-09 22:08:41,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca1d646f7ae8ff3367143 2026-03-09 22:08:41,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:41,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1d646f7ae8ff3367143 HTTP/1.1" 200 None 2026-03-09 22:08:41,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:41,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:08:41,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:08:41,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca1d646f7ae8ff3367143 HTTP/1.1" 200 None 2026-03-09 22:08:41,893 - lava_callback - ERROR - Error processing callback for node 69aca1d646f7ae8ff3367143: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48630 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:09:01,752 - lava_callback - INFO - Received callback for node 69aca1fa46f7ae8ff3367158 INFO: 172.21.0.1:48642 - "POST /node/69aca1fa46f7ae8ff3367158 HTTP/1.0" 202 Accepted 2026-03-09 22:09:01,758 - lava_callback - INFO - Starting async job submit for node 69aca1fa46f7ae8ff3367158 2026-03-09 22:09:01,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:09:01,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1fa46f7ae8ff3367158 HTTP/1.1" 200 None 2026-03-09 22:09:01,782 - lava_callback - DEBUG - Processing node 69aca1fa46f7ae8ff3367158: baseline-arm-baylibre 2026-03-09 22:09:01,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca1fa46f7ae8ff3367158 2026-03-09 22:09:01,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:09:01,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca1fa46f7ae8ff3367158 HTTP/1.1" 200 None 2026-03-09 22:09:01,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:09:01,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750a55a0c0e10c481a794 HTTP/1.1" 200 None 2026-03-09 22:09:01,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:09:01,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca1fa46f7ae8ff3367158 HTTP/1.1" 200 None 2026-03-09 22:09:01,839 - lava_callback - ERROR - Error processing callback for node 69aca1fa46f7ae8ff3367158: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:21:31,967 - lava_callback - INFO - Received callback for node 69af454cce46d8ec186ad567 INFO: 172.21.0.1:52720 - "POST /node/69af454cce46d8ec186ad567 HTTP/1.0" 202 Accepted 2026-03-09 22:21:31,974 - lava_callback - INFO - Starting async job submit for node 69af454cce46d8ec186ad567 2026-03-09 22:21:32,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:21:32,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af454cce46d8ec186ad567 HTTP/1.1" 200 None 2026-03-09 22:21:32,063 - lava_callback - DEBUG - Processing node 69af454cce46d8ec186ad567: baseline-nfs-arm64 2026-03-09 22:21:32,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:21:32,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:21:32,523 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af454cce46d8ec186ad567/log.txt.gz 2026-03-09 22:21:32,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:21:32,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:21:32,910 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af454cce46d8ec186ad567/lava-logs.yaml 2026-03-09 22:21:32,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-09 22:21:33,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-09 22:21:33,073 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69af454cce46d8ec186ad567/lava_callback.json.gz 2026-03-09 22:21:33,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:21:33,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af454cce46d8ec186ad567 HTTP/1.1" 200 None 2026-03-09 22:21:33,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:21:33,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-09 22:21:33,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:21:33,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69af454cce46d8ec186ad567 HTTP/1.1" 200 None 2026-03-09 22:21:33,234 - lava_callback - INFO - Completed processing callback for node 69af454cce46d8ec186ad567 INFO: 172.21.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52238 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:21:47,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:21:47,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34426 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:38:23,439 - lava_callback - INFO - Received callback for node 69aca8da46f7ae8ff33672a9 INFO: 172.21.0.1:39580 - "POST /node/69aca8da46f7ae8ff33672a9 HTTP/1.0" 202 Accepted 2026-03-09 22:38:23,445 - lava_callback - INFO - Starting async job submit for node 69aca8da46f7ae8ff33672a9 2026-03-09 22:38:23,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:38:23,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca8da46f7ae8ff33672a9 HTTP/1.1" 200 None 2026-03-09 22:38:23,479 - lava_callback - DEBUG - Processing node 69aca8da46f7ae8ff33672a9: baseline-arm64-baylibre 2026-03-09 22:38:23,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca8da46f7ae8ff33672a9 2026-03-09 22:38:23,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:38:23,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca8da46f7ae8ff33672a9 HTTP/1.1" 200 None 2026-03-09 22:38:23,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:38:23,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:38:23,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:38:23,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca8da46f7ae8ff33672a9 HTTP/1.1" 200 None 2026-03-09 22:38:23,578 - lava_callback - ERROR - Error processing callback for node 69aca8da46f7ae8ff33672a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51966 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:39:43,574 - lava_callback - INFO - Received callback for node 69aca92a46f7ae8ff33672c2 INFO: 172.21.0.1:58722 - "POST /node/69aca92a46f7ae8ff33672c2 HTTP/1.0" 202 Accepted 2026-03-09 22:39:43,579 - lava_callback - INFO - Starting async job submit for node 69aca92a46f7ae8ff33672c2 2026-03-09 22:39:43,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:39:43,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca92a46f7ae8ff33672c2 HTTP/1.1" 200 None 2026-03-09 22:39:43,601 - lava_callback - DEBUG - Processing node 69aca92a46f7ae8ff33672c2: baseline-arm64-baylibre 2026-03-09 22:39:43,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca92a46f7ae8ff33672c2 2026-03-09 22:39:43,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:39:43,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca92a46f7ae8ff33672c2 HTTP/1.1" 200 None 2026-03-09 22:39:43,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:39:43,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:39:43,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:39:43,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca92a46f7ae8ff33672c2 HTTP/1.1" 200 None 2026-03-09 22:39:43,658 - lava_callback - ERROR - Error processing callback for node 69aca92a46f7ae8ff33672c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41732 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:42:23,692 - lava_callback - INFO - Received callback for node 69aca9ca46f7ae8ff3367318 INFO: 172.21.0.1:36422 - "POST /node/69aca9ca46f7ae8ff3367318 HTTP/1.0" 202 Accepted 2026-03-09 22:42:23,697 - lava_callback - INFO - Starting async job submit for node 69aca9ca46f7ae8ff3367318 2026-03-09 22:42:23,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:42:23,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca9ca46f7ae8ff3367318 HTTP/1.1" 200 None 2026-03-09 22:42:23,717 - lava_callback - DEBUG - Processing node 69aca9ca46f7ae8ff3367318: baseline-arm64-baylibre 2026-03-09 22:42:23,717 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aca9ca46f7ae8ff3367318 2026-03-09 22:42:23,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:42:23,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aca9ca46f7ae8ff3367318 HTTP/1.1" 200 None 2026-03-09 22:42:23,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:42:23,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:42:23,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:42:23,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aca9ca46f7ae8ff3367318 HTTP/1.1" 200 None 2026-03-09 22:42:23,768 - lava_callback - ERROR - Error processing callback for node 69aca9ca46f7ae8ff3367318: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40396 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:45:03,834 - lava_callback - INFO - Received callback for node 69acaa6246f7ae8ff336738c INFO: 172.21.0.1:40404 - "POST /node/69acaa6246f7ae8ff336738c HTTP/1.0" 202 Accepted 2026-03-09 22:45:03,839 - lava_callback - INFO - Starting async job submit for node 69acaa6246f7ae8ff336738c 2026-03-09 22:45:03,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:45:03,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acaa6246f7ae8ff336738c HTTP/1.1" 200 None 2026-03-09 22:45:03,865 - lava_callback - DEBUG - Processing node 69acaa6246f7ae8ff336738c: baseline-arm64-baylibre 2026-03-09 22:45:03,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acaa6246f7ae8ff336738c 2026-03-09 22:45:03,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:45:03,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acaa6246f7ae8ff336738c HTTP/1.1" 200 None 2026-03-09 22:45:03,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:45:03,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:45:03,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:45:03,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acaa6246f7ae8ff336738c HTTP/1.1" 200 None 2026-03-09 22:45:03,956 - lava_callback - ERROR - Error processing callback for node 69acaa6246f7ae8ff336738c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:53:33,898 - lava_callback - INFO - Received callback for node 69acac6b46f7ae8ff3367492 INFO: 172.21.0.1:43690 - "POST /node/69acac6b46f7ae8ff3367492 HTTP/1.0" 202 Accepted 2026-03-09 22:53:33,903 - lava_callback - INFO - Starting async job submit for node 69acac6b46f7ae8ff3367492 2026-03-09 22:53:33,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:53:33,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acac6b46f7ae8ff3367492 HTTP/1.1" 200 None 2026-03-09 22:53:33,925 - lava_callback - DEBUG - Processing node 69acac6b46f7ae8ff3367492: baseline-arm-clabbe 2026-03-09 22:53:33,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acac6b46f7ae8ff3367492 2026-03-09 22:53:33,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:53:33,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acac6b46f7ae8ff3367492 HTTP/1.1" 200 None 2026-03-09 22:53:33,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:53:33,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 22:53:33,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:53:33,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acac6b46f7ae8ff3367492 HTTP/1.1" 200 None 2026-03-09 22:53:33,985 - lava_callback - ERROR - Error processing callback for node 69acac6b46f7ae8ff3367492: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53870 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:54:53,832 - lava_callback - INFO - Received callback for node 69acacb546f7ae8ff33674ae INFO: 172.21.0.1:43370 - "POST /node/69acacb546f7ae8ff33674ae HTTP/1.0" 202 Accepted 2026-03-09 22:54:53,837 - lava_callback - INFO - Starting async job submit for node 69acacb546f7ae8ff33674ae 2026-03-09 22:54:53,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:54:53,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acacb546f7ae8ff33674ae HTTP/1.1" 200 None 2026-03-09 22:54:53,977 - lava_callback - DEBUG - Processing node 69acacb546f7ae8ff33674ae: baseline-arm-clabbe 2026-03-09 22:54:53,977 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acacb546f7ae8ff33674ae 2026-03-09 22:54:53,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:54:53,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acacb546f7ae8ff33674ae HTTP/1.1" 200 None 2026-03-09 22:54:53,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:54:54,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 22:54:54,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:54:54,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acacb546f7ae8ff33674ae HTTP/1.1" 200 None 2026-03-09 22:54:54,035 - lava_callback - ERROR - Error processing callback for node 69acacb546f7ae8ff33674ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46408 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:55:33,863 - lava_callback - INFO - Received callback for node 69acacdd46f7ae8ff33674c5 INFO: 172.21.0.1:34402 - "POST /node/69acacdd46f7ae8ff33674c5 HTTP/1.0" 202 Accepted 2026-03-09 22:55:33,868 - lava_callback - INFO - Starting async job submit for node 69acacdd46f7ae8ff33674c5 2026-03-09 22:55:33,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:55:33,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acacdd46f7ae8ff33674c5 HTTP/1.1" 200 None 2026-03-09 22:55:33,919 - lava_callback - DEBUG - Processing node 69acacdd46f7ae8ff33674c5: baseline-arm-clabbe 2026-03-09 22:55:33,919 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acacdd46f7ae8ff33674c5 2026-03-09 22:55:33,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:55:33,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acacdd46f7ae8ff33674c5 HTTP/1.1" 200 None 2026-03-09 22:55:33,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:55:33,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f6846f7ae8ff3364c2c HTTP/1.1" 200 None 2026-03-09 22:55:33,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:55:33,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acacdd46f7ae8ff33674c5 HTTP/1.1" 200 None 2026-03-09 22:55:33,970 - lava_callback - ERROR - Error processing callback for node 69acacdd46f7ae8ff33674c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34352 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:56:24,571 - lava_callback - INFO - Received callback for node 69acad0b46f7ae8ff33674e4 INFO: 172.21.0.1:33164 - "POST /node/69acad0b46f7ae8ff33674e4 HTTP/1.0" 202 Accepted 2026-03-09 22:56:24,576 - lava_callback - INFO - Starting async job submit for node 69acad0b46f7ae8ff33674e4 2026-03-09 22:56:24,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:56:24,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad0b46f7ae8ff33674e4 HTTP/1.1" 200 None 2026-03-09 22:56:24,596 - lava_callback - DEBUG - Processing node 69acad0b46f7ae8ff33674e4: baseline-arm64-baylibre 2026-03-09 22:56:24,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acad0b46f7ae8ff33674e4 2026-03-09 22:56:24,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:56:24,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad0b46f7ae8ff33674e4 HTTP/1.1" 200 None 2026-03-09 22:56:24,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:56:24,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:56:24,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:56:24,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acad0b46f7ae8ff33674e4 HTTP/1.1" 200 None 2026-03-09 22:56:24,652 - lava_callback - ERROR - Error processing callback for node 69acad0b46f7ae8ff33674e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40334 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:57:33,950 - lava_callback - INFO - Received callback for node 69acad5346f7ae8ff3367517 INFO: 172.21.0.1:42856 - "POST /node/69acad5346f7ae8ff3367517 HTTP/1.0" 202 Accepted 2026-03-09 22:57:33,955 - lava_callback - INFO - Starting async job submit for node 69acad5346f7ae8ff3367517 2026-03-09 22:57:33,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:57:33,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad5346f7ae8ff3367517 HTTP/1.1" 200 None 2026-03-09 22:57:33,971 - lava_callback - DEBUG - Processing node 69acad5346f7ae8ff3367517: baseline-arm-clabbe 2026-03-09 22:57:33,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acad5346f7ae8ff3367517 2026-03-09 22:57:33,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:57:33,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad5346f7ae8ff3367517 HTTP/1.1" 200 None 2026-03-09 22:57:33,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:57:34,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 22:57:34,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:57:34,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acad5346f7ae8ff3367517 HTTP/1.1" 200 None 2026-03-09 22:57:34,023 - lava_callback - ERROR - Error processing callback for node 69acad5346f7ae8ff3367517: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56426 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:58:24,490 - lava_callback - INFO - Received callback for node 69acad8146f7ae8ff336752c INFO: 172.21.0.1:56406 - "POST /node/69acad8146f7ae8ff336752c HTTP/1.0" 202 Accepted 2026-03-09 22:58:24,496 - lava_callback - INFO - Starting async job submit for node 69acad8146f7ae8ff336752c 2026-03-09 22:58:24,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:58:24,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad8146f7ae8ff336752c HTTP/1.1" 200 None 2026-03-09 22:58:24,511 - lava_callback - DEBUG - Processing node 69acad8146f7ae8ff336752c: baseline-arm64-baylibre 2026-03-09 22:58:24,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acad8146f7ae8ff336752c 2026-03-09 22:58:24,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:58:24,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acad8146f7ae8ff336752c HTTP/1.1" 200 None 2026-03-09 22:58:24,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:58:24,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a750ca5a0c0e10c481a7c4 HTTP/1.1" 200 None 2026-03-09 22:58:24,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:58:24,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acad8146f7ae8ff336752c HTTP/1.1" 200 None 2026-03-09 22:58:24,582 - lava_callback - ERROR - Error processing callback for node 69acad8146f7ae8ff336752c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56304 - "GET / HTTP/1.0" 200 OK 2026-03-09 22:59:34,067 - lava_callback - INFO - Received callback for node 69acadc646f7ae8ff3367550 INFO: 172.21.0.1:59832 - "POST /node/69acadc646f7ae8ff3367550 HTTP/1.0" 202 Accepted 2026-03-09 22:59:34,073 - lava_callback - INFO - Starting async job submit for node 69acadc646f7ae8ff3367550 2026-03-09 22:59:34,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:59:34,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acadc646f7ae8ff3367550 HTTP/1.1" 200 None 2026-03-09 22:59:34,089 - lava_callback - DEBUG - Processing node 69acadc646f7ae8ff3367550: baseline-arm-clabbe 2026-03-09 22:59:34,089 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acadc646f7ae8ff3367550 2026-03-09 22:59:34,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:59:34,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acadc646f7ae8ff3367550 HTTP/1.1" 200 None 2026-03-09 22:59:34,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:59:34,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 22:59:34,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 22:59:34,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acadc646f7ae8ff3367550 HTTP/1.1" 200 None 2026-03-09 22:59:34,148 - lava_callback - ERROR - Error processing callback for node 69acadc646f7ae8ff3367550: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44574 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:00:14,272 - lava_callback - INFO - Received callback for node 69acadeb46f7ae8ff3367569 INFO: 172.21.0.1:41492 - "POST /node/69acadeb46f7ae8ff3367569 HTTP/1.0" 202 Accepted 2026-03-09 23:00:14,277 - lava_callback - INFO - Starting async job submit for node 69acadeb46f7ae8ff3367569 2026-03-09 23:00:14,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:00:14,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acadeb46f7ae8ff3367569 HTTP/1.1" 200 None 2026-03-09 23:00:14,293 - lava_callback - DEBUG - Processing node 69acadeb46f7ae8ff3367569: baseline-arm-clabbe 2026-03-09 23:00:14,293 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acadeb46f7ae8ff3367569 2026-03-09 23:00:14,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:00:14,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acadeb46f7ae8ff3367569 HTTP/1.1" 200 None 2026-03-09 23:00:14,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:00:14,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 23:00:14,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:00:14,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acadeb46f7ae8ff3367569 HTTP/1.1" 200 None 2026-03-09 23:00:14,357 - lava_callback - ERROR - Error processing callback for node 69acadeb46f7ae8ff3367569: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45590 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:06:34,318 - lava_callback - INFO - Received callback for node 69acaf7546f7ae8ff336760d INFO: 172.21.0.1:54020 - "POST /node/69acaf7546f7ae8ff336760d HTTP/1.0" 202 Accepted 2026-03-09 23:06:34,323 - lava_callback - INFO - Starting async job submit for node 69acaf7546f7ae8ff336760d 2026-03-09 23:06:34,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:06:34,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acaf7546f7ae8ff336760d HTTP/1.1" 200 None 2026-03-09 23:06:34,351 - lava_callback - DEBUG - Processing node 69acaf7546f7ae8ff336760d: baseline-arm-clabbe 2026-03-09 23:06:34,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acaf7546f7ae8ff336760d 2026-03-09 23:06:34,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:06:34,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acaf7546f7ae8ff336760d HTTP/1.1" 200 None 2026-03-09 23:06:34,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:06:34,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 23:06:34,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:06:34,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acaf7546f7ae8ff336760d HTTP/1.1" 200 None 2026-03-09 23:06:34,432 - lava_callback - ERROR - Error processing callback for node 69acaf7546f7ae8ff336760d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40722 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:14:05,722 - lava_callback - INFO - Received callback for node 69acb12c46f7ae8ff33676ab INFO: 172.21.0.1:40726 - "POST /node/69acb12c46f7ae8ff33676ab HTTP/1.0" 202 Accepted 2026-03-09 23:14:05,728 - lava_callback - INFO - Starting async job submit for node 69acb12c46f7ae8ff33676ab 2026-03-09 23:14:05,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:14:05,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb12c46f7ae8ff33676ab HTTP/1.1" 200 None 2026-03-09 23:14:05,748 - lava_callback - DEBUG - Processing node 69acb12c46f7ae8ff33676ab: baseline-arm64-baylibre 2026-03-09 23:14:05,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb12c46f7ae8ff33676ab 2026-03-09 23:14:05,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:14:05,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb12c46f7ae8ff33676ab HTTP/1.1" 200 None 2026-03-09 23:14:05,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:14:05,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 23:14:05,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:14:05,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb12c46f7ae8ff33676ab HTTP/1.1" 200 None 2026-03-09 23:14:05,804 - lava_callback - ERROR - Error processing callback for node 69acb12c46f7ae8ff33676ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45452 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:15:34,633 - lava_callback - INFO - Received callback for node 69acb18046f7ae8ff33676ca INFO: 172.21.0.1:51168 - "POST /node/69acb18046f7ae8ff33676ca HTTP/1.0" 202 Accepted 2026-03-09 23:15:34,639 - lava_callback - INFO - Starting async job submit for node 69acb18046f7ae8ff33676ca 2026-03-09 23:15:34,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:15:34,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb18046f7ae8ff33676ca HTTP/1.1" 200 None 2026-03-09 23:15:34,655 - lava_callback - DEBUG - Processing node 69acb18046f7ae8ff33676ca: baseline-arm64-clabbe 2026-03-09 23:15:34,655 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb18046f7ae8ff33676ca 2026-03-09 23:15:34,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:15:34,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb18046f7ae8ff33676ca HTTP/1.1" 200 None 2026-03-09 23:15:34,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:15:34,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 23:15:34,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:15:34,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb18046f7ae8ff33676ca HTTP/1.1" 200 None 2026-03-09 23:15:34,714 - lava_callback - ERROR - Error processing callback for node 69acb18046f7ae8ff33676ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43878 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:16:54,660 - lava_callback - INFO - Received callback for node 69acb1d146f7ae8ff33676ee INFO: 172.21.0.1:34778 - "POST /node/69acb1d146f7ae8ff33676ee HTTP/1.0" 202 Accepted 2026-03-09 23:16:54,666 - lava_callback - INFO - Starting async job submit for node 69acb1d146f7ae8ff33676ee 2026-03-09 23:16:54,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:16:54,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb1d146f7ae8ff33676ee HTTP/1.1" 200 None 2026-03-09 23:16:54,683 - lava_callback - DEBUG - Processing node 69acb1d146f7ae8ff33676ee: baseline-arm64-clabbe 2026-03-09 23:16:54,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb1d146f7ae8ff33676ee 2026-03-09 23:16:54,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:16:54,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb1d146f7ae8ff33676ee HTTP/1.1" 200 None 2026-03-09 23:16:54,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:16:54,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ac4f7446f7ae8ff3364c45 HTTP/1.1" 200 None 2026-03-09 23:16:54,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:16:54,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb1d146f7ae8ff33676ee HTTP/1.1" 200 None 2026-03-09 23:16:54,741 - lava_callback - ERROR - Error processing callback for node 69acb1d146f7ae8ff33676ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43350 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:17:54,709 - lava_callback - INFO - Received callback for node 69acb21246f7ae8ff336770a INFO: 172.21.0.1:40012 - "POST /node/69acb21246f7ae8ff336770a HTTP/1.0" 202 Accepted 2026-03-09 23:17:54,714 - lava_callback - INFO - Starting async job submit for node 69acb21246f7ae8ff336770a 2026-03-09 23:17:54,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:17:54,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb21246f7ae8ff336770a HTTP/1.1" 200 None 2026-03-09 23:17:54,738 - lava_callback - DEBUG - Processing node 69acb21246f7ae8ff336770a: baseline-arm-clabbe 2026-03-09 23:17:54,738 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb21246f7ae8ff336770a 2026-03-09 23:17:54,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:17:54,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb21246f7ae8ff336770a HTTP/1.1" 200 None 2026-03-09 23:17:54,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:17:54,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 23:17:54,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:17:54,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb21246f7ae8ff336770a HTTP/1.1" 200 None 2026-03-09 23:17:54,815 - lava_callback - ERROR - Error processing callback for node 69acb21246f7ae8ff336770a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52430 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:18:34,681 - lava_callback - INFO - Received callback for node 69acb23646f7ae8ff336771c INFO: 172.21.0.1:57624 - "POST /node/69acb23646f7ae8ff336771c HTTP/1.0" 202 Accepted 2026-03-09 23:18:34,686 - lava_callback - INFO - Starting async job submit for node 69acb23646f7ae8ff336771c 2026-03-09 23:18:34,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:18:34,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb23646f7ae8ff336771c HTTP/1.1" 200 None 2026-03-09 23:18:34,702 - lava_callback - DEBUG - Processing node 69acb23646f7ae8ff336771c: baseline-arm-clabbe 2026-03-09 23:18:34,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb23646f7ae8ff336771c 2026-03-09 23:18:34,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:18:34,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb23646f7ae8ff336771c HTTP/1.1" 200 None 2026-03-09 23:18:34,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:18:34,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acd99c5f93e4f05eb2ac HTTP/1.1" 200 None 2026-03-09 23:18:34,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:18:34,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb23646f7ae8ff336771c HTTP/1.1" 200 None 2026-03-09 23:18:34,797 - lava_callback - ERROR - Error processing callback for node 69acb23646f7ae8ff336771c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39344 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:23:06,017 - lava_callback - INFO - Received callback for node 69acb35446f7ae8ff3367793 INFO: 172.21.0.1:39352 - "POST /node/69acb35446f7ae8ff3367793 HTTP/1.0" 202 Accepted 2026-03-09 23:23:06,023 - lava_callback - INFO - Starting async job submit for node 69acb35446f7ae8ff3367793 2026-03-09 23:23:06,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:23:06,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb35446f7ae8ff3367793 HTTP/1.1" 200 None 2026-03-09 23:23:06,039 - lava_callback - DEBUG - Processing node 69acb35446f7ae8ff3367793: baseline-arm64-baylibre 2026-03-09 23:23:06,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb35446f7ae8ff3367793 2026-03-09 23:23:06,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:23:06,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb35446f7ae8ff3367793 HTTP/1.1" 200 None 2026-03-09 23:23:06,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:23:06,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 23:23:06,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:23:06,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb35446f7ae8ff3367793 HTTP/1.1" 200 None 2026-03-09 23:23:06,127 - lava_callback - ERROR - Error processing callback for node 69acb35446f7ae8ff3367793: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39114 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:28:35,213 - lava_callback - INFO - Received callback for node 69acb49a46f7ae8ff33677e2 INFO: 172.21.0.1:39592 - "POST /node/69acb49a46f7ae8ff33677e2 HTTP/1.0" 202 Accepted 2026-03-09 23:28:35,218 - lava_callback - INFO - Starting async job submit for node 69acb49a46f7ae8ff33677e2 2026-03-09 23:28:35,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:28:35,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb49a46f7ae8ff33677e2 HTTP/1.1" 200 None 2026-03-09 23:28:35,235 - lava_callback - DEBUG - Processing node 69acb49a46f7ae8ff33677e2: baseline-arm64-clabbe 2026-03-09 23:28:35,236 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb49a46f7ae8ff33677e2 2026-03-09 23:28:35,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:28:35,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb49a46f7ae8ff33677e2 HTTP/1.1" 200 None 2026-03-09 23:28:35,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:28:35,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 23:28:35,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:28:35,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb49a46f7ae8ff33677e2 HTTP/1.1" 200 None 2026-03-09 23:28:35,300 - lava_callback - ERROR - Error processing callback for node 69acb49a46f7ae8ff33677e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60058 - "GET / HTTP/1.0" 200 OK 2026-03-09 23:29:55,299 - lava_callback - INFO - Received callback for node 69acb4ed46f7ae8ff3367801 INFO: 172.21.0.1:35084 - "POST /node/69acb4ed46f7ae8ff3367801 HTTP/1.0" 202 Accepted 2026-03-09 23:29:55,305 - lava_callback - INFO - Starting async job submit for node 69acb4ed46f7ae8ff3367801 2026-03-09 23:29:55,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:29:55,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb4ed46f7ae8ff3367801 HTTP/1.1" 200 None 2026-03-09 23:29:55,321 - lava_callback - DEBUG - Processing node 69acb4ed46f7ae8ff3367801: baseline-arm64-clabbe 2026-03-09 23:29:55,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69acb4ed46f7ae8ff3367801 2026-03-09 23:29:55,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:29:55,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acb4ed46f7ae8ff3367801 HTTP/1.1" 200 None 2026-03-09 23:29:55,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:29:55,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a9acf29c5f93e4f05eb2cc HTTP/1.1" 200 None 2026-03-09 23:29:55,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-09 23:29:55,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69acb4ed46f7ae8ff3367801 HTTP/1.1" 200 None 2026-03-09 23:29:55,403 - lava_callback - ERROR - Error processing callback for node 69acb4ed46f7ae8ff3367801: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34034 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]